53 KiB
Hold Discovery
This document describes how AppView discovers available holds and presents them to users for selection.
TL;DR
Problem: Users currently enter hold URLs manually in a text field. They don't know what holds exist or which ones they can access.
Solution:
- Subscribe to Jetstream for
io.atcr.hold.captainandio.atcr.hold.crewcollections - Cache discovered holds and crew memberships in SQLite
- Replace the text input with a dropdown showing available holds grouped by access level
Key Changes:
- New table:
hold_crew_members(hold_did, member_did, rkey, permissions, ...) - Jetstream collections:
io.atcr.hold.captain,io.atcr.hold.crew - Settings UI: Text input →
<select>dropdown with optgroups - Form field:
hold_endpoint(URL) →hold_did(DID)
Hold Categories in Dropdown:
| Group | Who Can Use |
|---|---|
| Your Holds | User is captain (owner) |
| Crew Member | User has explicit crew record |
| Open Registration | allowAllCrew=true |
| Public Holds | public=true |
Overview
Users need to select a "default hold" for blob storage. The AppView must discover available holds and determine which ones each user can access. This enables a dropdown in user settings showing:
- Holds the user owns (captain)
- Holds where the user is a crew member
- Holds that allow all crew members (open registration)
- Public holds (anyone can read/write)
Architecture
Discovery Sources
Hold discovery leverages the ATProto network infrastructure:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Hold Service │────▶│ Relay │────▶│ Jetstream │
│ (embedded PDS) │ │ (BGS/bigsky) │ │ │
└─────────────────┘ └─────────────────┘ └────────┬────────┘
│
▼
┌─────────────────┐
│ AppView │
│ (subscriber) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ SQLite │
│ (cache) │
└─────────────────┘
- Hold services run embedded PDSes that store captain and crew records
- Relays crawl hold PDSes after
request-crawl.shis run - Jetstream streams record events filtered by collection
- AppView subscribes to Jetstream and caches records in SQLite
Record Types
Two ATProto record collections are relevant for discovery:
io.atcr.hold.captain
Singleton record (rkey: self) in each hold's embedded PDS describing the hold:
{
"$type": "io.atcr.hold.captain",
"ownerDid": "did:plc:abc123",
"public": false,
"allowAllCrew": true,
"deployedAt": "2025-01-07T12:00:00Z",
"region": "us-east-1",
"provider": "fly.io"
}
| Field | Type | Description |
|---|---|---|
ownerDid |
string | DID of the hold owner (captain) |
public |
boolean | If true, anyone can read and write blobs |
allowAllCrew |
boolean | If true, any authenticated user can self-register as crew |
deployedAt |
string | ISO 8601 timestamp of deployment |
region |
string | Optional geographic region identifier |
provider |
string | Optional hosting provider name |
io.atcr.hold.crew
One record per crew member in the hold's embedded PDS:
{
"$type": "io.atcr.hold.crew",
"memberDid": "did:plc:xyz789",
"role": "contributor",
"permissions": ["blob:read", "blob:write"],
"tier": "standard",
"addedAt": "2025-01-07T12:00:00Z"
}
| Field | Type | Description |
|---|---|---|
memberDid |
string | DID of the crew member |
role |
string | Human-readable role name |
permissions |
string[] | Permission grants: blob:read, blob:write, crew:admin |
tier |
string | Optional tier for quota management |
addedAt |
string | ISO 8601 timestamp when added |
Record key derivation: Crew records use a deterministic rkey based on the member's DID:
func CrewRecordKey(memberDID string) string {
hash := sha256.Sum256([]byte(memberDID))
return base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(hash[:16])
}
This enables O(1) lookup of a specific member's crew record.
Data Model
Database Schema
Add to pkg/appview/db/schema.sql:
-- Cached hold captain records from Jetstream
-- Primary discovery source for available holds
CREATE TABLE IF NOT EXISTS hold_captain_records (
did TEXT PRIMARY KEY, -- Hold's DID (did:web:hold01.atcr.io)
owner_did TEXT NOT NULL, -- Captain's DID
public INTEGER NOT NULL DEFAULT 0, -- 1 if public hold
allow_all_crew INTEGER NOT NULL DEFAULT 0, -- 1 if open registration
deployed_at TEXT, -- ISO 8601 deployment timestamp
region TEXT, -- Geographic region
provider TEXT, -- Hosting provider
endpoint TEXT, -- Resolved HTTP endpoint (cached)
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_hold_captain_owner ON hold_captain_records(owner_did);
CREATE INDEX IF NOT EXISTS idx_hold_captain_public ON hold_captain_records(public);
CREATE INDEX IF NOT EXISTS idx_hold_captain_allow_all ON hold_captain_records(allow_all_crew);
-- Cached hold crew memberships from Jetstream
-- Enables reverse lookup: "which holds is user X a member of?"
CREATE TABLE IF NOT EXISTS hold_crew_members (
hold_did TEXT NOT NULL, -- Hold's DID
member_did TEXT NOT NULL, -- Crew member's DID
rkey TEXT NOT NULL, -- ATProto record key (for delete handling)
role TEXT, -- Human-readable role
permissions TEXT, -- JSON array of permissions
tier TEXT, -- Optional quota tier
added_at TEXT, -- ISO 8601 timestamp
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (hold_did, member_did),
FOREIGN KEY (hold_did) REFERENCES hold_captain_records(did) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_hold_crew_member ON hold_crew_members(member_did);
CREATE INDEX IF NOT EXISTS idx_hold_crew_hold ON hold_crew_members(hold_did);
CREATE INDEX IF NOT EXISTS idx_hold_crew_rkey ON hold_crew_members(hold_did, rkey);
Migration
Add to pkg/appview/db/migrations/:
# 006_hold_discovery.yaml
id: 006_hold_discovery
description: Add hold crew members table for discovery
up: |
CREATE TABLE IF NOT EXISTS hold_crew_members (
hold_did TEXT NOT NULL,
member_did TEXT NOT NULL,
rkey TEXT NOT NULL,
role TEXT,
permissions TEXT,
tier TEXT,
added_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (hold_did, member_did),
FOREIGN KEY (hold_did) REFERENCES hold_captain_records(did) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_hold_crew_member ON hold_crew_members(member_did);
CREATE INDEX IF NOT EXISTS idx_hold_crew_hold ON hold_crew_members(hold_did);
CREATE INDEX IF NOT EXISTS idx_hold_crew_rkey ON hold_crew_members(hold_did, rkey);
down: |
DROP INDEX IF EXISTS idx_hold_crew_rkey;
DROP INDEX IF EXISTS idx_hold_crew_hold;
DROP INDEX IF EXISTS idx_hold_crew_member;
DROP TABLE IF EXISTS hold_crew_members;
Jetstream Integration
Subscription Configuration
Update the Jetstream worker to subscribe to hold collections:
// pkg/appview/jetstream/worker.go
var wantedCollections = []string{
"io.atcr.manifest",
"io.atcr.tag",
"io.atcr.hold.stats",
"io.atcr.hold.captain", // NEW: Hold discovery
"io.atcr.hold.crew", // NEW: Crew membership discovery
}
Event Processing
Add processors for captain and crew records:
// pkg/appview/jetstream/processor.go
func (p *Processor) ProcessEvent(evt *Event) error {
switch evt.Collection {
case "io.atcr.manifest":
return p.ProcessManifest(evt)
case "io.atcr.tag":
return p.ProcessTag(evt)
case "io.atcr.hold.stats":
return p.ProcessStats(evt)
case "io.atcr.hold.captain":
return p.ProcessCaptain(evt)
case "io.atcr.hold.crew":
return p.ProcessCrew(evt)
default:
return nil
}
}
func (p *Processor) ProcessCaptain(evt *Event) error {
// The repo DID IS the hold DID (hold's embedded PDS)
holdDID := evt.DID
if evt.Operation == "delete" {
return p.db.DeleteCaptainRecord(holdDID)
}
var record atproto.CaptainRecord
if err := json.Unmarshal(evt.Record, &record); err != nil {
return fmt.Errorf("unmarshal captain record: %w", err)
}
// Resolve hold DID to HTTP endpoint for caching
endpoint, err := p.resolver.ResolveHoldURL(holdDID)
if err != nil {
// Log but don't fail - endpoint can be resolved later
log.Warn().Err(err).Str("did", holdDID).Msg("failed to resolve hold endpoint")
}
// Verify this is actually a hold by checking /.well-known/did.json
// for #atcr_hold service type
if !p.verifyHoldService(holdDID, endpoint) {
log.Debug().Str("did", holdDID).Msg("skipping non-hold captain record")
return nil
}
return p.db.UpsertCaptainRecord(holdDID, &db.CaptainRecord{
DID: holdDID,
OwnerDID: record.OwnerDID,
Public: record.Public,
AllowAllCrew: record.AllowAllCrew,
DeployedAt: record.DeployedAt,
Region: record.Region,
Provider: record.Provider,
Endpoint: endpoint,
})
}
func (p *Processor) ProcessCrew(evt *Event) error {
// The repo DID IS the hold DID (hold's embedded PDS)
holdDID := evt.DID
if evt.Operation == "delete" {
// Need to determine member DID from rkey or record
// For delete events, we may not have the record body
return p.db.DeleteCrewMemberByRkey(holdDID, evt.Rkey)
}
var record atproto.CrewRecord
if err := json.Unmarshal(evt.Record, &record); err != nil {
return fmt.Errorf("unmarshal crew record: %w", err)
}
// Verify the hold exists in our captain records
// If not, this crew record is for an unknown hold - skip it
if _, err := p.db.GetCaptainRecord(holdDID); err != nil {
log.Debug().Str("hold", holdDID).Msg("skipping crew record for unknown hold")
return nil
}
permissionsJSON, _ := json.Marshal(record.Permissions)
return p.db.UpsertCrewMember(holdDID, &db.CrewMember{
HoldDID: holdDID,
MemberDID: record.MemberDID,
Role: record.Role,
Permissions: string(permissionsJSON),
Tier: record.Tier,
AddedAt: record.AddedAt,
})
}
func (p *Processor) verifyHoldService(did, endpoint string) bool {
// Fetch /.well-known/did.json and check for #atcr_hold service
didDoc, err := p.resolver.ResolveDIDDocument(did)
if err != nil {
return false
}
for _, svc := range didDoc.Service {
if svc.ID == did+"#atcr_hold" || svc.Type == "AtcrHold" {
return true
}
}
return false
}
Hold Service Verification
Before caching a captain record, verify the DID document contains the #atcr_hold service:
// pkg/atproto/resolver.go
type DIDDocument struct {
ID string `json:"id"`
Service []Service `json:"service"`
// ... other fields
}
type Service struct {
ID string `json:"id"`
Type string `json:"type"`
ServiceEndpoint string `json:"serviceEndpoint"`
}
func (r *Resolver) HasHoldService(did string) (bool, string, error) {
doc, err := r.ResolveDIDDocument(did)
if err != nil {
return false, "", err
}
for _, svc := range doc.Service {
// Check for #atcr_hold fragment or AtcrHold type
if strings.HasSuffix(svc.ID, "#atcr_hold") || svc.Type == "AtcrHold" {
return true, svc.ServiceEndpoint, nil
}
}
return false, "", nil
}
Backfill Strategy
Initial Backfill
For holds that existed before AppView started listening to Jetstream, use the existing backfill mechanism:
// pkg/appview/jetstream/backfill.go
func (b *Backfiller) BackfillHolds(ctx context.Context) error {
// List all repos from relay that have io.atcr.hold.captain collection
repos, err := b.listReposWithCollection(ctx, "io.atcr.hold.captain")
if err != nil {
return err
}
for _, repo := range repos {
// Fetch captain record
captain, err := b.fetchRecord(ctx, repo.DID, "io.atcr.hold.captain", "self")
if err != nil {
log.Warn().Err(err).Str("did", repo.DID).Msg("failed to fetch captain record")
continue
}
// Verify it's a hold service
hasService, endpoint, _ := b.resolver.HasHoldService(repo.DID)
if !hasService {
continue
}
// Upsert captain record
if err := b.db.UpsertCaptainRecord(repo.DID, captain); err != nil {
log.Warn().Err(err).Str("did", repo.DID).Msg("failed to upsert captain record")
continue
}
// Fetch and upsert all crew records for this hold
if err := b.backfillCrewRecords(ctx, repo.DID); err != nil {
log.Warn().Err(err).Str("did", repo.DID).Msg("failed to backfill crew records")
}
}
return nil
}
func (b *Backfiller) backfillCrewRecords(ctx context.Context, holdDID string) error {
// List all records in io.atcr.hold.crew collection
records, err := b.listRecords(ctx, holdDID, "io.atcr.hold.crew")
if err != nil {
return err
}
for _, record := range records {
var crew atproto.CrewRecord
if err := json.Unmarshal(record.Value, &crew); err != nil {
continue
}
permissionsJSON, _ := json.Marshal(crew.Permissions)
if err := b.db.UpsertCrewMember(holdDID, &db.CrewMember{
HoldDID: holdDID,
MemberDID: crew.MemberDID,
Role: crew.Role,
Permissions: string(permissionsJSON),
Tier: crew.Tier,
AddedAt: crew.AddedAt,
}); err != nil {
log.Warn().Err(err).Msg("failed to upsert crew member")
}
}
return nil
}
Listing Repos by Collection
Query the relay for repos that have a specific collection:
func (b *Backfiller) listReposWithCollection(ctx context.Context, collection string) ([]Repo, error) {
// Use com.atproto.sync.listRepos to get all repos
// Then filter to those with the target collection
//
// Note: This is O(n) over all repos on the relay.
// For efficiency, could maintain a separate index or use
// Jetstream historical replay if available.
var repos []Repo
cursor := ""
for {
resp, err := b.client.SyncListRepos(ctx, cursor, 1000)
if err != nil {
return nil, err
}
for _, repo := range resp.Repos {
// Check if repo has the collection by attempting to list records
records, err := b.client.RepoListRecords(ctx, repo.DID, collection, "", 1)
if err == nil && len(records.Records) > 0 {
repos = append(repos, Repo{DID: repo.DID})
}
}
if resp.Cursor == nil || *resp.Cursor == "" {
break
}
cursor = *resp.Cursor
}
return repos, nil
}
Bootstrap Configuration
For known holds that may not yet be on relays, support a bootstrap list in configuration:
# Environment variable
ATCR_BOOTSTRAP_HOLDS="did:web:hold01.atcr.io,did:web:hold02.atcr.io"
func (b *Backfiller) BackfillBootstrapHolds(ctx context.Context, holdDIDs []string) error {
for _, did := range holdDIDs {
// Verify it's a hold
hasService, endpoint, err := b.resolver.HasHoldService(did)
if err != nil || !hasService {
log.Warn().Str("did", did).Msg("bootstrap hold is not a valid hold service")
continue
}
// Fetch captain record directly from hold's PDS
captain, err := b.fetchCaptainFromHold(ctx, did, endpoint)
if err != nil {
log.Warn().Err(err).Str("did", did).Msg("failed to fetch captain from hold")
continue
}
if err := b.db.UpsertCaptainRecord(did, captain); err != nil {
log.Warn().Err(err).Str("did", did).Msg("failed to upsert bootstrap captain")
continue
}
// Also backfill crew records
if err := b.backfillCrewFromHold(ctx, did, endpoint); err != nil {
log.Warn().Err(err).Str("did", did).Msg("failed to backfill bootstrap crew")
}
}
return nil
}
func (b *Backfiller) fetchCaptainFromHold(ctx context.Context, did, endpoint string) (*db.CaptainRecord, error) {
// GET {endpoint}/xrpc/com.atproto.repo.getRecord?repo={did}&collection=io.atcr.hold.captain&rkey=self
url := fmt.Sprintf("%s/xrpc/com.atproto.repo.getRecord?repo=%s&collection=io.atcr.hold.captain&rkey=self",
endpoint, did)
resp, err := http.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
var result struct {
Value atproto.CaptainRecord `json:"value"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return nil, err
}
return &db.CaptainRecord{
DID: did,
OwnerDID: result.Value.OwnerDID,
Public: result.Value.Public,
AllowAllCrew: result.Value.AllowAllCrew,
DeployedAt: result.Value.DeployedAt,
Region: result.Value.Region,
Provider: result.Value.Provider,
Endpoint: endpoint,
}, nil
}
Database Queries
Hold Store Functions
Add to pkg/appview/db/hold_store.go:
// CrewMember represents a cached crew membership
type CrewMember struct {
HoldDID string
MemberDID string
Role string
Permissions string // JSON array
Tier string
AddedAt string
CreatedAt string
UpdatedAt string
}
// UpsertCrewMember inserts or updates a crew member record
func UpsertCrewMember(db *sql.DB, holdDID string, member *CrewMember) error {
_, err := db.Exec(`
INSERT INTO hold_crew_members (hold_did, member_did, role, permissions, tier, added_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, datetime('now'))
ON CONFLICT(hold_did, member_did) DO UPDATE SET
role = excluded.role,
permissions = excluded.permissions,
tier = excluded.tier,
added_at = excluded.added_at,
updated_at = datetime('now')
`, holdDID, member.MemberDID, member.Role, member.Permissions, member.Tier, member.AddedAt)
return err
}
// DeleteCrewMember removes a crew member record
func DeleteCrewMember(db *sql.DB, holdDID, memberDID string) error {
_, err := db.Exec(`
DELETE FROM hold_crew_members WHERE hold_did = ? AND member_did = ?
`, holdDID, memberDID)
return err
}
// DeleteCrewMemberByRkey removes a crew member by rkey (for delete events)
func DeleteCrewMemberByRkey(db *sql.DB, holdDID, rkey string) error {
// We need to find the member by rkey hash
// This is tricky because we store member_did, not rkey
// Option 1: Store rkey in the table
// Option 2: Iterate and check (slow)
// Option 3: Store both member_did and rkey
// For now, we'll need to add rkey to the schema
_, err := db.Exec(`
DELETE FROM hold_crew_members WHERE hold_did = ? AND rkey = ?
`, holdDID, rkey)
return err
}
// AvailableHold represents a hold available to a user
type AvailableHold struct {
DID string
OwnerDID string
Public bool
AllowAllCrew bool
Region string
Provider string
Endpoint string
Membership string // "owner", "crew", "eligible", "public"
Permissions []string // nil if not crew
}
// GetAvailableHolds returns all holds available to a user
func GetAvailableHolds(db *sql.DB, userDID string) ([]AvailableHold, error) {
rows, err := db.Query(`
SELECT
h.did,
h.owner_did,
h.public,
h.allow_all_crew,
h.region,
h.provider,
h.endpoint,
CASE
WHEN h.owner_did = ?1 THEN 'owner'
WHEN c.member_did IS NOT NULL THEN 'crew'
WHEN h.allow_all_crew = 1 THEN 'eligible'
WHEN h.public = 1 THEN 'public'
ELSE 'none'
END as membership,
c.permissions
FROM hold_captain_records h
LEFT JOIN hold_crew_members c
ON h.did = c.hold_did AND c.member_did = ?1
WHERE h.public = 1
OR h.allow_all_crew = 1
OR h.owner_did = ?1
OR c.member_did IS NOT NULL
ORDER BY
CASE
WHEN h.owner_did = ?1 THEN 0
WHEN c.member_did IS NOT NULL THEN 1
WHEN h.allow_all_crew = 1 THEN 2
ELSE 3
END,
h.did
`, userDID)
if err != nil {
return nil, err
}
defer rows.Close()
var holds []AvailableHold
for rows.Next() {
var h AvailableHold
var permissionsJSON sql.NullString
err := rows.Scan(
&h.DID,
&h.OwnerDID,
&h.Public,
&h.AllowAllCrew,
&h.Region,
&h.Provider,
&h.Endpoint,
&h.Membership,
&permissionsJSON,
)
if err != nil {
return nil, err
}
if permissionsJSON.Valid {
json.Unmarshal([]byte(permissionsJSON.String), &h.Permissions)
}
holds = append(holds, h)
}
return holds, rows.Err()
}
// GetHoldsOwnedBy returns holds owned by a specific DID
func GetHoldsOwnedBy(db *sql.DB, ownerDID string) ([]CaptainRecord, error) {
rows, err := db.Query(`
SELECT did, owner_did, public, allow_all_crew, deployed_at, region, provider, endpoint
FROM hold_captain_records
WHERE owner_did = ?
ORDER BY deployed_at DESC
`, ownerDID)
if err != nil {
return nil, err
}
defer rows.Close()
var holds []CaptainRecord
for rows.Next() {
var h CaptainRecord
err := rows.Scan(&h.DID, &h.OwnerDID, &h.Public, &h.AllowAllCrew,
&h.DeployedAt, &h.Region, &h.Provider, &h.Endpoint)
if err != nil {
return nil, err
}
holds = append(holds, h)
}
return holds, rows.Err()
}
// GetCrewMemberships returns all holds where a user is a crew member
func GetCrewMemberships(db *sql.DB, memberDID string) ([]CrewMember, error) {
rows, err := db.Query(`
SELECT hold_did, member_did, role, permissions, tier, added_at
FROM hold_crew_members
WHERE member_did = ?
ORDER BY added_at DESC
`, memberDID)
if err != nil {
return nil, err
}
defer rows.Close()
var memberships []CrewMember
for rows.Next() {
var m CrewMember
err := rows.Scan(&m.HoldDID, &m.MemberDID, &m.Role, &m.Permissions, &m.Tier, &m.AddedAt)
if err != nil {
return nil, err
}
memberships = append(memberships, m)
}
return memberships, rows.Err()
}
UI Integration
Current State
The settings page (pkg/appview/templates/pages/settings.html) currently has a text input field for the default hold:
<!-- Current implementation (to be replaced) -->
<section class="settings-section">
<h2>Default Hold</h2>
<p>Current: <strong id="current-hold">{{ if .Profile.DefaultHold }}{{ .Profile.DefaultHold }}{{ else }}Not set{{ end }}</strong></p>
<form hx-post="/api/profile/default-hold" ...>
<div class="form-group">
<label for="hold-endpoint">Hold Endpoint:</label>
<input type="text"
id="hold-endpoint"
name="hold_endpoint"
value="{{ .Profile.DefaultHold }}"
placeholder="https://hold.example.com" />
<small>Leave empty to use AppView default storage</small>
</div>
<button type="submit" class="btn-primary">Save</button>
</form>
</section>
Problems with the current approach:
- Users must know hold URLs - Requires users to manually find and copy hold endpoint URLs
- No validation - Users can enter invalid or inaccessible URLs
- No discovery - Users don't know what holds are available to them
- Poor UX - Text input is error-prone and unfriendly
- No membership visibility - Users can't see which holds they're crew on
Proposed Change: Dropdown with Discovered Holds
Replace the text input with a <select> dropdown populated from the hold discovery cache:
<!-- New implementation -->
<section class="settings-section">
<h2>Default Hold</h2>
<p class="help-text">
Select where your container images will be stored. Holds are organized by your access level.
</p>
<form hx-post="/api/profile/default-hold"
hx-target="#hold-status"
hx-swap="innerHTML"
id="hold-form">
<div class="form-group">
<label for="default-hold">Storage Hold:</label>
<select id="default-hold" name="hold_did" class="form-select">
<option value="">AppView Default ({{ .DefaultHoldDisplayName }})</option>
{{if .OwnedHolds}}
<optgroup label="Your Holds">
{{range .OwnedHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} ({{.Region}}){{end}}
</option>
{{end}}
</optgroup>
{{end}}
{{if .CrewHolds}}
<optgroup label="Crew Member">
{{range .CrewHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} ({{.Region}}){{end}}
{{if not .HasWritePermission}}[read-only]{{end}}
</option>
{{end}}
</optgroup>
{{end}}
{{if .EligibleHolds}}
<optgroup label="Open Registration">
{{range .EligibleHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} ({{.Region}}){{end}}
</option>
{{end}}
</optgroup>
{{end}}
{{if .PublicHolds}}
<optgroup label="Public Holds">
{{range .PublicHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} ({{.Region}}){{end}}
</option>
{{end}}
</optgroup>
{{end}}
</select>
<small>Your images will be stored on the selected hold</small>
</div>
<button type="submit" class="btn-primary">Save</button>
</form>
<div id="hold-status"></div>
<!-- Hold details panel (shows when hold selected) -->
<div id="hold-details" class="hold-details" style="display: none;">
<h3>Hold Details</h3>
<dl>
<dt>DID:</dt>
<dd id="hold-did"></dd>
<dt>Provider:</dt>
<dd id="hold-provider"></dd>
<dt>Region:</dt>
<dd id="hold-region"></dd>
<dt>Your Access:</dt>
<dd id="hold-access"></dd>
</dl>
</div>
</section>
Dropdown Option Groups
The dropdown organizes holds into logical groups based on user's relationship:
| Group | Description | Access Level |
|---|---|---|
| Your Holds | Holds where user is the captain (owner) | Full control |
| Crew Member | Holds where user has explicit crew membership | Based on permissions |
| Open Registration | Holds with allowAllCrew=true |
Can self-register |
| Public Holds | Holds with public=true |
Anyone can use |
Visual Indicators
Each option should show relevant context:
┌─ Storage Hold: ─────────────────────────────────────┐
│ ▼ hold01.atcr.io (us-east) │
├─────────────────────────────────────────────────────┤
│ AppView Default (hold01.atcr.io) │
│ ───────────────────────────────────── │
│ Your Holds │
│ my-hold.fly.dev (us-west) │
│ ───────────────────────────────────── │
│ Crew Member │
│ team-hold.company.com (eu-central) │
│ shared-hold.org (asia-pacific) [read-only] │
│ ───────────────────────────────────── │
│ Open Registration │
│ community-hold.dev (us-east) │
│ ───────────────────────────────────── │
│ Public Holds │
│ public-hold.example.com (global) │
└─────────────────────────────────────────────────────┘
Form Submission Change
The form now submits hold_did (a DID) instead of hold_endpoint (a URL):
Before:
POST /api/profile/default-hold
Content-Type: application/x-www-form-urlencoded
hold_endpoint=https://hold01.atcr.io
After:
POST /api/profile/default-hold
Content-Type: application/x-www-form-urlencoded
hold_did=did:web:hold01.atcr.io
The UpdateDefaultHoldHandler needs to be updated to accept DIDs:
// pkg/appview/handlers/settings.go
func (h *UpdateDefaultHoldHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
user := middleware.GetUser(r)
if user == nil {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
// Accept DID (new) or endpoint (legacy/fallback)
holdDID := r.FormValue("hold_did")
if holdDID == "" {
// Fallback for legacy form submissions
holdDID = r.FormValue("hold_endpoint")
}
// Validate the hold DID if provided
if holdDID != "" {
// Check it's in our discovered holds cache
captain, err := h.DB.GetCaptainRecord(holdDID)
if err != nil {
http.Error(w, "Unknown hold: "+holdDID, http.StatusBadRequest)
return
}
// Verify user has access to this hold
available, err := db.GetAvailableHolds(h.DB, user.DID)
if err != nil {
http.Error(w, "Failed to check hold access", http.StatusInternalServerError)
return
}
hasAccess := false
for _, h := range available {
if h.DID == holdDID {
hasAccess = true
break
}
}
if !hasAccess {
http.Error(w, "You don't have access to this hold", http.StatusForbidden)
return
}
}
// ... rest of profile update logic
}
Settings Handler
Update the settings handler to include available holds:
// pkg/appview/handlers/settings.go
func (h *Handler) SettingsPage(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
userDID := auth.GetDID(ctx)
// Get user's current profile
profile, err := h.storage.GetProfile(ctx, userDID)
if err != nil {
// Handle error
}
// Get available holds for dropdown
availableHolds, err := db.GetAvailableHolds(h.db, userDID)
if err != nil {
// Handle error
}
data := SettingsPageData{
Profile: profile,
AvailableHolds: availableHolds,
CurrentHoldDID: profile.DefaultHold,
}
h.renderTemplate(w, "settings.html", data)
}
Settings Template
<!-- pkg/appview/templates/pages/settings.html -->
<div class="settings-section">
<h2>Default Hold</h2>
<p class="help-text">
Select where your container images will be stored by default.
</p>
<form method="POST" action="/settings/hold">
<select name="defaultHold" id="defaultHold" class="form-select">
<option value="">-- Select a Hold --</option>
{{if .OwnedHolds}}
<optgroup label="Your Holds">
{{range .OwnedHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}} (Owner)
{{if .Region}} - {{.Region}}{{end}}
</option>
{{end}}
</optgroup>
{{end}}
{{if .CrewHolds}}
<optgroup label="Crew Member">
{{range .CrewHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} - {{.Region}}{{end}}
</option>
{{end}}
</optgroup>
{{end}}
{{if .EligibleHolds}}
<optgroup label="Open Registration">
{{range .EligibleHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} - {{.Region}}{{end}}
</option>
{{end}}
</optgroup>
{{end}}
{{if .PublicHolds}}
<optgroup label="Public Holds">
{{range .PublicHolds}}
<option value="{{.DID}}" {{if eq $.CurrentHoldDID .DID}}selected{{end}}>
{{.DisplayName}}
{{if .Region}} - {{.Region}}{{end}}
</option>
{{end}}
</optgroup>
{{end}}
</select>
<button type="submit" class="btn btn-primary">Save</button>
</form>
</div>
Template Data Preparation
// pkg/appview/handlers/settings.go
type SettingsPageData struct {
Profile *atproto.SailorProfile
CurrentHoldDID string
OwnedHolds []HoldDisplay
CrewHolds []HoldDisplay
EligibleHolds []HoldDisplay
PublicHolds []HoldDisplay
}
type HoldDisplay struct {
DID string
DisplayName string // Derived from DID or endpoint
Region string
Provider string
Permissions []string
}
func (h *Handler) prepareSettingsData(userDID string, holds []db.AvailableHold, currentHold string) SettingsPageData {
data := SettingsPageData{
CurrentHoldDID: currentHold,
}
for _, hold := range holds {
display := HoldDisplay{
DID: hold.DID,
DisplayName: deriveDisplayName(hold.DID, hold.Endpoint),
Region: hold.Region,
Provider: hold.Provider,
Permissions: hold.Permissions,
}
switch hold.Membership {
case "owner":
data.OwnedHolds = append(data.OwnedHolds, display)
case "crew":
data.CrewHolds = append(data.CrewHolds, display)
case "eligible":
data.EligibleHolds = append(data.EligibleHolds, display)
case "public":
data.PublicHolds = append(data.PublicHolds, display)
}
}
return data
}
func deriveDisplayName(did, endpoint string) string {
// For did:web, extract the domain
if strings.HasPrefix(did, "did:web:") {
return strings.TrimPrefix(did, "did:web:")
}
// For did:plc, use the endpoint hostname if available
if endpoint != "" {
if u, err := url.Parse(endpoint); err == nil {
return u.Host
}
}
// Fallback to truncated DID
if len(did) > 20 {
return did[:20] + "..."
}
return did
}
CSS Styles
Add styles for the hold dropdown and details panel:
/* pkg/appview/templates/pages/settings.html - add to <style> section */
/* Hold Selection Styles */
.form-select {
width: 100%;
padding: 0.75rem;
font-size: 1rem;
border: 1px solid var(--border);
border-radius: 4px;
background: var(--bg);
color: var(--fg);
cursor: pointer;
}
.form-select:focus {
outline: none;
border-color: var(--primary);
box-shadow: 0 0 0 2px var(--primary-bg);
}
.form-select optgroup {
font-weight: bold;
color: var(--fg-muted);
padding-top: 0.5rem;
}
.form-select option {
padding: 0.5rem;
font-weight: normal;
color: var(--fg);
}
/* Hold Details Panel */
.hold-details {
margin-top: 1rem;
padding: 1rem;
background: var(--code-bg);
border-radius: 4px;
border: 1px solid var(--border);
}
.hold-details h3 {
margin-top: 0;
margin-bottom: 0.75rem;
font-size: 0.9rem;
color: var(--fg-muted);
text-transform: uppercase;
letter-spacing: 0.05em;
}
.hold-details dl {
display: grid;
grid-template-columns: auto 1fr;
gap: 0.5rem 1rem;
margin: 0;
}
.hold-details dt {
color: var(--fg-muted);
font-weight: 500;
}
.hold-details dd {
margin: 0;
font-family: monospace;
}
/* Access Level Badges */
.access-badge {
display: inline-block;
padding: 0.125rem 0.5rem;
border-radius: 4px;
font-size: 0.85rem;
font-weight: 500;
}
.access-owner {
background: #fef3c7;
color: #92400e;
}
.access-crew {
background: #dcfce7;
color: #166534;
}
.access-eligible {
background: #e0e7ff;
color: #3730a3;
}
.access-public {
background: #f3f4f6;
color: #374151;
}
/* Read-only indicator */
.read-only-indicator {
color: var(--warning);
font-size: 0.85rem;
margin-left: 0.25rem;
}
JavaScript Interaction
Add JavaScript to show hold details when selection changes:
<!-- Add to settings.html <script> section -->
<script>
(function() {
// Hold selection and details display
const holdSelect = document.getElementById('default-hold');
const holdDetails = document.getElementById('hold-details');
// Hold data embedded from server (JSON in data attribute or inline)
const holdData = {{ .HoldDataJSON }};
if (holdSelect) {
holdSelect.addEventListener('change', function() {
const selectedDID = this.value;
if (!selectedDID || !holdData[selectedDID]) {
holdDetails.style.display = 'none';
return;
}
const hold = holdData[selectedDID];
document.getElementById('hold-did').textContent = hold.did;
document.getElementById('hold-provider').textContent = hold.provider || 'Unknown';
document.getElementById('hold-region').textContent = hold.region || 'Global';
// Set access level with badge
const accessEl = document.getElementById('hold-access');
const accessClass = 'access-' + hold.membership;
const accessLabel = {
'owner': 'Owner (Full Control)',
'crew': 'Crew Member',
'eligible': 'Open Registration',
'public': 'Public Access'
}[hold.membership] || hold.membership;
accessEl.innerHTML = `<span class="access-badge ${accessClass}">${accessLabel}</span>`;
// Show permissions for crew members
if (hold.membership === 'crew' && hold.permissions) {
const perms = hold.permissions.join(', ');
accessEl.innerHTML += `<br><small>Permissions: ${perms}</small>`;
}
holdDetails.style.display = 'block';
});
// Trigger on page load if a hold is already selected
if (holdSelect.value) {
holdSelect.dispatchEvent(new Event('change'));
}
}
})();
</script>
Server-Side Hold Data
The handler needs to serialize hold data for the JavaScript:
// pkg/appview/handlers/settings.go
import "encoding/json"
type HoldDataEntry struct {
DID string `json:"did"`
DisplayName string `json:"displayName"`
Provider string `json:"provider"`
Region string `json:"region"`
Membership string `json:"membership"`
Permissions []string `json:"permissions,omitempty"`
}
func (h *SettingsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// ... existing code ...
// Get available holds
availableHolds, err := db.GetAvailableHolds(h.DB, user.DID)
if err != nil {
slog.Error("Failed to get available holds", "error", err)
availableHolds = []db.AvailableHold{}
}
// Build hold data map for JavaScript
holdDataMap := make(map[string]HoldDataEntry)
for _, hold := range availableHolds {
holdDataMap[hold.DID] = HoldDataEntry{
DID: hold.DID,
DisplayName: deriveDisplayName(hold.DID, hold.Endpoint),
Provider: hold.Provider,
Region: hold.Region,
Membership: hold.Membership,
Permissions: hold.Permissions,
}
}
holdDataJSON, _ := json.Marshal(holdDataMap)
data := SettingsPageData{
// ... existing fields ...
HoldDataJSON: template.JS(holdDataJSON), // Safe for embedding in <script>
}
// ... render template ...
}
Empty State Handling
When no holds are discovered yet, show a helpful message:
{{if and (not .OwnedHolds) (not .CrewHolds) (not .EligibleHolds) (not .PublicHolds)}}
<div class="empty-holds-notice">
<p>
<i data-lucide="info"></i>
No holds discovered yet. Using AppView default storage.
</p>
<p class="help-text">
Holds are discovered automatically via the ATProto network.
If you've deployed your own hold, make sure it has requested a relay crawl.
</p>
</div>
{{else}}
<!-- Show the dropdown -->
{{end}}
Refresh Button
Allow users to manually trigger hold refresh:
<div class="hold-actions">
<button type="button"
class="btn-secondary"
hx-post="/api/holds/refresh"
hx-target="#hold-refresh-status"
hx-swap="innerHTML">
<i data-lucide="refresh-cw"></i> Refresh Holds
</button>
<span id="hold-refresh-status"></span>
</div>
// pkg/appview/handlers/settings.go
func (h *RefreshHoldsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
user := middleware.GetUser(r)
if user == nil {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
// Trigger async refresh of hold cache
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := h.Backfiller.RefreshAllHolds(ctx); err != nil {
slog.Error("Failed to refresh holds", "error", err)
}
}()
w.Header().Set("Content-Type", "text/html")
w.Write([]byte(`<span class="success">Refreshing... reload page in a moment</span>`))
}
Cache Invalidation
Real-time Updates via Jetstream
Jetstream events automatically update the cache:
- Captain record created/updated: Upsert to
hold_captain_records - Captain record deleted: Delete from
hold_captain_records(cascades to crew) - Crew record created/updated: Upsert to
hold_crew_members - Crew record deleted: Delete from
hold_crew_members
Manual Refresh
For cases where Jetstream may be delayed or missed events:
// pkg/appview/handlers/settings.go
func (h *Handler) RefreshHoldCache(w http.ResponseWriter, r *http.Request) {
holdDID := r.URL.Query().Get("did")
if holdDID == "" {
http.Error(w, "missing did parameter", http.StatusBadRequest)
return
}
// Verify it's a hold service
hasService, endpoint, err := h.resolver.HasHoldService(holdDID)
if err != nil || !hasService {
http.Error(w, "invalid hold DID", http.StatusBadRequest)
return
}
// Fetch and update captain record
captain, err := h.backfiller.fetchCaptainFromHold(r.Context(), holdDID, endpoint)
if err != nil {
http.Error(w, "failed to fetch captain record", http.StatusInternalServerError)
return
}
if err := h.db.UpsertCaptainRecord(holdDID, captain); err != nil {
http.Error(w, "failed to update cache", http.StatusInternalServerError)
return
}
// Also refresh crew records
if err := h.backfiller.backfillCrewFromHold(r.Context(), holdDID, endpoint); err != nil {
log.Warn().Err(err).Str("did", holdDID).Msg("failed to refresh crew records")
}
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
TTL-based Refresh
Optionally, run periodic refresh of cached records:
// pkg/appview/jetstream/backfill.go
func (b *Backfiller) RefreshStaleHolds(ctx context.Context, maxAge time.Duration) error {
// Find holds not updated recently
rows, err := b.db.Query(`
SELECT did, endpoint FROM hold_captain_records
WHERE updated_at < datetime('now', ?)
`, fmt.Sprintf("-%d seconds", int(maxAge.Seconds())))
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
var did, endpoint string
if err := rows.Scan(&did, &endpoint); err != nil {
continue
}
// Refresh this hold's data
if err := b.refreshHold(ctx, did, endpoint); err != nil {
log.Warn().Err(err).Str("did", did).Msg("failed to refresh stale hold")
}
}
return rows.Err()
}
Security Considerations
Trust Model
- Captain records are authoritative: The hold's embedded PDS is the source of truth
- Crew records are authoritative: Same as captain records
- Cache is for performance: Always validate against source for sensitive operations
- No user-provided data: All data comes from Jetstream or direct PDS queries
Access Control
- Read access: Any authenticated user can view available holds
- Write access: Only hold owners can modify captain records
- Crew management: Only hold owners and crew admins can add/remove crew
Data Validation
func validateCaptainRecord(record *atproto.CaptainRecord) error {
if record.OwnerDID == "" {
return errors.New("owner DID is required")
}
if !strings.HasPrefix(record.OwnerDID, "did:") {
return errors.New("invalid owner DID format")
}
return nil
}
func validateCrewRecord(record *atproto.CrewRecord) error {
if record.MemberDID == "" {
return errors.New("member DID is required")
}
if !strings.HasPrefix(record.MemberDID, "did:") {
return errors.New("invalid member DID format")
}
for _, perm := range record.Permissions {
if !isValidPermission(perm) {
return fmt.Errorf("invalid permission: %s", perm)
}
}
return nil
}
func isValidPermission(perm string) bool {
valid := map[string]bool{
"blob:read": true,
"blob:write": true,
"crew:admin": true,
}
return valid[perm]
}
Implementation Checklist
Phase 1: Database Schema
- Add
hold_crew_memberstable topkg/appview/db/schema.sql - Create migration file
pkg/appview/db/migrations/006_hold_discovery.yaml - Verify
rkeycolumn included for delete event handling - Run migration on dev/staging databases
- Verify foreign key cascade works correctly
Phase 2: Jetstream Integration
- Add
io.atcr.hold.captainto wanted collections inpkg/appview/jetstream/worker.go - Add
io.atcr.hold.crewto wanted collections - Implement
ProcessCaptainfunction inpkg/appview/jetstream/processor.go - Implement
ProcessCrewfunction - Add hold service verification (
#atcr_holdcheck via DID document) - Handle delete events for captain records (cascade to crew)
- Handle delete events for crew records (by rkey lookup)
- Test with local hold service connected to local relay
Phase 3: Backfill
- Implement
BackfillHoldsfunction inpkg/appview/jetstream/backfill.go - Implement
backfillCrewRecordsfunction - Implement
listReposWithCollectionhelper - Add
ATCR_BOOTSTRAP_HOLDSenvironment variable support - Implement
BackfillBootstrapHoldsfunction - Implement
fetchCaptainFromHolddirect fetch - Test backfill with production relay
- Add backfill command to CLI (optional)
Phase 4: Database Queries
- Implement
UpsertCrewMemberinpkg/appview/db/hold_store.go - Implement
DeleteCrewMember(holdDID, memberDID) - Implement
DeleteCrewMemberByRkey(holdDID, rkey) - Implement
GetAvailableHolds(userDID)with membership categorization - Implement
GetHoldsOwnedBy(ownerDID) - Implement
GetCrewMemberships(memberDID) - Add unit tests for all queries
Phase 5: UI Integration - Settings Handler
- Add
DB *sql.DBfield toSettingsHandlerstruct - Call
db.GetAvailableHolds()in handler - Create
SettingsPageDatastruct with hold lists - Implement
prepareSettingsDatahelper function - Implement
deriveDisplayName(did, endpoint)helper - Create
HoldDataEntrystruct for JSON serialization - Serialize hold data to JSON for JavaScript
Phase 6: UI Integration - Template Changes
- Replace text input with
<select>dropdown insettings.html - Add
<optgroup>sections: Your Holds, Crew Member, Open Registration, Public - Add
[read-only]indicator for crew without write permission - Add hold details panel (
#hold-detailsdiv) - Add empty state notice when no holds discovered
- Add "Refresh Holds" button
- Update form to submit
hold_didinstead ofhold_endpoint
Phase 7: UI Integration - Styles & JavaScript
- Add
.form-selectstyles for dropdown - Add
.hold-detailsstyles for details panel - Add
.access-badgestyles (owner, crew, eligible, public) - Add JavaScript for hold selection change handler
- Show hold details on selection change
- Display permissions for crew members
- Handle initial page load with pre-selected hold
Phase 8: Form Handler Updates
- Update
UpdateDefaultHoldHandlerto accepthold_didparameter - Add fallback for legacy
hold_endpointparameter - Validate hold DID exists in cache
- Verify user has access to selected hold
- Return appropriate error for unknown/inaccessible holds
- Add
RefreshHoldsHandlerfor manual refresh button
Phase 9: Testing
- Unit tests for database queries
- Unit tests for Jetstream processors
- Integration test: discover hold via Jetstream
- Integration test: backfill existing holds
- E2E test: settings page displays holds
- E2E test: change default hold via dropdown
- E2E test: verify push uses new default hold
Phase 10: Cache Management & Monitoring
- Implement
RefreshStaleHoldsfor TTL-based refresh (optional) - Add Prometheus metrics for cache operations
- Monitor cache hit/miss rates
- Add logging for discovery events
- Document operational procedures
Future Enhancements
Hold Search
Add search/filter capabilities:
SELECT * FROM hold_captain_records
WHERE region LIKE ?
OR provider LIKE ?
ORDER BY ...
Hold Recommendations
Suggest holds based on:
- Geographic proximity (region matching)
- Provider preference
- Existing crew memberships
Hold Statistics
Display usage information:
- Storage used
- Number of images
- Number of crew members
- Uptime/availability
Hold Comparison
Side-by-side comparison of:
- Storage limits
- Supported features
- Geographic regions
- Pricing (if applicable)