Around 2026-08-20 the Bluesky relay marked every ATCR hold offline and stopped
dialing. Nobody noticed for two weeks, and it surfaced only indirectly as "pull
and push counts are up to 24 h stale". The stats were a symptom; the fleet was
simply disconnected.
It cannot recover on its own. Indigo's relay gives up on a host after 16
consecutive dial failures and returns from the redialer, and only a fresh
requestCrawl revives it. The hold sends requestCrawl exactly once, at boot
(server.go:400), with no ticker and no check that any relay is subscribed. So a
dropped hold is invisible until its process restarts, silently.
Documents the current mechanics, the failure mode, how to diagnose it with
getHostStatus and a frozen repo rev, and how to recover. The automatic
re-crawl is described as a deferred proposal and explicitly NOT implemented,
by decision: a jittered ticker guarded on subscriber liveness, plus surfacing
the subscriber count, since the deeper problem is that this was silent.
Two things found while writing it, both recorded. The proposal needs plumbing
that does not exist: EventBroadcaster has no exported subscriber count, and
Subscriber does not retain the userAgent, so "is a relay listening" cannot
currently be answered. And ResubscribeAllHosts selects only active hosts, so an
offline host is not recovered even by a relay restart.
Carries a replay warning. ca539b1 fixed a panic on subscriber disconnect during
firehose backfill, and the exposure condition is that a backfill goroutine
exists at all, which Subscribe skips when the cursor is current. So a caught-up
relay never triggered it and a hold whose relays are far behind is exposed on
every reconnect. Verify a deployed hold contains ca539b1 before provoking a
re-crawl; efabb677 does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
- diff view gains a Packages tab with added/removed/changed/unchanged
package tables and purl-derived type/license/upstream links
- captain records verified against the DID's atcr_hold service before
caching (processor + batch backfill), preventing forged holds
- fix empty-handle updates clobbering cached handles and colliding on
the UNIQUE constraint
- move fillPrevCIDs into repo.go; DirectRepoOperator is now canonical,
repomgr kept as a test oracle
- surface read-only crew status in hold selector
- reconcile docs
did:plc Identity Support (pkg/hold/pds/did.go, pkg/hold/config.go, pkg/hold/server.go)
The big feature — holds can now use did:plc identities instead of only did:web. This adds:
- LoadOrCreateDID() — resolves hold DID by priority: config DID > did.txt on disk > create new
- CreatePLCIdentity() — builds a genesis operation, signs with rotation key, submits to PLC directory
- EnsurePLCCurrent() — on boot, compares local signing key + URL against PLC directory and auto-updates if they've drifted (requires rotation key)
- New config fields: did_method (web/plc), did, plc_directory_url, rotation_key_path
- GenerateDIDDocument() now uses the stored DID instead of always deriving did:web from URL
- NewHoldServer wired up to call LoadOrCreateDID instead of GenerateDIDFromURL
CAR Export/Import (pkg/hold/pds/export.go, pkg/hold/pds/import.go, cmd/hold/repo.go)
New CLI subcommands for repo backup/restore:
- atcr-hold repo export — streams the hold's repo as a CAR file to stdout
- atcr-hold repo import <file>... — reads CAR files, upserts all records in a single atomic commit. Uses a bulkImportRecords method that opens a delta session, checks each record for
create vs update, commits once, and fires repo events.
- openHoldPDS() helper to spin up a HoldPDS from config for offline CLI operations
Admin UI Fixes (pkg/hold/admin/)
- Logout changed from GET to POST — nav template now uses a <form method=POST> instead of an <a> link (prevents CSRF on logout)
- Removed return_to parameter from login flow — simplified redirect logic, auth middleware now redirects to /admin/auth/login without query params
Config/Deploy
- config-hold.example.yaml and deploy/upcloud/configs/hold.yaml.tmpl updated with the four new did:plc config fields
- go.mod / go.sum — added github.com/did-method-plc/go-didplc dependency
Evan JarrettandClaude Opus 5