Around 2026-08-20 the Bluesky relay marked every ATCR hold offline and stopped
dialing. Nobody noticed for two weeks, and it surfaced only indirectly as "pull
and push counts are up to 24 h stale". The stats were a symptom; the fleet was
simply disconnected.
It cannot recover on its own. Indigo's relay gives up on a host after 16
consecutive dial failures and returns from the redialer, and only a fresh
requestCrawl revives it. The hold sends requestCrawl exactly once, at boot
(server.go:400), with no ticker and no check that any relay is subscribed. So a
dropped hold is invisible until its process restarts, silently.
Documents the current mechanics, the failure mode, how to diagnose it with
getHostStatus and a frozen repo rev, and how to recover. The automatic
re-crawl is described as a deferred proposal and explicitly NOT implemented,
by decision: a jittered ticker guarded on subscriber liveness, plus surfacing
the subscriber count, since the deeper problem is that this was silent.
Two things found while writing it, both recorded. The proposal needs plumbing
that does not exist: EventBroadcaster has no exported subscriber count, and
Subscriber does not retain the userAgent, so "is a relay listening" cannot
currently be answered. And ResubscribeAllHosts selects only active hosts, so an
offline host is not recovered even by a relay restart.
Carries a replay warning. ca539b1 fixed a panic on subscriber disconnect during
firehose backfill, and the exposure condition is that a backfill goroutine
exists at all, which Subscribe skips when the cursor is current. So a caught-up
relay never triggered it and a hold whose relays are far behind is exposed on
every reconnect. Verify a deployed hold contains ca539b1 before provoking a
re-crawl; efabb677 does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
- diff view gains a Packages tab with added/removed/changed/unchanged
package tables and purl-derived type/license/upstream links
- captain records verified against the DID's atcr_hold service before
caching (processor + batch backfill), preventing forged holds
- fix empty-handle updates clobbering cached handles and colliding on
the UNIQUE constraint
- move fillPrevCIDs into repo.go; DirectRepoOperator is now canonical,
repomgr kept as a test oracle
- surface read-only crew status in hold selector
- reconcile docs
Evan JarrettandClaude Opus 5