mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-29 13:35:35 +00:00
Evan JarrettandClaude Opus 5
a95c89aaef
Around 2026-08-20 the Bluesky relay marked every ATCR hold offline and stopped dialing. Nobody noticed for two weeks, and it surfaced only indirectly as "pull and push counts are up to 24 h stale". The stats were a symptom; the fleet was simply disconnected. It cannot recover on its own. Indigo's relay gives up on a host after 16 consecutive dial failures and returns from the redialer, and only a fresh requestCrawl revives it. The hold sends requestCrawl exactly once, at boot (server.go:400), with no ticker and no check that any relay is subscribed. So a dropped hold is invisible until its process restarts, silently. Documents the current mechanics, the failure mode, how to diagnose it with getHostStatus and a frozen repo rev, and how to recover. The automatic re-crawl is described as a deferred proposal and explicitly NOT implemented, by decision: a jittered ticker guarded on subscriber liveness, plus surfacing the subscriber count, since the deeper problem is that this was silent. Two things found while writing it, both recorded. The proposal needs plumbing that does not exist: EventBroadcaster has no exported subscriber count, and Subscriber does not retain the userAgent, so "is a relay listening" cannot currently be answered. And ResubscribeAllHosts selects only active hosts, so an offline host is not recovered even by a relay restart. Carries a replay warning.ca539b1fixed a panic on subscriber disconnect during firehose backfill, and the exposure condition is that a backfill goroutine exists at all, which Subscribe skips when the cursor is current. So a caught-up relay never triggered it and a hold whose relays are far behind is exposed on every reconnect. Verify a deployed hold containsca539b1before provoking a re-crawl;efabb677does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9