mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-29 05:25:35 +00:00
47a107058dccae46a67d5441942c2bd1d26757ba
383
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Evan JarrettandClaude Fable 5.1
|
47a107058d |
appview: bound upload buffer memory, reap abandoned uploads, and pin the flush boundary
Each in-flight blob upload buffers up to 16MB, Docker pushes five layers at once per client, and nothing bounded the total. Writers also lived in the package-level map forever: a client that died mid-push left its writer, its buffer, and any hold-side S3 multipart session behind with no expiry. A process-wide budget (golang.org/x/sync semaphore, default 512MB, server.upload_buffer_budget_mb) now caps memory held in upload buffers. A writer charges its buffer's projected backing capacity before growing, so a config blob costs kilobytes and a full writer costs exactly one buffer, and releases once, on Commit, Cancel, or reap. A write that needs budget waits on the request's context with a five minute cap, outside the writer's lock so Cancel and the sweeper cannot queue behind it; that wait is backpressure on the client. The budget is clamped to at least one buffer so a single upload can never deadlock. A sweeper started with the other appview workers reaps writers idle past server.upload_idle_timeout (default 1h), aborting the hold-side multipart on a detached context and releasing the budget. It measures inactivity, not age, so a slow push is never reaped, and it skips a writer whose lock is held so it cannot race a live part upload. Write also gains a fix the budget made visible. It appended a whole chunk and checked afterwards, so the last chunk before a flush could land a few bytes past 16MB, which did not fit the backing array; bytes.Buffer doubled it to 32MB and Reset kept that for the rest of the upload. Only chunk sizes that tile 16MB exactly avoided it, and the network read loop promises no such thing. Every large layer could hold 32MB while the budget charged 16. Write now fills to exactly the threshold, flushes, and continues with the remainder, so capacity is pinned at 16MB for any chunk size, every part is exactly one buffer, and a single oversized Write streams through as parts instead of buffering whole. The test streams 24KB chunks across the boundary and fails against the old code with cap 33554432. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5 |
||
|
Evan JarrettandClaude Fable 5.1
|
f4343d7956 |
appview: upload small blobs with one presigned PUT, and verify every digest
Every blob went through the multipart machinery: an S3 multipart started on Docker's initial POST, a hold round trip per part, and a complete on the hold that finished the multipart, HEADed the temp object, copied it to its final key, and deleted the temp. For a 2KB config blob that was three hold calls and six S3 operations. On production data 86% of distinct layers and every config blob fit in a 16MB buffer, and 49% of image manifests have no layer larger than that. The writer now buffers up to 16MB (also the multipart part size) and makes no hold call until it has to. A blob that never overflows the buffer is written at Commit with a single presigned PUT to its final key, via the hold's existing method=PUT presign; the multipart only starts on the first flush. The hold's completeUpload does nothing the direct path skips: quota, layer records, stats and scan dispatch all hang off notifyManifest, which is unchanged. The buffer starts empty and grows on demand, with the doubling capped so capacity never overshoots 16MB: a config blob costs kilobytes, and only layers that approach the threshold fill it. Bytes are hashed as they arrive. Commit compares the computed sha256 to the digest the client claimed before any network call, and returns DIGEST_INVALID on mismatch, aborting a multipart if one was started. Previously nothing verified the content, so a pusher could store wrong bytes under a digest in the shared content-addressed space. Tests observe request counts on a fake hold and fake S3 rather than return values. The growth test streams in 24KB chunks because power-of-two chunks land on 16MB by luck and hid an earlier weaker guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5 |
||
|
Evan JarrettandClaude Fable 5.1
|
034ea5988b |
hold: report blob size on read presigns so the appview can skip its S3 HEAD
distribution calls Stat before every blob GET and HEAD. The appview's Stat asked the hold for a presigned HEAD URL and then HEADed S3 with it purely to read Content-Length for the descriptor: two round trips to learn one number. The hold's getBlob response for OCI digests on GET and HEAD now carries "size". It comes from the records index when a layer record exists (a SQLite lookup on a new digest index, no network) and from a HeadObject otherwise, which is where config blobs land. If storage says the object does not exist the hold answers 404 instead of signing a URL that can only fail. The PUT and ATProto CID paths are untouched. The appview builds the descriptor from the reported size and makes no S3 request. When the field is absent it HEADs the presigned URL as before, so a new appview works against a hold that has not been upgraded, and an old appview ignores the extra field. A hold 404 maps to ErrBlobUnknown. Tests prove the index answered by leaving the mock bucket empty and counting zero HeadObject calls, prove the fallback with exactly one, and count requests reaching the fake S3 origin on the appview side rather than trusting the returned size. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5 |
||
|
Evan JarrettandClaude Fable 5.1
|
9228579b07 |
appview: share one HTTP transport across blob proxy requests
NewProxyBlobStore built a fresh http.Client and http.Transport on every
call, and it is called once per registry request because the routing
repository is created per request. So no connection to the hold or to a
presigned S3 URL was ever reused: every XRPC call and every part upload
paid a TCP and TLS handshake, HTTP/2 was never negotiated, the idle pool
settings were dead config, and each discarded transport kept its idle
sockets open for the full 90s timeout.
One package-level transport and client now back every store. Settings
are unchanged; ForceAttemptHTTP2 is explicit to document intent. The
load balancer negotiates h2 on the frontend (
|
||
|
Evan JarrettandClaude Opus 5
|
8d7ccd7cb7 |
apply go fix modernizations across the workspace
`go fix` carries the modernize analyzers now, and the tree had drifted behind
them. This is the mechanical result, reviewed rather than trusted: the tool is
capable of rewriting code into something that no longer tests or does what it
did, so every non-test change was read individually and the concurrency-bearing
packages were re-run under -race.
Production code, four changes, all semantics-preserving:
- leases/manager.go: wg.Add(1) + go + defer wg.Done() becomes wg.Go. The
comment above that function turns on Add happening before the goroutine
starts, so that a Wait cannot return before the worker has run. wg.Go does
the Add synchronously on the calling goroutine, so the invariant it
describes still holds.
- auth/token/handler.go: strings.Fields -> strings.FieldsSeq, same splitting,
iterated rather than allocated.
- hold/gc/gc.go: a hand-written map copy -> maps.Copy.
- hold/pds/scan_broadcaster.go: three-clause loop -> range over int.
The rest are tests. The one worth naming is carstore_contention_test.go, where a
careless rewrite could have quietly stopped exercising contention: go fix
converted the reader and side-table goroutines to loopWG.Go but correctly
declined to touch the writer loop, which passes its index as a parameter. The
writer/reader/side-table shape and the stop channel are unchanged, so the test
still contends over the same carstore transactions.
Verified: go build for hold and appview, `make lint` 0 issues, the deploy and
credential-helper modules 0 issues, `make test` green across all 43 packages,
and -race green on leases, hold/pds, hold/gc and auth/token. The scanner module's
two lint findings are unchanged from HEAD and are in files go fix never touched.
Kept separate from the HTTP/2 commit so that one stays readable, and so this can
be reverted on its own if a modernization turns out to matter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TA9D4DjaLZTvzQ7dJbu4eg
|
||
|
Evan JarrettandClaude Opus 5
|
416ba4a2eb |
serve HTTP/2, and reach it through the load balancer
The admin crew tab renders one row per crew member and gives each row its own hx-get, so opening it on a hold with 551 crew issues 551 requests. Over HTTP/1.1 a browser runs at most ~6 per origin, so they queue six at a time and every other request to the same host queues behind them — which is why loading the relay page stalls while the crew rows are still resolving, and why the rows that lose the race come back as "Server error" toasts. The 504 behind that toast is the load balancer's, not the hold's: the hold logs those requests as 200. HTTP/2 multiplexes them over one connection and the queue disappears. It does not make the slow rows fast — that is a separate fix to the per-row identity lookup — but it stops one slow surface from blocking the rest of the panel. Two halves, because neither works alone. The load balancer terminates TLS and speaks cleartext to the origin, so ALPN never runs on the backend leg and net/http can only answer HTTP/1.1 there. Both servers now wrap their handler in h2c. The wrapper is opt-in per connection: it upgrades only for a client sending the h2c preface or "Upgrade: h2c", and passes everything else through untouched, so an HTTP/1.1 WebSocket upgrade is unaffected. Verified both directions against this wiring — HTTP/1.1 for a plain client, HTTP/2.0 with --http2-prior-knowledge. The frontend's http2_enabled was never set, so it sat at the UpCloud default of off. That is the half the browser actually sees. timeout_client is now stated explicitly at its current 10s rather than left implicit: it is the boundary that produces the 504s above, so it belongs somewhere visible. It is deliberately unchanged — raising it without fixing the slow lookup would only make a stalled row stall longer. The hold's *backend* stays on HTTP/1.1. It serves subscribeRepos over WebSocket to external relays and to the scanner, and WebSocket over HTTP/2 needs the RFC 8441 Extended CONNECT that Go's http2 server does not implement for Upgrade:. Routing that backend over h2 would break the firehose. The appview accepts no inbound WebSocket and has no such constraint. Both origins carry h2c regardless, so enabling it for the hold later is a config change, not a code change. createLoadBalancer only runs when there is no LB yet, so properties set there would reach a new deployment and never an existing one. ensureLBHTTP2 reconciles them onto an LB that already exists, following ensureLBForwardedHeaders: read what is there, change only what differs, report what it did, and no-op on a second run. Backend modifies carry the existing health check back, since Properties replaces the object wholesale. Also gofmt: provision.go was not gofmt-clean at HEAD, unrelated to this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TA9D4DjaLZTvzQ7dJbu4eg |
||
|
Evan JarrettandClaude Opus 5
|
a63f668de0 |
scanner: fix five crash and halt classes found by a pipeline audit
An audit of the scan pipeline and the hold side of scanning found several ways scanning stops without saying so. Each fix here was written test-first: a test expressing the wanted behaviour, confirmed failing for the right reason, then the change. A summary-less result crash-looped both processes. worker.go dereferenced result.Summary unconditionally, but processJob only sets it when Grype runs, and SendResult puts the nil on the wire before the scanner dies on it, so handleResult's unguarded log killed the hold too. A nil Summary now means "not scanned for vulnerabilities", deliberately distinct from "scanned, found zero" — inventing a zeroed summary would report every image as clean when Grype never ran. The hold writes a record rather than orphaning the uploaded SBOM, and the appview renders an "SBOM only" state instead of a green Clean badge. The Grype database could wedge with no way back short of a restart. All three throttles in loadVulnDatabase were guarded by vulnDB != nil, so a scanner holding no provider retried a full download on every scan under the exclusive lock. Two earlier attempts at this bug each added one more condition to the same chain; this replaces the chain with a single decision function over a state snapshot, consulted by both call sites so they cannot disagree. That disagreement was itself a bug: the 50-scan reload had never once executed. Two independent halts. An unparseable frame was dropped in silence, stranding a row that held the hold's only dispatch slot forever; it is now answered "skipped" on first delivery. The 10-minute sweep leaked the in-flight digest and wrote no record, permanently retiring one image per timeout. A digest went unvalidated into filepath.Join and os.Create, so a layer digest of sha256:../../../x wrote outside the scan directory, and nothing verified that downloaded bytes hashed to the digest naming them. Digests come from records in a user's own PDS. Both are fixed together: verification is what makes an escaping write self-defeating. Concurrency did not work on either axis. The proactive capacity gate was depth-one hold-wide, so neither extra workers nor extra scanner processes received work. Depth is now the sum of the worker counts scanners advertise on connect, the gate is scoped to proactive work, and dispatch prefers the least-loaded scanner. Disconnects no longer hand a running scan to someone else: a scanner keeps a stable per-process identity and reclaims its own rows within a grace window, while a process that truly restarted returns with a new identity and has its work reclaimed, which is correct because the restart did lose it. The hold's scanning deadline measured queueing rather than scanning, because the scanner acks on receipt and handleAck never refreshed assigned_at. A new "started" message, sent by the worker that dequeues the job, separates the two budgets. An older scanner never sends it and falls under the queueing budget, which is more forgiving than the deadline it gets today. Adds an in-process mock hold and an e2e harness that runs the real client, queue and worker pool, seeded with 84 real manifest records fetched from a live PDS. Real image layouts and the Grype database are fetched by scripts and gitignored; suites needing them skip cleanly, so the default run stays offline and fast. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U1Km3N3uUmeGaj7VbaM8PF |
||
|
Evan JarrettandClaude Opus 5
|
f16a8eaa82 |
appview: warn about a deprecated chart on the tab people actually land on
|
||
|
Evan JarrettandClaude Opus 5
|
3589473feb |
appview: run the hold purge on a worker pool, not the request context
Deleting a tag while over quota could leave the user in the worst available state. DeleteTagHandler deleted the tag and manifest rows first, then called PurgeOnHold, which bounded itself at 10s against the *request* context. The UpCloud load balancer in front of the appview cuts at its default backend timeout at about the same moment, wins the race, hands the client a 504 and cancels that context, killing the purge partway. The appview logged a warning and returned 200. So: gateway error, nothing freed, still locked out, the image gone from the UI so the purge cannot be retried through it, blobs orphaned until the hold's GC, and the appview considering it a success. Measured on production at 10.002367218s. Purges now go to a fixed pool of 4 workers rooted at context.Background(), so they survive the request ending. Following the shape of the hold's startJob helper, minus the progress fragment, since nobody is watching a purge. The buffer is bounded at 256 and sheds with an ERROR rather than growing: an unbounded queue turns a slow hold into an appview memory leak. Submissions are deduplicated on holdDID|manifestURI so a double-clicked delete does one purge and one service-token fetch. The channel send happens under the mutex that guards close, so a concurrent drain cannot send on a closed channel, and the drain is wired into both exit paths before logging shuts down. Failures are now classified and surfaced instead of swallowed: transient ones retry three times under a 90s budget (the hold's purge is idempotent), an unauthorized third-party hold logs at DEBUG since it is expected, and anything else that exhausts its retries logs at ERROR naming the manifest and hold, which is enough to re-drive by hand. Deliberately not reordered. Purge-first-then-delete requires waiting for the purge to know whether to delete, which puts the 10s call straight back on the request. So the orphaned-blob window remains, materially narrower but real: a purge that exhausts its retries still leaves blobs referenced by nothing until the hold's GC, and there is no row left to say so. Closing that needs a durable pending-purge record, which was judged out of proportion here. server.go in this commit also carries one line belonging to the next one, the token handler's display-name wiring, since the two changes landed in the same file concurrently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
b4fccce4d1 |
hold: actually send subject, so the attestation scan guard can fire
The hold declines to enqueue a scan when a pushed manifest has a subject, which
is how it means to skip attestations, signatures and other referrer artifacts.
The AppView never sent one: notifyHoldAboutManifest built mediaType, config,
layers and manifests, and #manifestInfo defined only those four. So the
condition was always true, the guard never fired, and every referrer artifact
was enqueued for scanning.
The AppView already parsed subject. NewManifestRecord unmarshals it into
ManifestRecord.Subject, and Put hands that same pointer to
notifyHoldAboutManifest. The value was in scope and simply never serialized, so
this is one missing marshal step rather than a missing parse.
Adds subject to the lexicon as a #blobInfo ref, and mediaType to #blobInfo,
which config has always sent and the hold has always parsed. That only makes the
schema honest about what is already on the wire.
Hoists the hold's anonymous request struct to a named type with IsMultiArch,
IsReferrer and HasScannableContent, so the predicate is written once and
testable without standing up a HoldPDS.
Both directions degrade safely. An older hold ignores the unknown key and
behaves exactly as today, so shipping the appview alone is harmless but achieves
nothing until the hold catches up. An older appview sends no subject, leaving
the manifest enqueued as before.
Complements
|
||
|
Evan JarrettandClaude Opus 5
|
9566201377 |
appview: give the confirm modal an accessible name and a focus trap
The modal guards tag deletion, device revocation and webhook deletion, so it is the wrong control to leave unlabelled and escapable. Note the original report was half wrong: role="dialog" and aria-modal="true" were already set on the outer element, and are in the deployed bundle. The observation was most likely taken against the inner .modal-box. Neither was added here. What was actually missing: the h2 existed but nothing pointed at it, so a screen reader announced a dialog with no name; and the only focus management was focusing Cancel on open, so Tab walked straight out into the skip link and page header behind the backdrop. Adds aria-labelledby and aria-describedby with sequence-suffixed ids so two modals cannot cross-reference. aria-describedby matters more than usual here: none of the three call sites' messages contain ". ", so the title is always the generic "Are you sure?" and the specific text is the body. The trap re-queries on each keypress rather than caching at open, and handles three cases: focus escaping to body gets pulled back, first plus Shift+Tab wraps to last, last plus Tab wraps to first. Focus is restored to the opener on every close path, guarded by document.contains, because confirming a tag deletion fires an htmx swap that can remove the button that opened the modal. Restoration happens before onConfirm so htmx sees a sane focus state. Native <dialog> would give the trap and Escape for free, but it renders in the top layer while this is styled entirely with daisyUI .modal classes that assume a positioned div, so converting means CSS work plus the seamark theme fork. Worth doing as its own change, not smuggled into this one. Includes the bundle rebuild, since nothing in the dev loop keeps that artifact current. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
dbb195a4ab |
appview: process daily stats, and drop the dead README branch fallback
io.atcr.hold.stats.daily was handled in the backfill collection list and in processor.go's dispatch, but missing from isRelevantCollection, which gates events before ProcessRecord ever sees them. So daily stats records arrived over the socket and were discarded at the worker, and the trend charts that read them got nothing live. This is the "present in one list, missing from the other" shape CLAUDE.md's firehose checklist warns about. Checked the whole class: this was the only gap. LayerCollection and ImageConfigCollection are absent deliberately, having no processor handler, and the test now pins that intent. Note this is currently masked by the relay outage, so fixing the relay alone would not have restored the charts. Separately, the README resolution tried "main" and fell back to "master", but DeriveReadmeURL never fetches: it parses the source URL and interpolates the branch, returning empty only for an unsupported platform, which is branch-independent. So if the main call returned empty the master call returned empty for the same reason, and the fallback could never fire. Removed, with a comment recording that a branch fallback has to happen at fetch time after a 404. The other two call sites already do exactly that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
7a0769d8e4 |
appview: three small UI fixes, empty state, icon sizing, aria-selected
Artifacts filter had no empty state: filtering to zero matches left a blank panel, indistinguishable from something being broken. It now uses the existing state-empty partial, with no CTA since the filter input is right there. Rows appended by Load More now also obey the active filter, which is a change to existing behaviour but is required for the empty state to be truthful. The tag icon beside the tag selector computed to 5.44px by 24px on every repo page: the class was right, the flex parent was shrinking it. Adds shrink-0 to that instance only. 31 other icons are direct flex children without shrink-0 and are left alone, since a site-wide sweep found only this one squeezed. Digest-page scan tabs set role="tab" but never aria-selected, while the repo-page tabs do. Both are now set server-side so first paint is correct, with JS keeping them in sync afterwards, following switchRepoTab's pattern. The diff-content tabs had the identical defect and are fixed too, since the handler is generic over radio tabs and leaving them out would have meant attributes set by JS but never by the server. Includes the bundle rebuild for these and the two /auth/token-adjacent JS changes, since nothing in the dev loop keeps that artifact current on its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
9d8bd513da |
appview: render the install scripts from config instead of shipping ATCR's
seamark.dev's /install and /settings/devices told users to pipe
seamark.dev/static/install.sh into bash. That file was the unmodified ATCR
script: it announced itself as the "ATCR Credential Helper Installer",
installed docker-credential-atcr, and finished by telling the user to configure
credHelpers for atcr.io, the wrong registry for that deployment. Anyone
following the documented setup ended up pointed at another service. The
templates hardcoded docker-credential-atcr, "atcr" and ~/.atcr/device.json
alongside a correctly themed {{ .RegistryURL }}.
The scripts are now rendered from config by a handler, rather than forked per
brand. A theme overlay was the alternative and was worse: it needed a full copy
of both install.sh and install.ps1 per brand, four scripts to keep in sync, and
the operator asked for these values to come from config.
credential_helper.name is the single knob. Docker resolves a credHelpers value
x by exec'ing docker-credential-x, so the credHelpers value, the binary suffix
and the config directory are genuinely one word, not three that can drift. It
is validated against a strict pattern because it is interpolated into a shell
script.
install.sh renders byte-identical to the deleted static file under the atcr
default, so existing installs are unaffected. install.ps1 differs by one line,
where a stale usage comment named a path the script is not served at.
Two behaviour changes worth noting: these two URLs drop from a one-year
Cache-Control to five minutes, since the body now depends on deployment config;
and credential_helper.tangled_repo becomes a real overridable default. It was
previously assigned over unconditionally and read by nothing, while the shipped
script used a different URL form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
|
||
|
Evan JarrettandClaude Opus 5
|
84872580a3 |
appview: show a chart's deprecation where people actually look
digest.html rendered a Deprecated chip, but the shared helm-metadata partial never read .Deprecated, so the repo page carried no deprecation signal at all. The data was already on the struct; this was a display omission. The notice leads the metadata card, above the description: deprecation decides whether you use the chart at all, so it has to be read before the prose that sells it. It uses the alert/alert-warning callout the codebase already uses for this kind of thing, including in the adjacent helm-digest-content partial, rather than a bare badge. A lone small badge reads as a stray tag once it is outside the digest page's row of status chips, and leaves no room to say why it matters. The digest page now shows deprecation twice, deliberately: its header chip is the scannable signal above the fold, and this carries the explanation further down. Removing the chip would push the only signal below the install command, which is the burial this fix is meant to undo. Note the repo page's Overview tab, which is where most people land, still shows nothing: it renders the README and never touches chart metadata. Only the Chart tab gains the notice here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
0490278fb8 |
appview: don't report a failed README guess as an error
Four of the twelve home-page repos showed "We couldn't load the README, it may be rate-limited or private". The URL in those cases was not configured by anyone: it was derived from org.opencontainers.image.source, a label images inherit from their base image, so the raw URL named an unrelated project and 404ed. A 404 on a URL the appview guessed is an expected outcome the owner cannot act on. The failure flag is now set only when the owner actually pointed us at the URL, via the io.atcr.readme annotation. A derived URL that fails renders as if there were no README. Both paths keep their debug log, now carrying an "explicit" field so the two cases stay distinguishable. Render failures are suppressed for derived URLs too. The panel's copy and its "Edit README" action address an owner who configured a source; on a derived URL there is no configured source, and content that failed to render is very likely another project's README anyway. This is the alarming half of the finding. Rendering the wrong project's README when the fetch succeeds is the larger half and is untouched: there is no reliable way to detect an inherited label, since the only signal is org.opencontainers.image.base.name, which is not always set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
44a17cbcdc |
appview: stop the upgrade banner inventing an improvement across arch mismatch
On a public repo, to anonymous visitors, the digest banner read "2 fixes, 12 Critical / 27 High / 26 Medium vulns, -22 layers, -53.6 MB" while the same page showed 244 vulnerabilities and Layers (22), and its own "View diff" link landed on "Layers 22 -> 22, every layer Unchanged". Platform matching only ran when both sides were manifest lists. The comment after that block said the mismatched case would "fall through and show a basic banner without layer/vuln details", but no such branch was ever written and nothing guarded the fallthrough, so execution continued into the layer and vuln computation with the unresolved originals still in place. The newer side was the multi-arch index, which carries no layers of its own and is not scanned, so all 22 layers of the other side read as removed and the vuln delta was computed against an absent scan. Returns 204 for either mismatch direction, as the no-common-platform path already does. The promised "basic banner" is not implementable as the function stands, which is presumably why it never appeared: the template renders only NewerTag, DiffURL and Summary, and DiffSummary is nothing but deltas, so a delta-less banner collapses to the tag name and would be suppressed by the existing "nothing meaningful changed" guard anyway. The comment is replaced with one that says what is actually true. Showing a real banner here would mean resolving the index to the child matching the single-arch side's platform, and that side's os/arch is not in the appview DB at all: Platforms is populated only for manifest lists and the manifests table has no os/arch columns. It would need either a config fetch from the hold at render time or denormalising os/arch during ingest. Not attempted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
39919cc832 |
appview: stop webhooks reaching private addresses
The URL check accepted http:// while telling the user "must be https", and guarded no addresses at all. POST /api/webhooks with http://127.0.0.1:9/hook returned 200 and created the webhook, so both scheduled deliveries and the synchronous Test button would dial arbitrary destinations from the appview host, on demand, for any authenticated user. Loopback, link-local (including the cloud metadata endpoint at 169.254.169.254) and RFC1918 were all reachable. Enforces https, and refuses non-public destinations. The load-bearing half is the dial-time check, not the creation-time one. An attacker controls their own DNS, so a hostname that resolves publicly when the webhook is created can resolve to loopback when it is delivered, and a creation-time check cannot see a redirect either. The guard is therefore a net.Dialer Control hook on the delivery client, which inspects the resolved address on every connection attempt. Transport.Proxy is explicitly nil: honouring HTTP(S)_PROXY would route around the Control hook and hand the bypass straight back. Redirects are re-validated per hop and capped at 3. The creation-time check stays so the user gets an immediate, comprehensible error instead of a silent delivery failure later. IPv4-mapped IPv6 is unmapped before every check, so ::ffff:127.0.0.1 and friends hit the IPv4 rules. Ranges with no net.IP helper are listed explicitly: CGNAT, NAT64, ::/96, TEST-NET and reserved space. Both outbound paths are covered, since the scheduled dispatcher and the Test button both funnel through attemptDelivery. The dispatcher's other client is deliberately left unguarded: it fetches quota stats from holds, which legitimately live on private addresses, and those URLs are not user-supplied. Note this removes the ability to point a webhook at a localhost receiver in local development. There is deliberately no environment-variable escape hatch, since a security toggle read from the environment is the same bypass wearing a nicer coat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
af7522b154 |
appview: stop /auth/token granting anonymous pull the registry will refuse
An unauthenticated token request for a repo on a private hold came back with a signed token granting pull, and /v2/ then 401ed that exact token. The authorization server and the resource server disagreed about the same request. The old behaviour was deliberate — "minting a pull-only token is not a grant", with the hold owning the decision via captain.Public — but a token spec expects the server to issue the subset it will authorize, so granting pull and then refusing it is the wrong shape. Adds an optional AnonymousAuthorizer, kept separate from Authorizer because the anonymous path has no DID and no auth method (three of Authorize's four arguments are meaningless) and because it must drop whole entries rather than narrow actions in place, where entries can belong to different owners. Denied entries are dropped; if nothing granting survives, the caller gets the standard 401 challenge rather than a token with an empty access list, so docker prompts for credentials instead of proceeding to a second 401. The scope-less /v2/ ping and the actionless entry NarrowToPullOnly preserves on purpose both bypass the gate entirely — no identity resolution, no hold lookup — since anonymous discovery depends on them. Fails open on any lookup error, matching the /v2/ check, which states the reason: the hold is the enforcing authority and a transient failure must not break anonymous pulls of public images. /v2/ still enforces; this is a correctness and UX fix, not a security fix, and nothing was exposed. Also closes the successor asymmetry documented under finding 3: /v2/ applies a single-hop migration redirect before checking read access, so judging the pre-migration identity here would have reintroduced the disagreement this gate removes. It was one extra local read of hold_captain_records. Tests pin both directions and prove the chain is not followed past one hop. Costs one directory-cached identity resolution plus two local SQL reads per granting entry, and no call to the hold. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
4caeb25031 |
appview: rebuild the JS bundle so the committed asset matches src
Picks up the two JS changes in this batch: the crane destination argument in
updatePullCommand (app.js) and syncDiffMenu (repository.js).
The bundle is a committed build artifact, and nothing in the dev loop keeps it
current on its own. Air's pre_cmd is go generate and its cmd is a Go build; it
never invokes esbuild. Bundling happens in npm run js:watch, a separate
process. So a src-only change leaves the committed bundle stale until someone
runs npm run js:build by hand, which is how
|
||
|
Evan JarrettandClaude Opus 5
|
85f312f953 |
appview: refuse /v2/_catalog with UNSUPPORTED, and send Allow on every 405
/v2/_catalog answered a bare request with 200 {"repositories":[]} but 400ed
any n, because buildDistributionConfig leaves Catalog.MaxEntries at 0 and the
library rejects n > max. crane catalog sends n=1000, so it always failed. The
same endpoint both worked and rejected a legal parameter.
Setting MaxEntries was the obvious fix and is the wrong one. This registry has
no global catalog and will not grow one: repositories live in per-user ATProto
PDS namespaces, and buildStorageConfig hands the library a placeholder
inmemory driver, so its enumeration is empty by construction. An empty 200
asserts that this registry contains no repositories, which is false and
silently so. UNSUPPORTED says the true thing, and matches the vocabulary
DomainRoutingMiddleware already uses to refuse /v2/* on the UI domain.
There is no conformance cost: _catalog is not in the OCI distribution spec at
all. It is a Docker Registry HTTP API V2 extension, and the spec places
repository discovery out of scope. Docker Hub and GHCR both refuse it outright
and Quay returns an unconditional empty list; none of them 400s a legal n.
The path had three distinct behaviours, not two, and all three now collapse to
one: GET varied by parameter, HEAD was answered by gorilla's MethodHandler
with a bare 405, and the trailing-slash form 301-redirected because
distribution sets StrictSlash(true). Both path forms are registered, and chi
prefers a static pattern over the /v2/* wildcard regardless of declaration
order (verified against the pinned chi version, and pinned by a test that
fails if a request reaches the distribution stand-in).
Also adds the Allow header that RFC 9110 requires on any 405 — a MUST in both
15.5.6 and 10.2.1, not a SHOULD. The value is empty, which 10.2.1 defines as
"the resource allows no methods": true for the catalog, and true for /v2/* on
the UI domain, so the pre-existing gap in DomainRoutingMiddleware is closed
too. Naming a method there would advertise something that does not work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
|
||
|
Evan JarrettandClaude Opus 5
|
8bc6d65e1e |
appview: emit crane's mandatory destination in the pull command switcher
The client switcher built every command as "<client> pull <ref>". That is valid for docker, podman, buildah and nerdctl, but crane requires a destination: $ crane pull seamark.cr/user/bench8x1:v2 Error: requires at least 2 arg(s), only received 1 So selecting crane handed the user a command that cannot run. pullPrefix is a prefix-only helper, which is precisely why it could not express this; add a matching pullPostfix that returns " <image>.tar" for crane and "" for everything else, including "none" (image reference only), which must get neither prefix nor postfix. Both render paths change together, since fixing one leaves the bug visible in the other: the Go template helper paints first, and updatePullCommand in app.js re-renders when the dropdown changes. Repository names may contain slashes, so only the last path segment is used — otherwise the destination would name a subdirectory that does not exist. A name ending in "/" yields no destination at all rather than a bare ".tar", on the grounds that a visibly wrong-arity command beats silently writing a hidden file. That input is not reachable through the real repo-name path. The test asserts the whole command string rather than just the postfix, so it covers the prefix/postfix interaction and the "none" case where both vanish. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
3158460298 |
appview: hide the current tag from the repository Diff dropdown
The Diff menu listed every tag including the one being viewed, and clicking
that entry did nothing: diffToTag returns early on to === currentTag, with no
navigation, no toast, no feedback.
Fixed in JS rather than in the template, which is the part that is easy to
get wrong. #diff-dropdown sits outside #tag-content, and the tag selector
block is marked "stays in DOM, never swapped" — so a {{ range }} filter would
be correct on first paint and stale after the first htmx tag swap, omitting
the originally loaded tag and re-including the newly current one. Same dead
entry, harder to see.
syncDiffMenu() reads the live value from #tag-selector and is called from
initTabs(), which already runs on load and again from the htmx:afterSettle
handler for #tag-content, so it stays correct across swaps.
Hidden rather than disabled: a disabled row still takes space and still reads
as an item to a screen reader, and "diff against the tag you are already on"
is meaningless rather than temporarily unavailable. Uses style.display to
match filterTags() in the same file, since daisyUI's .menu li rules outrank
Tailwind's .hidden.
The template guards the dropdown with {{ if gt (len .AllTags) 1 }} and tag
names are unique per repo (tags PK is did+repository+tag), so exactly one
entry is ever hidden and the menu can never end up empty. The early return in
diffToTag stays as a backstop.
Pre-existing, not a deploy regression. The baseline missed it because the
test repo had one tag, so the check skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
|
||
|
Evan JarrettandClaude Opus 5
|
1631898005 |
appview: give every Repository() error an OCI code instead of 500 {}
A user with an unresolvable defaultHold (did:web:localhost%3A8080) got HTTP
500 with a body of literally {} on every request in their namespace, and
crane retried it three times because 500 is retryable.
The cause is in the distribution library. handlers/app.go:755 switches on the
error type returned by Repository() with cases for ErrRepositoryUnknown,
ErrRepositoryNameInvalid and errcode.Error, and no default. A bare fmt.Errorf
matches none of them, so context.Errors stays empty; ServeJSON then finds no
ErrorCoder, leaves sc == 0, falls through to 500, and Errors.MarshalJSON
renders the nil slice as {} via omitempty.
So every error leaving Repository() must be coded. Four were not:
hold URL unresolvable -> 404 NAME_UNKNOWN when errors.Is
atproto.ErrHoldDIDPermanent, else 503 UNAVAILABLE
no hold DID configured -> 500 UNKNOWN, but with a body and a log line
invalid image name -> 400 NAME_INVALID
name missing an owner -> 400 NAME_INVALID
The permanent/transient split is the point: a DNS blip must stay retryable,
but a did:web that can never resolve must not be retried at all. Stored user
data that cannot resolve is a 4xx condition, not a server fault, and the
NAME_UNKNOWN message now names the hold so the owner can fix their profile.
The hold DID is already world-readable in their sailor profile record.
Tests assert through a helper that replays distribution's exact type switch,
so an uncoded error still surfaces as 500 {} and the assertions bind to the
real behaviour rather than to the constructors.
Does not address the logrus line at app.go:757, which fires unconditionally
before the type switch and cannot be avoided by any returned error type.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
|
||
|
Evan JarrettandClaude Opus 5
|
2ee5a35525 |
appview: stop the schema-drift check reporting phantom index differences
reportSchemaDrift logged 76 differences on every boot, all false positives: 38 indexes each reported twice, once as missing from the database and once as present but undeclared. The only difference was a single space before the column list. The two sides of the comparison are built differently. referenceSnapshot() applies schema.sql to a local in-memory libsql, which stores the CREATE text verbatim as "ON t(col)". Production is an embedded replica syncing to Bunny, whose parser re-emits normalized DDL as "ON t (col)". describeIndexes collapsed whitespace runs but could not normalize a space that exists on one side only, and SchemaDrift compares by exact string. Normalize whitespace adjacent to ( ) and , so both spellings converge. Space after ) is deliberately left alone, and the space before DESC is untouched, so column order and direction still have to match. This mattered because the check exists to catch a migration recorded but not executed (0004 was, which is why 0009 exists). At 76 phantom findings, real drift would have been one line among 77, under a hint telling the operator to write a corrective migration that is not needed. The first boot after this lands is the first honest reading of that warning. The existing tests all passed because they run local-only, which is exactly how this shipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9 |
||
|
Evan JarrettandClaude Opus 5
|
a63b839613 |
appview: gzip UI and static responses, leaving /v2 alone
Go's net/http compresses nothing by default and the appview is served by a
raw Go server behind a load balancer that does not compress either, so
everything went out uncompressed: style.css at 180 KB, bundle.min.js at
105 KB, and the homepage HTML at 95 KB. Lighthouse put style.css alone at
1,768 ms of blocked first paint, and mobile Performance measured 77 against
98 on desktop, entirely on paint metrics (TBT 10ms, CLS 0).
klauspost/compress is already a direct dependency and ships gzhttp, so this
costs no new one. Brotli would save roughly 11 KB more across the three
largest assets in exchange for a runtime dependency, which is not worth it.
The /v2 skip is the part that needs care. The OCI registry API is mounted on
the same chi router as the UI (server.go:561), and container layers are
already gzipped tarballs, so compressing that path burns CPU for no gain.
The content-type allowlist would catch most of it, but /v2/* also serves
application/json for tag listings and errors, so the path check keeps the
registry out of the compression path entirely. The predicate matches the one
the domain-routing middleware already uses.
Measured against a local build, gzip vs identity:
/ 14,526 -> 4,298 71%
/css/style.css 183,990 -> 31,596 83%
/js/bundle.min.js 107,781 -> 32,648 70%
/icons.svg 26,360 -> 8,403 69%
total 332,657 -> 76,945 77%
Verified end to end against a running binary: UI and static responses carry
Content-Encoding: gzip with Vary: Accept-Encoding, /v2/ and /v2/*/tags/list
carry neither, and woff2 stays untouched. Tests cover both directions plus
the under-1 KB and no-Accept-Encoding cases.
HTTP/2 is the remaining half and cannot be fixed here: the load balancer
terminates TLS and negotiates no ALPN at all, which pins every request to
HTTP/1.1. That is an LB setting.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124r73LT4qoFE82TqwH2Gu9
|
||
|
Evan JarrettandClaude Opus 5
|
cbd0c5f05c |
appview: rebuild the JS bundle so the committed asset matches src
The tracked bundle predated
|
||
|
Evan JarrettandClaude Opus 5
|
27fab41c1c |
appview: make the webhook cap agree between creation and delivery
The UI and the dispatcher asked the same question and got different
answers. getWebhookLimits short-circuited on a disabled billing manager and
returned unlimited without consulting it, while server.go hands
BillingManager.GetWebhookLimits straight to NewDispatcher, bypassing that
short-circuit entirely. With billing compiled out the stub answers
non-captains with (1, false).
So a non-captain saw "N / unlimited webhooks configured", could create as
many as they liked, and only the oldest was ever delivered. allTriggers was
false on that same path, so even the surviving one was restricted to
FreeTriggerMask; a webhook set to a scan trigger fired nothing at all, with
no message anywhere and only an INFO line server-side.
Route both paths through the manager so they cannot drift. The intended
non-billing policy is one webhook with TriggerFirst | TriggerPush |
TriggerQuota, which is what the stub already returned and what the shipped
config's Free tier specifies (max_webhooks: 1, webhook_all_triggers: false),
so enabling billing leaves free users exactly where they were and only
unlocks upward. That also removes a downgrade cliff: nobody can accumulate
webhooks under a phantom unlimited and lose them when billing turns on.
Drop the dead webhookLimits{Max: 1}, overwritten on the following line, and
give a nil manager the same policy rather than a third answer.
Latent, not live: production has zero webhooks configured today.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UAqi2hS2dhZoatqcWoYZQk
|
||
|
Evan JarrettandClaude Opus 5
|
62aea5f3f0 |
credhelper: print the verification URL that carries the code
The interactive prompt named codeResp.VerificationURI while openBrowser was handed verificationURL, the one with ?user_code= appended. Pressing Enter therefore always worked, which is why this went unnoticed; copying the printed URL instead landed on /device with no code. That matters more than it looks, because the branch tests the wrong thing. isTerminal(os.Stdin) asks whether stdin is a TTY, not whether a browser exists, so an SSH session on a headless box takes the headed path and is told to press Enter to open a browser it does not have. The non-interactive branch, which already printed the full URL, is only reached by piping stdin. Printing the code-carrying URL in both branches makes that mismatch moot rather than requiring a smarter predicate. Also give /device without a code the styled device-error page instead of bare text/plain, matching the expired-code path beside it, and stop renderError panicking when Templates is nil. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UAqi2hS2dhZoatqcWoYZQk |
||
|
Evan JarrettandClaude Opus 5
|
a219df9545 |
appview/js: fix webhook test result, 400 toasts, and Layers tab init
Three unrelated client-side defects found while baselining production. The webhook Test button always reported success. renderAlert writes no status code, so both outcomes are HTTP 200 and the result lives in the markup, which partials/alert.html emits as "alert alert-error". testWebhook looked for class="error", which that string does not contain, and resp.ok is always true, so the failure branch was unreachable. A webhook pointed at a dead URL was reported as delivered. Match alert-error instead. Avatar upload rejections lost the server's reason. The htmx:responseError handler maps status codes to fixed strings and had no 400 case, so "File too large (max 3MB)" and "Invalid file type" both surfaced as "Something went wrong". Surface the body when it is short plain text; a rendered error page or a long trace is not toast material. The repository page's Layers tab was never initialised. initLayersTables runs from DOMContentLoaded, when the panel is still a spinner, and from htmx:afterSettle, which htmx.process() does not emit. So empty-layer hiding and no-history run collapsing never ran there, and the checkbox claimed rows were hidden while all of them were on screen. Export it and call it from the tab controller. It now also re-seeds checkboxes within the loaded scope, which fixes the digest page contradicting itself: the stored preference was honoured for the rows and ignored for the control. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UAqi2hS2dhZoatqcWoYZQk |
||
|
Evan JarrettandClaude Opus 5
|
1253ca15ec |
appview: report a never-scanned image as unscanned, not as a failure
digest_content.go branched on Error == "never-scanned" to pick the "not scanned yet" copy, but nothing anywhere produced that string: both FetchVulnDetails and FetchSbomDetails returned the human sentence "No scan record found" for a missing record. So vulnReason and sbomReason could never be "not-scanned", the friendly branches in vulns-section.html and sbom-section.html were dead code, and every unscanned image fell through to fetch-failed. Free-tier accounts have scan_on_push off, so this was every image they push, told "Scan data couldn't be loaded... try refreshing in a minute" about something that had never been scanned and never would be by refreshing. The digest page showed the raw internal string instead. Replace the prose sentinel with a NotScanned bool the 404 path actually sets, and give other non-200 statuses a distinct message so a 500 from the hold stops being indistinguishable from an absent record. The detail templates branch on it before Error, so nothing leaks the internal value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UAqi2hS2dhZoatqcWoYZQk |
||
|
Evan JarrettandClaude Opus 5
|
606338b33e |
appview: fix mockup-code contrast in both themes
daisyUI dims the gutter prefix with `opacity: .5`, which multiplies against whatever opacity the line's own text color carries. On bg-base-300 a plain `$` measured 3.20:1 in light and 4.19:1 in dark, and the `#` on a line already dimmed to /70 compounded to 2.15:1. The text needs 4.5:1, so this failed in dark mode too, just less visibly than the light-mode report that surfaced it. Give the prefix an absolute muted color instead of a multiplying opacity. The override has to sit unlayered: daisyUI ships this selector in `@layer daisyui`, declared after `@layer components`, so a rule in components loses on layer order however specific it is. A first attempt inside components left daisyUI's `opacity: .5` live on top of the new 70% color, which made the `$` worse (0.5 -> 0.35 effective) rather than better. The hero tagline drops its /70 and now reads at the same weight as the docker commands above it. install.html's two comment lines were at /50, which failed on the text itself (3.20:1 light), and move to /70 where they still read as comments. Measured in Chromium against the built stylesheet, compositing each pseudo-element color over its real background on a canvas: every prefix and comment is now 5.84:1 light / 6.71:1 dark, against 14.03:1 / 12.02:1 for the command text, so the gutter stays visibly de-emphasized. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LpqkSRqpcnAjsSZgFTUN5z |
||
|
Evan JarrettandClaude Opus 5
|
ee9ccc1fb8 |
appview: drop crossorigin from the imgs.blue preconnect
Connections are keyed by credentials mode. The crossorigin attribute opened the imgs.blue socket in anonymous-CORS mode, but every request to that origin is a plain <img src> avatar fetch (BlobCDNURL / resizeImage) in no-cors mode, so nothing could reuse it. The browser opened a second connection anyway and Lighthouse flagged the hint as unused while still listing imgs.blue as a preconnect candidate worth ~300ms of LCP. The attribute is correct for the font preloads below, which is where it was likely copied from; comment the distinction so it stays put. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LpqkSRqpcnAjsSZgFTUN5z |
||
|
Evan JarrettandClaude Opus 5
|
521cf143e5 |
appview: cover the half of 13edb71 that had no tests
That commit throttles two writes and states a statement order. Only one of the three claims was defended. touchLastSeen is the hotter of the two writes — it ran once per indexed record, so a busy firehose meant a database round trip per event for a timestamp read in hours or days. Deleting its throttle outright left every existing test green, as did keying it globally instead of per DID, which would let one busy account suppress every other account's first write. Both now fail. The statement order is the third claim: UpdateLastUsed stamps the throttle before the write rather than after, so a slow or failing write cannot let every concurrent caller through to queue another attempt behind it. That matters because this runs on the authentication path, once per layer during a push, and the pile-up is worst exactly when the database is least able to absorb it. A failing write makes the ordering observable without timing anything: with the stamp after the write every call retries, with it before only the first does. Dropping the table leaves no row to inspect, so the attempts are counted through the warning the function already logs. Under the reordering it reports 10 attempts across 10 calls. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011rmjvU2gSRL9wFnmqVsWaF |
||
|
Evan JarrettandClaude Opus 5
|
ff0942196e |
db: cover the orphan drop in 0034, the case production actually presents
TestMigration0034PreservesLayersAndReferences seeds no orphan and says why: the live foreign key refuses to create one, so the migration's join through manifest_id is "insurance for a database whose foreign keys were off at some point, not for anything reachable now". Production is that database. It carries 160 layers and 22 manifest_references pointing at manifests.id values that no longer exist, left by deletes performed under mattn/go-sqlite3, where the constraint the DDL declared was not enforced. libSQL turns foreign keys on by default and mattn did not, so the rows predate the driver swap. The insurance is load-bearing on the only database that matters, and nothing tested it. The new case rebuilds the pre-0034 shape with the child foreign keys absent, which is what that era's schema behaved like, and seeds three orphaned layers and two orphaned references beside live ones. It asserts the orphans are gone, the live rows survive attached to the right key, nothing lands keyless, and foreign_key_check is clean afterwards. Verified against the defect: with the child manifest_key made nullable and the joins turned into LEFT JOINs, all five orphans survive and the test fails on both counts. Recorded honestly, the two guards are redundant with each other — LEFT JOIN alone still drops them, because INSERT OR IGNORE swallows the NOT NULL violation. Only removing both carries an orphan forward, and such a row counts, selects, and joins to no manifest ever again. Confirmed on a copy of the production database: layers 18803 -> 18643 and manifest_references 2175 -> 2153, exactly the rows the join excludes, with manifests unchanged at 3864. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011rmjvU2gSRL9wFnmqVsWaF |
||
|
Evan JarrettandClaude Opus 5
|
8c9a85826d |
appview: stop leasing the billing tier refresh, and let it be cancelled
|
||
|
Evan JarrettandClaude Opus 5
|
f0b28c04f5 |
db: cover the DID guard on the batched tag delete
TestDeleteTagsNotInListScopedToDID passes a nil keep list, which takes the
early return and deletes with `DELETE FROM tags WHERE did = ?`. So the scoping
it proves is the shortcut's, not the one inside the chunk loop that
|
||
|
Evan JarrettandClaude Opus 5
|
fa6473a896 |
appview/holdclient: cover the tier fan-out itself, and its failure path
The existing tests covered updateCrewTierWithRetry and UpdateCrewTierOnHold. UpdateCrewTierOnAllHolds -- the function the Stripe webhook actually calls, and whose error decides whether a paid upgrade is retried or dropped -- had none. Three cases: the joined error names every failing hold and not the one that succeeded; a hold that accepts and never answers does not starve the holds after it (mutation-verified by making the fan-out serial, which leaves the healthy hold contacted zero times); and a context deadline aborts the retry loop rather than running to tierUpdateMaxAttempts. That last one records a real mismatch rather than an intent. Three attempts at a 5s client timeout need ~15s, and the webhook allows the whole fan-out 10s, so under a hang the budget funds two attempts and never three -- confirmed against a blackholed hold on the dev stack, which failed at exactly 10.0s with a bare context error rather than the "after N attempts" wrapper. If either constant or the deadline moves, that test is where the arithmetic gets re-checked. Also covers the other half in pkg/billing: a fan-out failure has to reach Stripe as a 5xx and leave stripe_processed_events empty. A hold that is briefly down otherwise costs the customer their tier permanently -- the same shape of loss as the customer-lookup hole, one layer further out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwxF2N3HuZ8xSkx6nkirgB |
||
|
Evan JarrettandClaude Opus 5
|
5112425673 |
appview: make the footer Bluesky link configurable
The link was hardcoded before |
||
|
Evan JarrettandClaude Opus 5
|
17e25a4df2 |
appview: guard the tag-listing paging that gates a shared-record delete
The delete path decides whether an io.atcr.manifest record is still wanted by
enumerating the DID's tag records. That enumeration is load-bearing in a way a
tag listing usually is not: records for every one of a DID's repositories share
one collection, so a single page is a per-account budget rather than a per-repo
one. Past it a live tag falls off the end, the digest reads as unreferenced,
and the shared record is deleted out from under a repository nobody touched —
along with its layers on the hold.
Two cases, both mutation-verified:
* the tag that keeps the digest alive sits on page 3. Stopping after the
first page deletes the record. cleanupUntaggedManifest has carried this
hazard in a comment since
|
||
|
Evan JarrettandClaude Opus 5
|
0b212a527f |
appview: guard the two UI delete paths against cross-repo destruction
Same defect as the OCI path, two more places. The io.atcr.manifest record is
keyed by digest alone, so one record backs every repository of a DID holding
identical content, and both of these deleted it without asking whether another
repository still wants it — then purged the layers on the hold.
DeleteManifestHandler already removes this repository's tags before deleting
the record, so a tag remaining at that point can only belong to another
repository. It now checks IsManifestTaggedAnyRepo there and keeps the shared
record when one does, reporting sharedRecordKept so the caller can tell the
difference between "deleted" and "deliberately left alone".
DeleteUntaggedManifestsHandler is the subtler one. Its digest list comes from
GetAllUntaggedManifestDigests, whose tag join is scoped to one repository
(m.repository = t.repository), so a digest tagged only in a DIFFERENT
repository is reported as untagged and swept. The query is a reasonable
per-repository view and a dangerous delete list; the guard goes at the delete,
not in the query, matching how DeleteTagHandler already works. Skips are
counted separately from failures, because a skip is the guard working and
folding it into "failed" would make a correct run look broken.
Both fail closed. Leaving a manifest behind is recoverable; deleting one
another repository is still serving is not.
The new db test pins both halves of the interaction: that the query really does
report a cross-repo-tagged digest as untagged, so a change there is noticed,
and that IsManifestTaggedAnyRepo answers DID-wide, which is the thing actually
standing between the sweep and another repo's live image.
DeleteTagHandler needed no change —
|
||
|
Evan JarrettandClaude Opus 5
|
701c866723 |
appview: stop an OCI manifest delete destroying another repo's image
The io.atcr.manifest record is keyed by digest alone (digestToRKey), so one
record backs every repository of a DID holding identical content.
ManifestStore.Delete removed it unconditionally and then called
purgeDeletedManifest, which asks the hold to drop the layer records and free
the blobs. Deleting through one repository therefore stripped the manifest out
from under every other repository sharing that digest and took their bytes with
it — reachable from the public OCI API with `crane delete`, and not recoverable.
Reproduced end to end before fixing: push identical content to shared-a and
shared-b, delete shared-a by digest, and shared-b:v1 answers 404.
The codebase already had the right guard and the right policy written down.
|
||
|
Evan JarrettandClaude Opus 5
|
85a07d660a |
appview: give the O(n) bcrypt scan a guard that can actually fail
TestDeviceStore_ValidateDoesNotScanIndexedRows is documented as "the
regression guard for the O(n) bcrypt scan", but it does not observe whether a
scan happened. It asserts that every row is indexed and that an unknown secret
errors, and both hold with or without the fix. Deleting the
`WHERE secret_lookup IS NULL` filter from the fallback query — which is the
defect
|
||
|
Evan JarrettandClaude Opus 5
|
8cd59a61f1 |
appview: stop a UI session outliving the OAuth session behind it
Only one oauth_sessions row is kept per account, so signing in again — on a
second device, or simply a second time — replaces it and leaves every earlier
ui_sessions row pointing at an oauth_session_id that no longer exists. Get
checked only expiry, so those still read back as usable.
Found on a live appview: four ui_sessions rows, three orphaned, and requesting
/settings/user with an orphaned cookie returned 200 with the account's handle
rendered throughout, where an anonymous request gets a 302. The browser looks
signed in while the credential behind it is gone, so every PDS-backed action
fails against a UI insisting the session is fine. It now fails closed and sends
the user back through login.
Get also never checked ownership. oauth_sessions is unique on
(account_did, session_id), so the existence check is scoped by both; matching
session_id alone would let one account's live OAuth session validate another
account's dangling reference. That has its own test.
An empty oauth_session_id stays valid, since Create makes sessions that never
had one, and a test pins that so the check cannot start rejecting them.
TestSessionStore_CreateWithOAuth referenced an OAuth session it never inserted,
which is an orphan by definition, so it now creates the row. Its intent was
that CreateWithOAuth persists the ID; it relied on the orphan behaviour only
incidentally. Its not-found branch used t.Error and then dereferenced the nil
session, so that is now t.Fatal.
Pre-existing at
|
||
|
Evan JarrettandClaude Opus 5
|
f8d9ad7fe9 |
appview: let a registry domain keep its port, and its /v2
DomainRoutingMiddleware normalized the request Host to a bare hostname but
matched server.registry_domains verbatim, so any configured domain carrying a
port could never match. config-appview.example.yaml ships
`registry_domains: [127.0.0.1:5000, atcr.io]`, which means that entry has been
inert since it was written.
It fails closed in the worst way. That same host is also the auto-detected UI
host, and `host == uiHost` was evaluated first, so /v2/* was answered with
"registry API is not available on this domain, use 127.0.0.1:5000" — naming the
exact host the client had just used. The registry API is unreachable on the dev
stack, and any single-host deployment hits the same wall: listing a host in
registry_domains does nothing if it is also the UI host.
Both sides are now normalized through hostWithoutPort, and a registry domain
takes /v2/* even when it doubles as the UI host, which is a legitimate
single-domain deployment. Everything else is unchanged: a UI-only host still
refuses /v2/, registry domains still redirect non-/v2 traffic to the UI, and
/auth/token and /auth/device/* are still served directly so a cross-host 307
cannot strip the Authorization header.
hostWithoutPort uses net.SplitHostPort instead of the previous LastIndex(":")
scan, which mangled bracketed IPv6 literals into "[::1" and could never match
the "::1" that url.URL.Hostname() yields for the UI host.
The middleware had no tests at all. The two failing cases are pinned first, and
the four pre-existing behaviours are pinned alongside them so the reorder
cannot quietly widen what /v2/ is served on.
Pre-existing at
|
||
|
Evan JarrettandClaude Opus 5
|
4c04983e23 |
appview: stop the backfill claiming every user was just active
last_seen means "this user did something recently". The backfill walks every historical record in the network, so stamping it there recorded when the backfill ran, not when the user was active — for every user at once, on every run. That destroys the only signal the column carries, and it is the one column in users that nothing upstream can rebuild. It is now written on the two paths that represent real activity: an interactive login, and a live commit event on the firehose, which does mean the user just wrote a record. The backfill still corrects handle, PDS endpoint and avatar, which is why it re-resolves rather than trusting a cache; it just no longer claims the user was present. UpsertUser grows an options form rather than a fourth named variant, since the avatar and last_seen decisions are independent and all four combinations occur. Anyone computing MAU from this column should know it was unreliable for every backfill run before this change. Also corrects docs/HORIZONTAL_SCALING.md, which claimed oci_client and registry_domain were local-only preferences. They are fields on io.atcr.sailor.profile: settings writes them to the user's PDS and ProcessSailorProfile refreshes the local cache. users is fully derived apart from last_seen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Evan JarrettandClaude Opus 5
|
13edb7184d |
appview: stop writing last_seen and last_used on every event
Neither is a correctness problem; both are round trips on hot paths for timestamps nothing reads at that resolution. UpdateUserLastSeen ran per Jetstream event for cached users, so once per indexed record. DeviceStore.UpdateLastUsed ran per /auth/token call, so once per docker push and pull including each layer's re-auth. Cheap against a local file, a network round trip each against a remote primary, and the second sat on the authentication path. Both are now throttled to once per five minutes per subject. The MAU queries and the admin views work in hours or days, so nothing loses meaning. The throttle state is per-process and lost on restart, costing at most one extra write per subject per boot; only the lease holder runs the consumer, so exactly one process is doing the first of these at a time. UpdateLastUsed stamps the throttle before writing rather than after, so a slow or failing write cannot let every concurrent layer upload through to pile on more of them. Verified by disabling the throttle: 50 back-to-back calls then rewrite the timestamp every time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Evan JarrettandClaude Opus 5
|
e9d43aa767 |
db: cover the recency change on all four surfaces, not one
The MAX(id) replacement touched four queries that each carry their own copy of the same CTE: SearchRepositories, GetRepoCards, GetUserRepoCards and GetStarredRepoCards. Only the third had a test. Fixing one and missing another would leave the UI disagreeing with itself about which manifest is current, depending on which page you were looking at. All four now assert that recency follows created_at rather than insert order, and that a tie between manifests pushed in the same second resolves the same way on every surface. Verified by regressing the CTEs back to rowid ordering: each of the four fails independently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Evan JarrettandClaude Opus 5
|
454a6bad3d |
db: key manifests by manifest_key and drop the rowid
Completes the swap 0033 set up. layers and manifest_references move onto manifest_key and manifests.id is gone, which removes the last node-allocated identifier in the AppView schema. Statement order in 0034 is load-bearing. With foreign keys on, DROP TABLE performs an implicit DELETE FROM, so dropping manifests while layers still holds an ON DELETE CASCADE reference deletes every layer row. Migration 0009 did exactly that; it went unnoticed because the Jetstream backfill rebuilds layers from PDS records, so the damage healed itself. PRAGMA foreign_keys is no help: it is a no-op inside a transaction and migrations run in one. So the new children are built pointing at manifests_new, the old children are dropped first, and only then is the old manifests table dropped, by which point nothing references it. Verified both behaviors before relying on them. manifest_key is declared NOT NULL as well as PRIMARY KEY, because in SQLite a PRIMARY KEY column still accepts NULL unless it is INTEGER PRIMARY KEY. That constraint immediately caught four test helpers inserting manifests without one. Five queries used MAX(id) as "the newest manifest in this repo", which I had previously reported as absent after grepping only for ORDER BY. A derived key has no ordering, so recency now comes from created_at with manifest_key as a deterministic tiebreak. This is a real behavior change, and a fix: the two disagree whenever a manifest is indexed out of order, which the backfill does routinely, and created_at is the push time these queries always wanted. Both directions are tested, including that ties resolve the same way every run. InsertManifest and BatchInsertManifests no longer read anything back. The key is derived from (did, repository, digest), so the writer knows it before the statement runs: the select-back, its per-DID IN list, and the "manifest missing id after batch insert" branch all go away, along with the UNIQUE-conflict fallback that existed only to recover a rowid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Evan JarrettandClaude Opus 5
|
11b85e5102 |
db: fill manifest_key inside its migration, not at runtime
0033 added the column and left the fill to the Jetstream backfill. That is fine for a running system and wrong for replay: a database several releases behind runs every pending migration back-to-back at boot, long before any worker starts. Anything built on top of manifest_key would, on that path, silently operate on NULLs while working perfectly on a system that had been up for a while. Establishing a migration's data precondition out-of-band means replay cannot see it. The runner now supports a Go step per migration version, running inside the same transaction as that migration's SQL, after the DDL it depends on and before the version is recorded. A version is never recorded without its Go half. 0033's step fills manifest_key for every row lacking one. It has to be Go: the value is a truncated sha256 and SQLite has no hash builtin. It pages through the table and writes one UPDATE ... CASE per 500 rows, because a statement per row would be correct and unusably slow against a remote primary. Verified by unregistering the hook: replay then leaves 3 of 3 seeded manifests with NULL keys. With it, all three are filled, from a snapshot of the pre-0009 schema forward. The upserts keep their "OR manifests.manifest_key IS NULL" clause. It is a self-healing net for rows that somehow arrive without a key rather than the mechanism anything depends on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |