Hold discovery in the registry middleware called getRecord on the
repository owner's PDS for every request under /v2/: every HEAD, POST,
PATCH, PUT and GET. A 10-layer push was 40 or more PDS round trips, and
it was the last per-request network call on the push path that had
nothing to do with moving bytes. Only two profile fields are used
there: the default hold and the auto-remove-untagged flag.
The users row already caches the default hold, written by the Jetstream
processor on every profile event and prefilled by the backfill, and the
auth gate already reads it from there. This makes the row a faithful
copy of what the registry needs and switches the middleware to it.
The auto-remove flag gets a nullable users column. NULL means the value
has never been learned; the processor writes 0 or 1 on every profile
event and never NULL. On a request whose row is missing or still NULL,
the middleware does one live fetch, uses it, and writes both fields
back, including a 0 for a user with no profile at all, so the fallback
runs at most once per user. A failed fetch writes nothing and uses the
appview default for that request, so a network error is never cached.
That single mechanism covers the minutes after a deploy while the
startup backfill fills the column, a brand-new user, and a user the
backfill has not reached.
The processor also stops returning early on an empty default hold,
which left a user who removed their custom hold pushing to it forever.
Empty is now written through and means the appview default, matching
what the auth gate already reads.
Tests count PDS requests with a test server: a populated row makes
none, a NULL row makes exactly one and then none, a missing profile is
cached as known, and a failed fetch degrades without writing. The
migration was applied to a fresh database and to one built from the
previous schema.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5
Every blob went through the multipart machinery: an S3 multipart started
on Docker's initial POST, a hold round trip per part, and a complete on
the hold that finished the multipart, HEADed the temp object, copied it
to its final key, and deleted the temp. For a 2KB config blob that was
three hold calls and six S3 operations. On production data 86% of
distinct layers and every config blob fit in a 16MB buffer, and 49% of
image manifests have no layer larger than that.
The writer now buffers up to 16MB (also the multipart part size) and
makes no hold call until it has to. A blob that never overflows the
buffer is written at Commit with a single presigned PUT to its final
key, via the hold's existing method=PUT presign; the multipart only
starts on the first flush. The hold's completeUpload does nothing the
direct path skips: quota, layer records, stats and scan dispatch all
hang off notifyManifest, which is unchanged.
The buffer starts empty and grows on demand, with the doubling capped so
capacity never overshoots 16MB: a config blob costs kilobytes, and only
layers that approach the threshold fill it.
Bytes are hashed as they arrive. Commit compares the computed sha256 to
the digest the client claimed before any network call, and returns
DIGEST_INVALID on mismatch, aborting a multipart if one was started.
Previously nothing verified the content, so a pusher could store wrong
bytes under a digest in the shared content-addressed space.
Tests observe request counts on a fake hold and fake S3 rather than
return values. The growth test streams in 24KB chunks because
power-of-two chunks land on 16MB by luck and hid an earlier weaker guard.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5
Design for moving push bandwidth off the appview and onto the hold,
mirroring the pull path's 307 redirect. Proposal only, not implemented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evan JarrettandClaude Fable 5.1