Evan JarrettandClaude Fable 5.1
f4343d7956
Every blob went through the multipart machinery: an S3 multipart started on Docker's initial POST, a hold round trip per part, and a complete on the hold that finished the multipart, HEADed the temp object, copied it to its final key, and deleted the temp. For a 2KB config blob that was three hold calls and six S3 operations. On production data 86% of distinct layers and every config blob fit in a 16MB buffer, and 49% of image manifests have no layer larger than that. The writer now buffers up to 16MB (also the multipart part size) and makes no hold call until it has to. A blob that never overflows the buffer is written at Commit with a single presigned PUT to its final key, via the hold's existing method=PUT presign; the multipart only starts on the first flush. The hold's completeUpload does nothing the direct path skips: quota, layer records, stats and scan dispatch all hang off notifyManifest, which is unchanged. The buffer starts empty and grows on demand, with the doubling capped so capacity never overshoots 16MB: a config blob costs kilobytes, and only layers that approach the threshold fill it. Bytes are hashed as they arrive. Commit compares the computed sha256 to the digest the client claimed before any network call, and returns DIGEST_INVALID on mismatch, aborting a multipart if one was started. Previously nothing verified the content, so a pusher could store wrong bytes under a digest in the shared content-addressed space. Tests observe request counts on a fake hold and fake S3 rather than return values. The growth test streams in 24KB chunks because power-of-two chunks land on 16MB by luck and hid an earlier weaker guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5
9.5 KiB
Hold Service XRPC Endpoints
This document lists all XRPC endpoints implemented in the Hold service (pkg/hold/).
PDS Endpoints (pkg/hold/pds/xrpc.go)
Public (No Auth Required)
| Endpoint | Method | Description |
|---|---|---|
/xrpc/_health |
GET | Health check |
/xrpc/com.atproto.server.describeServer |
GET | Server metadata |
/xrpc/com.atproto.repo.describeRepo |
GET | Repository information |
/xrpc/com.atproto.repo.getRecord |
GET | Retrieve a single record |
/xrpc/com.atproto.repo.listRecords |
GET | List records in a collection (paginated) |
/xrpc/com.atproto.sync.listBlobs |
GET | List blob CIDs for an account |
/xrpc/com.atproto.sync.listRepos |
GET | List all repositories |
/xrpc/com.atproto.sync.getRecord |
GET | Get record as CAR file |
/xrpc/com.atproto.sync.getRepo |
GET | Full repository as CAR file |
/xrpc/com.atproto.sync.getRepoStatus |
GET | Repository hosting status |
/xrpc/com.atproto.sync.getLatestCommit |
GET | Current commit CID and revision |
/xrpc/com.atproto.sync.subscribeRepos |
GET | WebSocket firehose |
/xrpc/com.atproto.identity.resolveHandle |
GET | Resolve handle to DID |
/xrpc/app.bsky.actor.getProfile |
GET | Get actor profile |
/xrpc/app.bsky.actor.getProfiles |
GET | Get multiple profiles |
/xrpc/io.atcr.hold.listTiers |
GET | List hold's available tiers with quotas and features |
/.well-known/did.json |
GET | DID document |
/.well-known/atproto-did |
GET | DID for handle resolution |
Conditional Auth (based on captain.public)
| Endpoint | Method | Description |
|---|---|---|
/xrpc/com.atproto.sync.getBlob |
GET/HEAD | Get blob (routes OCI vs ATProto) |
getBlob response shape
The endpoint routes on the cid parameter and answers differently per branch.
ATProto blob (cid is a CID): 307 redirect to a presigned URL.
OCI blob (cid starts with sha256:): JSON.
{
"url": "https://s3.example.com/...?X-Amz-Signature=...",
"size": 27129344
}
| Field | Type | Notes |
|---|---|---|
url |
string | Presigned URL for the requested method (GET, HEAD or PUT). Always present on a 200. |
size |
int64 | Blob size in bytes. Present on GET and HEAD only, and omitted when the size could not be resolved. |
size lets the AppView build an OCI descriptor without a second round trip: ProxyBlobStore.Stat used to fetch a presigned HEAD URL here and then HEAD it against S3 purely to read Content-Length. It is resolved from the hold's records index (any layer with an io.atcr.hold.layer record) and falls back to a HeadObject (image config blobs, and layers whose manifest notification has not landed yet).
The field is additive. A client that reads only url behaves exactly as before, and an AppView that gets no size falls back to HEADing the presigned URL. size is never sent for a PUT presign: the object does not exist yet.
A method=PUT presign is a write capability (gated on blob:write, same as the multipart endpoints) and is how the AppView uploads a blob small enough to fit in its 16MB buffer: one presigned PUT to the blob's final sha256: key, instead of initiateUpload plus part URLs plus completeUpload plus the server side copy out of the temp key. The URL is signed with Content-Type: application/octet-stream, so the PUT has to carry that header or S3 rejects the signature.
On GET and HEAD, a blob that is not in storage is answered with 404 and a JSON error body instead of a presigned URL:
{
"error": "BlobUnknown",
"message": "blob not found"
}
Owner/Crew Admin Required
| Endpoint | Method | Description |
|---|---|---|
/xrpc/com.atproto.repo.deleteRecord |
POST | Delete a record |
/xrpc/com.atproto.repo.uploadBlob |
POST | Upload ATProto blob |
Inline Auth (per-manifest caller check)
/xrpc/io.atcr.hold.purgeManifest (POST) does not use a router middleware. Auth is validated inline by ValidateManifestPurger, which accepts either a Bearer service token or a DPoP token, then checks the caller's role:
- Hold captain (any manifest)
- Crew member with
crew:adminpermission (any manifest) - Crew member whose DID matches the manifest URI's DID (own manifests only)
Idempotent. Does not delete S3 blobs — GC handles those.
Auth Required (Service Token)
The requireAuth middleware validates Bearer service tokens only. requestCrew additionally accepts DPoP tokens when called directly (the handler falls back to ValidateDPoPRequest if no user is in context).
| Endpoint | Method | Description |
|---|---|---|
/xrpc/io.atcr.hold.requestCrew |
POST | Request crew membership (service token or DPoP) |
/xrpc/io.atcr.hold.exportUserData |
GET | GDPR data export (returns user's records; service token only) |
/xrpc/io.atcr.hold.deleteUserData |
DELETE | GDPR data deletion (deletes crew, layer, and stats records; service token only) |
Appview Token Required
/xrpc/io.atcr.hold.updateCrewTier (POST) validates the caller inline via ValidateAppviewToken. Returns 503 if the appview DID is not configured on the hold, or 401 on token validation failure.
| Endpoint | Method | Description |
|---|---|---|
/xrpc/io.atcr.hold.updateCrewTier |
POST | Update a crew member's tier (appview JWT, ES256) |
Scanner WebSocket
| Endpoint | Method | Description |
|---|---|---|
/xrpc/io.atcr.hold.subscribeScanJobs |
GET (WebSocket) | Scanner job subscription. Auth via ?secret= query param or X-Scanner-Secret header (shared secret). Supports ?cursor= for backfill, ?workers= to declare how many scans the process runs at once (default 1), and ?instance= to declare a stable process identity so a reconnecting scanner resumes its own in-flight jobs. |
OCI Multipart Upload Endpoints (pkg/hold/oci/xrpc.go)
All require blob:write permission via service token:
| Endpoint | Method | Description |
|---|---|---|
/xrpc/io.atcr.hold.initiateUpload |
POST | Start multipart upload |
/xrpc/io.atcr.hold.getPartUploadUrl |
POST | Get presigned S3 URL for a part; the client PUTs the part bytes directly to S3 |
/xrpc/io.atcr.hold.completeUpload |
POST | Finalize multipart upload |
/xrpc/io.atcr.hold.abortUpload |
POST | Cancel multipart upload |
/xrpc/io.atcr.hold.notifyManifest |
POST | Notify manifest push (creates layer records + optional Bluesky post) |
ATCR Hold-Specific Endpoints (io.atcr.hold.*)
| Endpoint | Method | Auth | Description |
|---|---|---|---|
/xrpc/io.atcr.hold.initiateUpload |
POST | blob:write | Start multipart upload |
/xrpc/io.atcr.hold.getPartUploadUrl |
POST | blob:write | Get presigned S3 URL for a part; client PUTs bytes directly to S3 |
/xrpc/io.atcr.hold.completeUpload |
POST | blob:write | Finalize multipart upload |
/xrpc/io.atcr.hold.abortUpload |
POST | blob:write | Cancel multipart upload |
/xrpc/io.atcr.hold.notifyManifest |
POST | blob:write | Notify manifest push/pull (creates layer records, increments stats, optional Bluesky post) |
/xrpc/io.atcr.hold.requestCrew |
POST | service token or DPoP | Request crew membership |
/xrpc/io.atcr.hold.exportUserData |
GET | service token | GDPR data export |
/xrpc/io.atcr.hold.deleteUserData |
DELETE | service token | GDPR data deletion (crew, layer, stats records) |
/xrpc/io.atcr.hold.getQuota |
GET | none | Get user quota info |
/xrpc/io.atcr.hold.getLayersForManifest |
GET | none | Get layer records for a manifest AT-URI |
/xrpc/io.atcr.hold.image.getConfig |
GET | none | Get OCI image config record for a manifest digest |
/xrpc/io.atcr.hold.purgeManifest |
POST | inline (service token or DPoP; captain, crew:admin, or manifest owner) | Purge layer/scan/image-config records for a single manifest URI. Called by appview on UI delete; called internally on takedown receipt. Does not delete S3 blobs (GC handles those). |
/xrpc/io.atcr.hold.listTiers |
GET | none | List hold's available tiers with quotas and features (scanOnPush) |
/xrpc/io.atcr.hold.updateCrewTier |
POST | appview token (ES256 JWT; 503 if appview DID not configured) | Update crew member's tier |
/xrpc/io.atcr.hold.subscribeScanJobs |
GET (WebSocket) | shared secret (?secret= or X-Scanner-Secret) |
Scanner job subscription; supports ?cursor= for backfill, ?workers= for concurrency, ?instance= for reconnect resumption |
Standard ATProto Endpoints (excluding io.atcr.hold.*)
| Endpoint |
|---|
| /xrpc/_health |
| /xrpc/com.atproto.server.describeServer |
| /xrpc/com.atproto.repo.describeRepo |
| /xrpc/com.atproto.repo.getRecord |
| /xrpc/com.atproto.repo.listRecords |
| /xrpc/com.atproto.repo.deleteRecord |
| /xrpc/com.atproto.repo.uploadBlob |
| /xrpc/com.atproto.sync.listBlobs |
| /xrpc/com.atproto.sync.listRepos |
| /xrpc/com.atproto.sync.getRecord |
| /xrpc/com.atproto.sync.getRepo |
| /xrpc/com.atproto.sync.getRepoStatus |
| /xrpc/com.atproto.sync.getLatestCommit |
| /xrpc/com.atproto.sync.getBlob |
| /xrpc/com.atproto.sync.subscribeRepos |
| /xrpc/com.atproto.identity.resolveHandle |
| /xrpc/app.bsky.actor.getProfile |
| /xrpc/app.bsky.actor.getProfiles |
| /.well-known/did.json |
| /.well-known/atproto-did |
See Also
- DIRECT_HOLD_ACCESS.md - How to call hold endpoints directly without AppView (app passwords, curl examples)
- BYOS.md - Bring Your Own Storage architecture
- OAUTH.md - OAuth + DPoP authentication details