mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-23 10:44:16 +00:00
Evan JarrettandClaude Fable 5.1
47a107058d
Each in-flight blob upload buffers up to 16MB, Docker pushes five layers at once per client, and nothing bounded the total. Writers also lived in the package-level map forever: a client that died mid-push left its writer, its buffer, and any hold-side S3 multipart session behind with no expiry. A process-wide budget (golang.org/x/sync semaphore, default 512MB, server.upload_buffer_budget_mb) now caps memory held in upload buffers. A writer charges its buffer's projected backing capacity before growing, so a config blob costs kilobytes and a full writer costs exactly one buffer, and releases once, on Commit, Cancel, or reap. A write that needs budget waits on the request's context with a five minute cap, outside the writer's lock so Cancel and the sweeper cannot queue behind it; that wait is backpressure on the client. The budget is clamped to at least one buffer so a single upload can never deadlock. A sweeper started with the other appview workers reaps writers idle past server.upload_idle_timeout (default 1h), aborting the hold-side multipart on a detached context and releasing the budget. It measures inactivity, not age, so a slow push is never reaped, and it skips a writer whose lock is held so it cannot race a live part upload. Write also gains a fix the budget made visible. It appended a whole chunk and checked afterwards, so the last chunk before a flush could land a few bytes past 16MB, which did not fit the backing array; bytes.Buffer doubled it to 32MB and Reset kept that for the rest of the upload. Only chunk sizes that tile 16MB exactly avoided it, and the network read loop promises no such thing. Every large layer could hold 32MB while the budget charged 16. Write now fills to exactly the threshold, flushes, and continues with the remainder, so capacity is pinned at 16MB for any chunk size, every part is exactly one buffer, and a single oversized Write streams through as parts instead of buffering whole. The test streams 24KB chunks across the boundary and fails against the old code with cap 33554432. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5