Evan JarrettandClaude Fable 5.1 56bde61555 appview: keep one part upload in flight while the next buffer fills
A large layer went up strictly one step at a time: fill 16MB from the
client, stop reading, fetch a part URL and PUT the part to S3, reset,
resume reading. While the part was in flight Docker sat on a full TCP
window; while the buffer filled S3 sat idle. Wall clock was receive
time plus send time.

The writer now hands a full buffer to a goroutine that does the hold
call and the PUT, and keeps filling a second buffer from the client.
When that one fills it waits for the previous part, takes its buffer
back, and hands the new one off. At most one part is in flight, so part
numbers and ETags stay ordered, and a blob that never fills a buffer
never allocates the second one. No network runs under the writer lock
on the happy path.

Peak memory for a large upload is now two buffers, 32MB. Both are
charged to the process budget through the existing accounting, the
second as it grows, and the budget floor rises to match so a large
upload can never be refused outright. The 512MB default holds sixteen.

A failed part records a sticky error, closes the writer, and aborts the
multipart from the goroutine that still holds the upload ID; the next
Write, hand-off, or Commit reports the cause. Commit verifies the digest
first, then waits for the flight, sends the final part, and completes.
Cancel waits for the flight, bounded, before aborting so the abort
cannot overtake a PUT that has not yet been issued its upload ID. The
sweeper refuses to reap a writer with a part in flight, since last
activity is only stamped when a part lands.

Tests observe the overlap directly: the fake S3 blocks the first PUT and
the second buffer's writes are asserted to return before it is released,
while the third buffer's writes block. Also covered: the one-part-late
error, Commit waiting, Cancel during flight, peak budget, the sweeper,
and concurrent Cancel and Write under the race detector. The
integration suite passed with a 72MB layer pushed through the pipeline
by three clients.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yf1ZVA7sXYhQNb9tCo1m5
2026-09-09 20:52:52 -05:00
2026-05-03 14:51:05 -05:00
2026-01-18 17:44:15 -06:00
2026-01-06 23:56:17 -06:00
2026-05-09 21:21:20 -05:00
2026-06-14 04:02:43 +03:00
2025-11-02 22:11:19 -06:00

ATCR - ATProto Container Registry

https://atcr.io

An OCI-compliant container registry that uses the AT Protocol for manifest storage and S3 for blob storage.

What is ATCR?

ATCR integrates container registries with the AT Protocol ecosystem. Container image manifests are stored as ATProto records in your Personal Data Server (PDS), while layers are stored in S3-compatible storage.

Image names use your ATProto identity:

atcr.io/alice.bsky.social/myapp:latest
atcr.io/did:plc:xyz123/myapp:latest

Architecture

Three components:

  1. AppView - Registry API + web UI

    • Serves OCI Distribution API (Docker push/pull)
    • Resolves handles/DIDs to PDS endpoints
    • Routes manifests to user's PDS, blobs to hold services
    • Web interface for browsing/search
  2. Hold Service - Storage service with embedded PDS (optional BYOS)

    • Each hold has a full ATProto PDS for access control (captain + crew records)
    • Identified by did:web (e.g., did:web:hold01.atcr.io)
    • Generates presigned URLs for S3/Storj/Minio/etc.
    • Users can deploy their own storage and control access via crew membership
  3. Credential Helper - Client authentication

    • ATProto OAuth (DPoP handled transparently)
    • Automatic authentication on first push/pull

Storage model:

  • Manifests → ATProto records in user's PDS (small JSON, includes holdDid reference)
  • Blobs → Hold services via presigned S3 uploads, direct PUT for small blobs and XRPC multipart for large ones (stored in S3/etc.)
  • AppView uses service tokens to communicate with holds on behalf of users

Features

  • OCI-compliant - Works with Docker, containerd, podman
  • Decentralized - You own your manifest data via your PDS
  • ATProto OAuth - Secure authentication (DPoP-compliant)
  • BYOS - Deploy your own storage service
  • Web UI - Browse, search, star repositories
  • Multi-backend - S3, Storj, Minio, Azure, GCS, filesystem

Quick Start

Using the Registry

1. Install credential helper:

curl -fsSL https://atcr.io/static/install.sh | bash

2. Configure Docker (add to ~/.docker/config.json):

{
  "credHelpers": {
    "atcr.io": "atcr"
  }
}

3. Push/pull images:

docker tag myapp:latest atcr.io/yourhandle/myapp:latest
docker push atcr.io/yourhandle/myapp:latest  # Authenticates automatically
docker pull atcr.io/yourhandle/myapp:latest

See INSTALLATION.md for detailed installation instructions.

Running Your Own AppView

# Build
go build -o bin/atcr-appview ./cmd/appview

# Generate a config file with all defaults
./bin/atcr-appview config init config-appview.yaml
# Edit config-appview.yaml — set server.default_hold_did at minimum

# Run
./bin/atcr-appview serve --config config-appview.yaml

Using Docker:

docker build -f Dockerfile.appview -t atcr-appview:latest .
docker run -d -p 5000:5000 \
  -v ./config-appview.yaml:/config.yaml:ro \
  -v atcr-data:/var/lib/atcr \
  atcr-appview:latest serve --config /config.yaml

See deploy/README.md for production deployment.

Running Your Own Hold (BYOS Storage)

See docs/hold.md for deploying your own storage backend.

Development

Building from Source

# Build all binaries
go build -o bin/atcr-appview ./cmd/appview
go build -o bin/atcr-hold ./cmd/hold
go build -o bin/docker-credential-atcr ./cmd/credential-helper

# Run tests
go test ./...
go test -race ./...

Project Structure

cmd/
├── appview/           # Registry server + web UI
├── hold/              # Storage service (BYOS)
├── credential-helper/ # Docker credential helper
├── oauth-helper/      # OAuth debug tool
├── healthcheck/       # HTTP health check (for Docker)
├── db-migrate/        # SQLite → libsql migration
├── usage-report/      # Hold storage usage report
├── record-query/      # Query ATProto relay by collection
└── s3-test/           # S3 connectivity test

pkg/
├── appview/
│   ├── db/            # SQLite database (migrations, queries, stores)
│   ├── handlers/      # HTTP handlers (home, repo, search, auth, settings)
│   ├── holdhealth/    # Hold service health checker
│   ├── jetstream/     # ATProto Jetstream consumer
│   ├── middleware/    # Auth & registry middleware
│   ├── ogcard/        # OpenGraph image generation
│   ├── readme/        # Repository README fetcher
│   ├── routes/        # HTTP route registration
│   ├── storage/       # Storage routing (blob proxy, manifest store)
│   ├── public/        # Static assets (JS, CSS, install scripts)
│   └── templates/     # HTML templates
├── atproto/           # ATProto client, records, manifest/tag stores
├── auth/
│   ├── oauth/         # OAuth client, refresher, storage
│   ├── token/         # JWT issuer, validator, claims
│   └── holdlocal/     # Local hold authorization
├── config/            # Config marshaling (commented YAML)
├── hold/
│   ├── admin/         # Admin web UI
│   ├── billing/       # Stripe billing integration
│   ├── db/            # Vendored carstore (go-libsql)
│   ├── gc/            # Garbage collection
│   ├── oci/           # OCI upload endpoints
│   ├── pds/           # Embedded PDS (DID, captain, crew, stats, scans)
│   └── quota/         # Storage quotas
├── logging/           # Structured logging + remote shipping
└── s3/                # S3 client utilities

License

MIT

Contributing

Contributions welcome! Please open an issue or PR.

S
Description
No description provided
Readme
136 MiB
Languages
Go 88.6%
HTML 6.1%
JavaScript 3.1%
CSS 0.9%
Shell 0.7%
Other 0.6%