Evan JarrettandClaude Opus 5 265e533ad3 scanner: reclaim scan directories a killed process left behind
buildOCILayout already removes its scan dir on every error path, and syft.go
defers the stereoscope generator's Cleanup. What neither can do is clean up
after a process that dies mid-scan: the deferred call never runs, and nothing
afterwards ever looks at what was left. Every restart therefore leaks the
in-flight layout and extraction permanently, and a restart is routine — a
deploy is one.

On seamark-hold that reached 8.8 GB of orphaned scan-*, syft-scan-* and
syft-cataloger-* directories under a 20 GB disk, at which point the disk was
97% full and scans began failing on it:

  failed to load OCI image: unable to populate layer cache
    dir="/var/lib/seamark/scanner/tmp/syft-scan-1546187834/..."
    : no space left on device

  failed to download layer 5: failed to write blob:
    write /var/lib/seamark/scanner/tmp/scan-4160414849/blobs/sha256/...
    : no space left on device

The leaked directories cluster at the scanner's restart timestamps, which is
what identifies the killed process rather than the error paths as the source.
Manual removal reclaimed 8.8 GB and took the disk from 97% to 50%.

Startup is where this belongs: it is the one moment the previous process is
known to be gone, and it is immediately after the event that caused the leak.
The sweep runs in WorkerPool.Start after TMPDIR is set and before any worker
can dequeue, so nothing it removes can be work in progress here.

Three constraints shape what it will touch:

  - Only the three per-job prefixes, only as direct children, only
    directories. The Grype database lives beside the tmp dir at
    <parent>/vulndb and go-getter unpacks into grype-dl underneath it; both
    are state the scanner needs and neither matches a prefix. The prefixes now
    have one definition each, used by both the creator and the sweeper, so
    renaming a directory cannot silently take it out of the sweep's scope.

  - An age threshold, vuln.sweep_max_age, default 1h. A second scanner sharing
    the directory has an in-flight scan-* dir that is minutes old, and
    scanner.job_timeout is 8m, so an hour clears both with room to spare. 0
    disables the sweep rather than removing a peer's live work.

  - Nothing is fatal. A stat or removal failure is a WARN and the sweep moves
    on, so a permission problem in the tmp dir cannot keep the scanner from
    starting.

The sweep only runs at startup, so a scanner that is killed twice between
deploys carries the first leak until its next restart. That is the tradeoff
for never racing a live peer; a periodic sweep would be the follow-up if
processes ever live long enough for it to matter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TA9D4DjaLZTvzQ7dJbu4eg
2026-09-08 21:59:58 -05:00
2026-05-03 14:51:05 -05:00
2026-01-18 17:44:15 -06:00
2026-01-06 23:56:17 -06:00
2026-05-09 21:21:20 -05:00
2026-06-14 04:02:43 +03:00
2025-11-02 22:11:19 -06:00
2026-04-23 10:38:55 -05:00

ATCR - ATProto Container Registry

https://atcr.io

An OCI-compliant container registry that uses the AT Protocol for manifest storage and S3 for blob storage.

What is ATCR?

ATCR integrates container registries with the AT Protocol ecosystem. Container image manifests are stored as ATProto records in your Personal Data Server (PDS), while layers are stored in S3-compatible storage.

Image names use your ATProto identity:

atcr.io/alice.bsky.social/myapp:latest
atcr.io/did:plc:xyz123/myapp:latest

Architecture

Three components:

  1. AppView - Registry API + web UI

    • Serves OCI Distribution API (Docker push/pull)
    • Resolves handles/DIDs to PDS endpoints
    • Routes manifests to user's PDS, blobs to hold services
    • Web interface for browsing/search
  2. Hold Service - Storage service with embedded PDS (optional BYOS)

    • Each hold has a full ATProto PDS for access control (captain + crew records)
    • Identified by did:web (e.g., did:web:hold01.atcr.io)
    • Generates presigned URLs for S3/Storj/Minio/etc.
    • Users can deploy their own storage and control access via crew membership
  3. Credential Helper - Client authentication

    • ATProto OAuth (DPoP handled transparently)
    • Automatic authentication on first push/pull

Storage model:

  • Manifests → ATProto records in user's PDS (small JSON, includes holdDid reference)
  • Blobs → Hold services via XRPC multipart upload (large binaries, stored in S3/etc.)
  • AppView uses service tokens to communicate with holds on behalf of users

Features

  • OCI-compliant - Works with Docker, containerd, podman
  • Decentralized - You own your manifest data via your PDS
  • ATProto OAuth - Secure authentication (DPoP-compliant)
  • BYOS - Deploy your own storage service
  • Web UI - Browse, search, star repositories
  • Multi-backend - S3, Storj, Minio, Azure, GCS, filesystem

Quick Start

Using the Registry

1. Install credential helper:

curl -fsSL https://atcr.io/static/install.sh | bash

2. Configure Docker (add to ~/.docker/config.json):

{
  "credHelpers": {
    "atcr.io": "atcr"
  }
}

3. Push/pull images:

docker tag myapp:latest atcr.io/yourhandle/myapp:latest
docker push atcr.io/yourhandle/myapp:latest  # Authenticates automatically
docker pull atcr.io/yourhandle/myapp:latest

See INSTALLATION.md for detailed installation instructions.

Running Your Own AppView

# Build
go build -o bin/atcr-appview ./cmd/appview

# Generate a config file with all defaults
./bin/atcr-appview config init config-appview.yaml
# Edit config-appview.yaml — set server.default_hold_did at minimum

# Run
./bin/atcr-appview serve --config config-appview.yaml

Using Docker:

docker build -f Dockerfile.appview -t atcr-appview:latest .
docker run -d -p 5000:5000 \
  -v ./config-appview.yaml:/config.yaml:ro \
  -v atcr-data:/var/lib/atcr \
  atcr-appview:latest serve --config /config.yaml

See deploy/README.md for production deployment.

Running Your Own Hold (BYOS Storage)

See docs/hold.md for deploying your own storage backend.

Development

Building from Source

# Build all binaries
go build -o bin/atcr-appview ./cmd/appview
go build -o bin/atcr-hold ./cmd/hold
go build -o bin/docker-credential-atcr ./cmd/credential-helper

# Run tests
go test ./...
go test -race ./...

Project Structure

cmd/
├── appview/           # Registry server + web UI
├── hold/              # Storage service (BYOS)
├── credential-helper/ # Docker credential helper
├── oauth-helper/      # OAuth debug tool
├── healthcheck/       # HTTP health check (for Docker)
├── db-migrate/        # SQLite → libsql migration
├── usage-report/      # Hold storage usage report
├── record-query/      # Query ATProto relay by collection
└── s3-test/           # S3 connectivity test

pkg/
├── appview/
│   ├── db/            # SQLite database (migrations, queries, stores)
│   ├── handlers/      # HTTP handlers (home, repo, search, auth, settings)
│   ├── holdhealth/    # Hold service health checker
│   ├── jetstream/     # ATProto Jetstream consumer
│   ├── middleware/    # Auth & registry middleware
│   ├── ogcard/        # OpenGraph image generation
│   ├── readme/        # Repository README fetcher
│   ├── routes/        # HTTP route registration
│   ├── storage/       # Storage routing (blob proxy, manifest store)
│   ├── public/        # Static assets (JS, CSS, install scripts)
│   └── templates/     # HTML templates
├── atproto/           # ATProto client, records, manifest/tag stores
├── auth/
│   ├── oauth/         # OAuth client, refresher, storage
│   ├── token/         # JWT issuer, validator, claims
│   └── holdlocal/     # Local hold authorization
├── config/            # Config marshaling (commented YAML)
├── hold/
│   ├── admin/         # Admin web UI
│   ├── billing/       # Stripe billing integration
│   ├── db/            # Vendored carstore (go-libsql)
│   ├── gc/            # Garbage collection
│   ├── oci/           # OCI upload endpoints
│   ├── pds/           # Embedded PDS (DID, captain, crew, stats, scans)
│   └── quota/         # Storage quotas
├── logging/           # Structured logging + remote shipping
└── s3/                # S3 client utilities

License

MIT

Contributing

Contributions welcome! Please open an issue or PR.

S
Description
No description provided
Readme
136 MiB
Languages
Go 88.6%
HTML 6.1%
JavaScript 3.1%
CSS 0.9%
Shell 0.7%
Other 0.6%