TestCarstoreConcurrentWritesTwoOpeners failed in CI on 2026-09-13 with
"database is locked" while the code was correct. It hammered the file with
five concurrent writers and asserted that no lock error surfaced within the
5 s busy_timeout, which on a loaded runner with every package testing in
parallel is a statement about the disk, not the code. The property it guards
(busy_timeout applied to every connection of every pool on a hold database,
c44a874) is now tested directly: one connection holds the write lock via
BEGIN IMMEDIATE, a second writer is shown to block rather than fail, and to
succeed once the lock is released. Both topologies are covered (the shared
OpenHoldDB pool, and a second opener on the same file, in both directions),
and a control shows a pool without busy_timeout fails immediately under the
same lock, so the passing tests are known to observe the mechanism.
The failure was also buried under the INFO lines every hold and PDS test
emits while booting. internal/testlog.Quiet swaps the default slog handler
for a discard handler unless the run is verbose or ATCR_TEST_LOGS is set,
and every package that produced that output now calls it from TestMain.
`go test` only shows a package's output when it fails, so this changes
nothing for passing runs and leaves a failing one readable.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hho5da4daoCoPBJ9tCrL7s
The hold declines to enqueue a scan when a pushed manifest has a subject, which
is how it means to skip attestations, signatures and other referrer artifacts.
The AppView never sent one: notifyHoldAboutManifest built mediaType, config,
layers and manifests, and #manifestInfo defined only those four. So the
condition was always true, the guard never fired, and every referrer artifact
was enqueued for scanning.
The AppView already parsed subject. NewManifestRecord unmarshals it into
ManifestRecord.Subject, and Put hands that same pointer to
notifyHoldAboutManifest. The value was in scope and simply never serialized, so
this is one missing marshal step rather than a missing parse.
Adds subject to the lexicon as a #blobInfo ref, and mediaType to #blobInfo,
which config has always sent and the hold has always parsed. That only makes the
schema honest about what is already on the wire.
Hoists the hold's anonymous request struct to a named type with IsMultiArch,
IsReferrer and HasScannableContent, so the predicate is written once and
testable without standing up a HoldPDS.
Both directions degrade safely. An older hold ignores the unknown key and
behaves exactly as today, so shipping the appview alone is harmless but achieves
nothing until the hold catches up. An older appview sends no subject, leaving
the manifest enqueued as before.
Complements dfd604b rather than duplicating it. That guard lives in the scanner
after a job is created and dispatched, and catches unscannable work from any
source including the hold's proactive discovery pass. This one stops the row
being created at all, which matters because the row that froze all scanning for
nine days was exactly such an attestation. One gap neither closes: an
attestation with tar-shaped layers pushed by an old appview.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDqoCE1j3njokkZ9b1C5n9
1. Removing distribution/distribution from the Hold Service (biggest change)
The hold service previously used distribution's StorageDriver interface for all blob operations. This replaces it with direct AWS SDK v2 calls through ATCR's own pkg/s3.S3Service:
- New S3Service methods: Stat(), PutBytes(), Move(), Delete(), WalkBlobs(), ListPrefix() added to pkg/s3/types.go
- Pull zone fix: Presigned URLs are now generated against the real S3 endpoint, then the host is swapped to the CDN URL post-signing (previously the CDN URL was set as the endpoint, which
broke SigV4 signatures)
- All hold subsystems migrated: GC, OCI uploads, XRPC handlers, profile uploads, scan broadcaster, manifest posts — all now use *s3.S3Service instead of storagedriver.StorageDriver
- Config simplified: Removed configuration.Storage type and buildStorageConfigFromFields(); replaced with a simple S3Params() method
- Mock expanded: MockS3Client gains an in-memory object store + 5 new methods, replacing duplicate mockStorageDriver implementations in tests (~160 lines deleted from each test file)
2. Vulnerability Scan UI in AppView (new feature)
Displays scan results from the hold's PDS on the repository page:
- New lexicon: io/atcr/hold/scan.json with vulnReportBlob field for storing full Grype reports
- Two new HTMX endpoints: /api/scan-result (badge) and /api/vuln-details (modal with CVE table)
- New templates: vuln-badge.html (severity count chips) and vuln-details.html (full CVE table with NVD/GHSA links)
- Repository page: Lazy-loads scan badges per manifest via HTMX
- Tests: ~590 lines of test coverage for both handlers
3. S3 Diagnostic Tool
New cmd/s3-test/main.go (418 lines) — tests S3 connectivity with both SDK v1 and v2, including presigned URL generation, pull zone host swapping, and verbose signing debug output.
4. Deployment Tooling
- New syncServiceUnit() for comparing/updating systemd units on servers
- Update command now syncs config keys (adds missing keys from template) and service units with daemon-reload
5. DB Migration
0011_fix_captain_successor_column.yaml — rebuilds hold_captain_records to add the successor column that was missed in a previous migration.
6. Documentation
- APPVIEW-UI-FUTURE.md rewritten as a status-tracked feature inventory
- DISTRIBUTION.md renamed to CREDENTIAL_HELPER.md
- New REMOVING_DISTRIBUTION.md — 480-line analysis of fully removing distribution from the appview side
7. go.mod
aws-sdk-go v1 moved from indirect to direct (needed by cmd/s3-test).
Evan JarrettandClaude Fable 5.1