at-container-registry/test/e2e/README.md

Browser-driven batch validation

Checks for the val/* validation stack — the branch-per-batch series used to sign off the range between the deployed commit and main.

These complement, and do not replace, go test and the in-process integration harness (internal/testharness, test/integration, make integration-test). Pick by what needs proving:

Prove	Use
A function's logic, a query's shape, a guard's behaviour	Go unit test
A push/pull/delete round trip across appview + hold + S3	make integration-test
A fragment renders into the right target, a job outlives its request, a 500 dressed as an empty state	these scripts

Running

npm i -D @playwright/test && npx playwright install chromium

node test/e2e/login.mjs               # interactive, once per hold rebuild
node test/e2e/batch00-admin-jobs.mjs  # then the batch checks

Env overrides: ATCR_HOLD_URL, ATCR_APPVIEW_URL, ATCR_E2E_PROFILE, ATCR_E2E_SEED.

Things that will cost you an afternoon

Use 127.0.0.1, never localhost. The appview canonicalises to http://127.0.0.1:5000 and answers localhost with a 307. Any snippet written against localhost:5000 measures the redirect, not the endpoint.

Admin sessions are in-memory and die on every hold rebuild. They live in ui.sessions (pkg/hold/admin/admin.go), not the admin_sessions table, which is vestigial for this path. Air rebuilds the hold whenever tracked source changes — including a batch checkout — so budget one interactive login per switch. There is no test-mode bypass; server.test_mode only affects OAuth redirect URLs.

Never drive the admin panel with curl. Sessions are pinned to User-Agent and client IP prefix, and a mismatch does not merely reject the request — it calls deleteSession and logs you out. Drive everything through ctx.request, which inherits the browser's cookie jar and UA.

Closing every Playwright page disposes ctx.request. It fails with "Request context disposed". Keep one about:blank page open when the test needs the browser out of the way.

Crew delete is a <button hx-post>, not a <form>. Scraping for forms finds nothing, deletes nothing, and cheerfully reports a clean tab while every seeded member is still live. Assert against page text after a reload, not against the scrape that just ran.

Crew rows hydrate per-row via hx-trigger="load". The tab needs a real settle window (~6s here) before anything is scrapeable.

A seeded fixture makes the second run lie. Crew import skips DIDs that already exist, so a re-run finishes instantly and the detachment check silently passes without ever exercising a running job. Purge before re-running.

Preconditions are easy to miss. The tier reconciliation card only renders for crew on a tier absent from quota config (handleCrewList), so it is invisible on a healthy hold. Crew add/update do not validate the tier against config, which is how these tests manufacture the condition without restarting the hold.

The repo page is /r/{handle}/*. Not /{handle}/{repo} — that is a 404 "Lost at Sea" page, which reads exactly like an access denial if you are checking whether a logged-out visitor gets denied. Verify the route before concluding anything from a 404.

Tags are <option>s in a <select>. Scraping a, td or span for tag text finds nothing and reports "no tags render" against a page that is rendering them correctly.

A headed browser will not launch from an agent shell here. Both chromium.launch() and launchPersistentContext() time out on the handshake despite DISPLAY=:0 being set; headless works. Interactive flows (the admin login, docker-credential-atcr login) still need a human at a real browser — but non-interactive page checks can run headless, and batch10-anonpull.mjs does.

Logged-out checks need their own profile. lib.mjs's PROFILE is signed in, and clearing its cookies costs an interactive re-login for everything else. Use a throwaway launchPersistentContext dir instead of a fresh chromium.launch().

The dev hold is public: false by default, so anything testing the allowed half of anonymous pull is unreachable until you flip it. Do not edit the captain record: hold_pds.go:335 reconciles captain.Public from config on every boot, so an untracked docker-compose.override.yml setting HOLD_SERVER_PUBLIC: "true" flips it, and deleting the file flips it back. Never commit that file. Allow ~15s after the container comes up for the appview to see the new value — a token minted too early is still judged against the old one, which looks exactly like a failing test.

Per-batch stack switching

Use val-switch.sh. The appview DB migrates forward only, so older batch code hits a schema from the future: batch 00 selects tags.id (dropped by 0032) and cannot write manifests.manifest_key (added NOT NULL by 0033/0034, which kills every backfill insert). The script destroys and re-migrates the appview volume so the DB matches the branch.

It deliberately does not touch atcrio_atcr-hold, which holds the hold's did:web signing key and the CAR store — captain, crew, layer, stats and scan records. Losing it means a new hold identity and every pushed layer gone.

docker-compose.yml is pinned to main throughout. a7c7db6 (batch 01) is what makes the appview share the hold's netns so did:web:localhost%3A8080 resolves, and every compose-based batch needs it — including batch 00, which lands before it. The file is dev-only, so pinning it is a fixture decision rather than a change to what is under validation. Never commit it from a batch branch; the script unstages it for you, because git checkout main -- <path> stages what it restores.