Evan JarrettandClaude Opus 5
18c77ace28
Three scripts covering what only a browser can see, run end to end against the sandbox with a real checkout and a real portal cancellation. batch13-billing-drive.mjs runs the same account either side of one config change -- whether its default hold appears in server.managed_holds. Managed: the billing tab offers real tiers, checkout 302s to Stripe, the portal is reachable. Self-hosted: checkout 403s, the portal still 302s, and the advisor answers managed_hold_required rather than upgrade_required, which matters because telling a paying subscriber to "upgrade" would sell them a tier they already hold. batch13-portal-cancel.mjs walks into Stripe's portal instead of asserting the redirect, because the batch card calls a subscriber who cannot cancel the worst outcome here and a 302 does not prove a cancel control exists at the far end. batch13-webhook-downgrade.mjs creates three webhooks under an allowance of ten, then reads the page back after the downgrade. Every one of these is invisible on a hold owner's account: GetSubscriptionInfo returns a synthetic "Captain" tier before any Stripe lookup, and GetWebhookLimits / HasAIAdvisor / GetSupporterBadge bypass on the same first line. The first run used the shared e2e profile, which still had the owner signed in, and reported a clean pass built entirely on that bypass. The scripts now assert the page does not render "Captain", and take a separate profile. Two traps worth keeping: a bare button[type=submit] matches the nav's hidden logout button before the form's own submit, and hx-confirm here renders a custom modal whose backdrop swallows clicks -- strip the attribute rather than trying to dismiss a dialog that never fires. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwxF2N3HuZ8xSkx6nkirgB
Browser-driven batch validation
Checks for the val/* validation stack — the branch-per-batch series used to
sign off the range between the deployed commit and main.
These complement, and do not replace, go test and the in-process integration
harness (internal/testharness, test/integration, make integration-test).
Pick by what needs proving:
| Prove | Use |
|---|---|
| A function's logic, a query's shape, a guard's behaviour | Go unit test |
| A push/pull/delete round trip across appview + hold + S3 | make integration-test |
| A fragment renders into the right target, a job outlives its request, a 500 dressed as an empty state | these scripts |
Running
npm i -D @playwright/test && npx playwright install chromium
node test/e2e/login.mjs # interactive, once per hold rebuild
node test/e2e/batch00-admin-jobs.mjs # then the batch checks
Env overrides: ATCR_HOLD_URL, ATCR_APPVIEW_URL, ATCR_E2E_PROFILE,
ATCR_E2E_SEED.
Things that will cost you an afternoon
Use 127.0.0.1, never localhost. The appview canonicalises to
http://127.0.0.1:5000 and answers localhost with a 307. Any snippet written
against localhost:5000 measures the redirect, not the endpoint.
Admin sessions are in-memory and die on every hold rebuild. They live in
ui.sessions (pkg/hold/admin/admin.go), not the admin_sessions table, which
is vestigial for this path. Air rebuilds the hold whenever tracked source
changes — including a batch checkout — so budget one interactive login per
switch. There is no test-mode bypass; server.test_mode only affects OAuth
redirect URLs.
Never drive the admin panel with curl. Sessions are pinned to User-Agent and
client IP prefix, and a mismatch does not merely reject the request — it calls
deleteSession and logs you out. Drive everything through ctx.request, which
inherits the browser's cookie jar and UA.
Closing every Playwright page disposes ctx.request. It fails with "Request
context disposed". Keep one about:blank page open when the test needs the
browser out of the way.
Crew delete is a <button hx-post>, not a <form>. Scraping for forms
finds nothing, deletes nothing, and cheerfully reports a clean tab while every
seeded member is still live. Assert against page text after a reload, not
against the scrape that just ran.
Crew rows hydrate per-row via hx-trigger="load". The tab needs a real
settle window (~6s here) before anything is scrapeable.
A seeded fixture makes the second run lie. Crew import skips DIDs that already exist, so a re-run finishes instantly and the detachment check silently passes without ever exercising a running job. Purge before re-running.
Preconditions are easy to miss. The tier reconciliation card only renders
for crew on a tier absent from quota config (handleCrewList), so it is invisible
on a healthy hold. Crew add/update do not validate the tier against config,
which is how these tests manufacture the condition without restarting the hold.
The repo page is /r/{handle}/*. Not /{handle}/{repo} — that is a 404
"Lost at Sea" page, which reads exactly like an access denial if you are
checking whether a logged-out visitor gets denied. Verify the route before
concluding anything from a 404.
Tags are <option>s in a <select>. Scraping a, td or span for tag
text finds nothing and reports "no tags render" against a page that is
rendering them correctly.
Headed works; if it times out, retry before concluding otherwise. A headed
launchPersistentContext normally comes up in well under a second — the
display is reachable (DISPLAY=:0, XAUTHORITY set to the mutter XWayland
cookie, both the Wayland socket and /tmp/.X11-unix/X0 present; xdpyinfo
confirms it). Two launches did once hang for the full 180s handshake timeout
under heavy concurrent docker and test load, which is easy to misread as "headed
is impossible here" and switch everything to headless. It is not. Check
xdpyinfo and retry.
Logged-out checks need their own profile. lib.mjs's PROFILE is signed
in, and clearing its cookies costs an interactive re-login for everything else.
Use a throwaway launchPersistentContext dir instead of a fresh
chromium.launch().
The dev hold is public: false by default, so anything testing the
allowed half of anonymous pull is unreachable until you flip it. Do not edit
the captain record: hold_pds.go:335 reconciles captain.Public from config
on every boot, so an untracked docker-compose.override.yml setting
HOLD_SERVER_PUBLIC: "true" flips it, and deleting the file flips it back.
Never commit that file. Allow ~15s after the container comes up for the appview
to see the new value — a token minted too early is still judged against the old
one, which looks exactly like a failing test.
Per-batch stack switching
Use val-switch.sh. The appview DB migrates forward only, so older batch
code hits a schema from the future: batch 00 selects tags.id (dropped by
0032) and cannot write manifests.manifest_key (added NOT NULL by 0033/0034,
which kills every backfill insert). The script destroys and re-migrates the
appview volume so the DB matches the branch.
It deliberately does not touch atcrio_atcr-hold, which holds the hold's
did:web signing key and the CAR store — captain, crew, layer, stats and scan
records. Losing it means a new hold identity and every pushed layer gone.
docker-compose.yml is pinned to main throughout. a7c7db6 (batch 01) is
what makes the appview share the hold's netns so did:web:localhost%3A8080
resolves, and every compose-based batch needs it — including batch 00, which
lands before it. The file is dev-only, so pinning it is a fixture decision
rather than a change to what is under validation. Never commit it from a batch
branch; the script unstages it for you, because git checkout main -- <path>
stages what it restores.