Dmitry VerkhoturovandGitHub
0b651dddd4
* Make backend tests wait on conditions instead of durations The backend workflow has a long tail of runs that fail once and pass on a rerun. Every one of them comes down to a test assuming an operation finishes within some duration rather than waiting for the state it needs. Three were reproducible and each was reproduced against the old code before being changed: TestServerAuthHooks minted a token that lived one second and never tested expiry, so a slow runner turned the first POST into a 401; TestServerApp_AnonMode saw "connection refused" because waitForHTTPServerStart returned silently after three seconds and left a later assertion to fail with something unrelated; TestFsStore_Cleanup slept 200ms against a 300ms ttl that Cleanup widens to 400ms with its commit grace, so roughly 100ms of stall collected an image meant to survive. Fixed sleeps before asserting on asynchronous work are replaced with polls on the condition itself, using require.Eventually and require.EventuallyWithT, and require.Never where the assertion is that something did not happen. Polling closures assert on the CollectT they are handed rather than on t, since testify runs them on another goroutine, and polls that issue HTTP requests stay under the rate limit on the routes they poll through. Where a test needs time to have passed, the clock input is pinned instead: staging ages are stamped with os.Chtimes on both sides of the cleanup boundary right before each call, which also makes the 100ms commit grace an exact case rather than something no assertion reaches, and the RSS tests set store.Comment.Timestamp explicitly rather than racing the wall clock into the first 100ms of a second so pubDate matches. chooseUnusedPort takes a port from the kernel's ephemeral range. Picking at random out of a fixed 10000-port window let two package binaries, which go test ./... runs concurrently, land on the same number between the probe closing and the server binding. The start helpers fail naming the port they waited on, and the SSL tests wait on the redirect port as well as the TLS one. Arbitrary budgets that nothing tests are gone: ten HTTP clients with a one-second timeout against bolt-backed import and export, the "should take about 100msec" assertions, and a one-second bound on noticing an already cancelled context. Shutdown stays bounded at ten seconds so a hang is still caught. Two assertions get stronger. TestServerAuthHooks accepted 403 or 401 from a blocked user, an alternative that existed only because the short token could expire mid-test; it is deterministically 403 now. TestAdmin_BlockedList asserted two users blocked while one carried the same 150ms ttl the next step waits to lapse, so the halves raced each other. goleak stops reporting the regexp2 clock goroutine, which chroma pulls in for syntax highlighting and which lives for up to a second after the last match with a timeout; it ends on its own but a binary finishing inside that window was reported as leaking, and this suite now finishes sooner. The ignore for net/http.(*Server).Shutdown goes the other way: it no longer matches anything, with both packages run fifteen times each under CPU oversubscription to confirm. Two gaps the change would otherwise have opened are covered directly rather than left to the side effects that used to cover them. The one-second token was the only thing exercising the authenticator's ClaimsUpd hook on refresh, so TestServerApp_ClaimsUpd now calls the hook itself and checks admin, blocked, email and restricted-name impersonation, including the two pass-through cases. Lifting the open-route limit removed the last incidental exercise of the rate limiter, so TestRateLimiter drives a burst past the allowance and checks the refusals and that the limit is per client. Both run without a wall clock, and both were confirmed to fail when the behaviour they cover is removed. Production code is untouched. The two sleeps outside test code, the 429 backoff in cmd/cleanup.go and the submit poll in store/image/image.go, are left alone: no CI failure implicates them. Test sleeps drop from 67 to 21, all of them either inside a testing/synctest bubble or a poll interval. The suite runs in about 22 seconds instead of 46, mostly because TestPublic_FindCommentsCtrl_ConsistentCount no longer paces a hundred subtests with an 80ms sleep each to stay under the open route limit. The 300s per-package budget now matches across both workflows, the race_test target and the documented command, and CLAUDE.md records the convention. with '#' will be ignored, and an empty message aborts the commit. # # Date: Sat Aug 22 01:12:31 2026 +0100 # # interactive rebase in progress; onto7c312da1# Last command done (1 command done): # reword deb6cbf1 # Make backend tests wait on conditions instead of durations # Next command to do (1 remaining command): # reword 262e6dc2 # Apply go fix under Go 1.27 # You are currently editing a commit while rebasing branch 'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed: .github/workflows/release.yml # modified: CLAUDE.md # modified: Makefile modified: backend/_example/memory_store/server/rpc_test.go # modified: backend/app/cmd/import_test.go # modified: backend/app/cmd/server_test.go # modified: backend/app/main_test.go # modified: backend/app/rest/api/admin_test.go # modified: backend/app/rest/api/middleware_test.go # modified: backend/app/rest/api/migrator_test.go # modified: backend/app/rest/api/rest_private_test.go # modified: backend/app/rest/api/rest_public_test.go # modified: backend/app/rest/api/rest_test.go # modified: backend/app/rest/api/rss_test.go # modified: backend/app/rest/proxy/image_test.go # modified: backend/app/store/image/fs_store_test.go # modified: backend/app/store/service/service_test.go # modified: docs/backlog/api-tests-deadlock-on-macos.md # * Apply go fix under Go 1.27 Go 1.27 extends go fix with the modernizers, so `go fix ./...` now rewrites patterns the language has since replaced. Running it across all three modules produces this: legacy sync/atomic calls on plain integers become the atomic types (notify.Service.closed, image.Service.term and submitCount, and several test counters), reverse index loops become slices.Backward, a Split-then-index becomes strings.Cut, counted loops become range over an int, and interface{} becomes any in the e2e suite. The example module needed no changes. The e2e module is behind a build tag, so it only matches with `go fix -tags e2e ./...`. One knock-on: prealloc can see the bound of a loop once it is written as range over an int, so the slice it feeds is now preallocated. with '#' will be ignored, and an empty message aborts the commit. # # Date: Sat Aug 22 01:32:09 2026 +0100 # # interactive rebase in progress; onto7c312da1# Last commands done (2 commands done): # reword deb6cbf1 262e6dc2 # Apply go fix under Go 1.27 # No commands remaining. # You are currently editing a commit while rebasing branch 'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed: backend/app/migrator/native.go # modified: backend/app/notify/notify.go backend/app/rest/api/rest_private_test.go # modified: backend/app/store/comment.go # modified: backend/app/store/image/image.go # modified: backend/app/store/service/service_test.go # modified: backend/app/store/service/title_test.go # modified: e2e/e2e_test.go # modified: e2e/widgets_test.go #
End-to-end tests
Drives the widget in a real browser through playwright-go against a remark42 built from this checkout.
The import path is github.com/mxschmitt/playwright-go, which is what the module declares even though its repository is playwright-community/playwright-go. Do not rewrite it to match the repository URL: the versions that carry the matching path cannot install their driver.
Prerequisites
- Docker with compose, which the suite shells out to
- A Go toolchain matching
e2e/go.mod - Network access on the first run: the Playwright driver and the browsers are downloaded into the user cache directory,
~/.cache/…on Linux and~/Library/Caches/…on macOS, and that download is the slowest part of a cold run
Running
make e2e
The suite brings the compose stack up itself when it does not find one already answering, and tears it down again afterwards. To keep the containers between runs, start them first:
make e2e-up
make e2e
make e2e-down
Run from e2e/; the compose path is relative to it. A single test:
cd e2e && go test -tags=e2e -run TestComment_ReplyNestsUnderItsParent -v ./...
make e2e-ui runs with a visible browser and leaves the stack up. The env vars behind it:
E2E_HEADLESS=falseshows the browser and slows it to 50ms a stepE2E_KEEP=1leaves the containers running afterwards, which only matters when the suite brought them up itselfE2E_DEBUG=1logs every HTTP response of status 400 or aboveE2E_BROWSERS=chromiumnarrows the engines the rendering tests use, which is the quickest way to shorten a local run
Rate-limit responses are logged whether or not E2E_DEBUG is set, because they surface otherwise as unexplained locator timeouts.
The build tag keeps these out of go test ./...; nothing runs without -tags=e2e.
When something fails
A failed test writes a Playwright trace to e2e/traces/, which CI uploads as an artifact whether or not the job went green. Open one with npx playwright show-trace e2e/traces/<name>.zip.
CI runs the suite through gotestsum and gives a failing test one rerun, so a test that fails and then passes leaves the job green. That is the case worth looking at: it is named in rerun-report.txt, uploaded beside the traces. Only attempts that failed leave a trace, and they do not overwrite each other, so a flake leaves exactly one to open. make e2e locally does not rerun anything, so a test red on a laptop and green in CI is a flake with a report to read rather than a disagreement.
Beyond that: docker compose -f compose-e2e-test.yml logs for the server side, and mailpit's web UI on http://127.0.0.1:8025 for anything email.
Before pushing, cd e2e && go vet -tags=e2e ./... and golangci-lint run --build-tags=e2e --config ../backend/.golangci.yml. CI runs both, and neither is covered by a plain go vet ./... because of the build tag.
The stack
compose-e2e-test.yml at the repository root runs three services, each bound to the loopback interface since it holds a known secret and an admin shared id:
- remark42 on
:8080, with the dev oauth2 provider on:8084, anonymous and email sign-in - remark42-shortedit on
:8081, withEDIT_TIME=15sand anonymous sign-in only, since the dev oauth2 provider's port is fixed at 8084 and cannot be published twice. It exists so the expired-edit path is observable without holding a test open for the default five minutes - mailpit on
:8025, which catches the email-auth verification message for the suite to read back
Three settings exist for the tests rather than for realism, and each is there for a reason:
REMARK_URLuses a hostname, not127.0.0.1. The dev oauth2 server binds whatever host it reads out ofREMARK_URL(localBindAddrin go-pkgz/auth), and a loopback bind inside a container cannot be published. The browser maps the names back with--host-resolver-rules.UPDATE_LIMIT=100, because the default of 0.5 updates a second rejects any test that posts twice in a row.- The suite paces its own calls to
/auth/, which is limited to two requests a second by a bare literal atbackend/app/rest/api/rest.go:242rather than by a setting. SeepauseForAuthLimit.
Isolation
Each test gets its own comment thread from a query string on the demo page, since the demo page passes window.location.href as remark_config.url and remark42 keys comments by it. A per-run id keeps threads apart from those an earlier run left behind.
Thread URLs deliberately keep the underscores a test name carries, since collapse persistence keys off the page url and a url containing an underscore is the case worth covering.
Browsers
The iframe_test.go group runs in Chromium, Firefox and WebKit. It is about rendering rather than logic: the widget holds the frame hidden until its document reports itself inited, and the opaque canvas that guards against is a WebKit behaviour, so Chromium alone would not exercise it.
Those tests address the server as 127.0.0.1 rather than by name, since --host-resolver-rules is a Chromium flag and they need no dev oauth2, which is the only reason the hostname exists.
Everything else runs in Chromium alone, for the same reason inverted: those tests sign in, sign-in needs the dev oauth2 provider, and reaching it by name from the host is Chromium-only. Running them in the other engines would mean putting the suite back inside the compose network.
Selectors
The production bundle strips data-testid, so tests use what ships: the stable class hooks the widget keeps outside CSS modules (.auth-button, .auth-submit, .comment-actions, .sort-picker, .preloader), title attributes on icon-only controls, and visible text. Three shapes are worth knowing:
.authonly exists while signed out, so waiting on it hangs after sign-in.widget()waits on the comment form, which is present either way.- Comments render through an IntersectionObserver, so one below the fold is an empty
articlewith no text in it. That makes any absence assertion written as a text filter pass whether the comment is gone or merely off screen; count articles instead, which is whatarticleCountis for. - Collapsing a thread hides the comment text, so a locator filtered by that text stops matching the element under test.
TestThread_CollapsePersistsAcrossReloadanchors on the comment's id instead.