fix/plan-r1-oauth-scope
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Dmitry VerkhoturovandGitHub
|
2640aaee9e |
Reach what http cannot: the widget over TLS, embedded cross-origin (#2214)
Every service in the suite spoke http, and the browser gates a whole class of behaviour on the page protocol: Secure cookies, SameSite=None, Partitioned, and any code reading location.protocol. None of it was executed, which is how setAuthCookie came to decorate its cookies with __Host- on https pages and survive for years. A TLS pair joins the stack: remark42 with SSL_TYPE=static on 8443, and an nginx serving a host page on its own name on 8444, both on a self-signed certificate that e2e/tls/generate.sh makes and .gitignore keeps out. Every context accepts it, and so does the readiness client, since those are the only servers either talks to. The instance also runs with AUTH_SEND_JWT_HEADER, which is what makes the widget write cookies of its own: without it the client-side writer never runs on any https page here and every assertion about the attributes it chooses is vacuous. Three cases. Signing in across origins and then reloading, which is the one the http cross-origin case cannot make: the widget holds its token in memory for the life of a page, so signing in and posting says nothing about persistence and only the reload asks whether the cookie was delivered, stored under a name the backend reads and sent back from a third-party frame. The cookies themselves, read out of the browser store while the widget is embedded elsewhere, since a cookie the browser refused is absent from that list entirely and one it kept but will not send is worse than useless: every copy of both names has to be Secure and SameSite=None, at least one has to be partitioned, and none may carry a __Host- prefix nothing on either side reads. And the same reload under a browser that blocks third-party cookies, which the widget's own partitioned pair is the only reason to survive. That last one needs a browser playwright does not offer: its default arguments disable ThirdPartyStoragePartitioning outright, so a run configured wrongly keeps every third-party cookie and the case would pass while asserting nothing. IgnoreDefaultArgs drops that list and re-supplies it without the one feature, and a control cookie set from inside the frame has to be refused before anything else is read, so a playwright release that changes the list fails as itself instead of going quietly vacuous. All three pass against master. What TLS still cannot reach, the OAuth popup above all, is written down in the README. |
||
|
Dmitry VerkhoturovandGitHub
|
4793c1cd2c |
Fill the instance URL into the embedded frontend at serve time, and stop pinning compressor output in tests (#2198)
* Assert what the image endpoints promise rather than the compressor's output Three tests pinned the exact bytes or the exact length of an encoded image, so they fail on any toolchain whose deflate or png encoder emits something different. CI pins go 1.25 and passes; go 1.27 fails all three, while the images themselves are perfectly valid. TestRest_QR now decodes both the golden file and the response and compares the pixels, which is the same assertion about the qr code and none about the encoder. The two resize cases assert the decoded image fits the box resize was given and touches one of its sides, which is what fitting to a box means and what the function actually promises. Resolves #2200. * Fill the instance URL into the embedded frontend at serve time The widget falls back to a compiled-in URL whenever a page omits `remark_config.host`. The bundler cannot know that URL, so it emits `{% REMARK_URL %}` and each distribution substitutes it: the docker image rewrites the files under its web root at container start, and the release binary, which serves the build embedded in itself, had nothing doing it. `prepare-release-assets.sh` filled the marker with `http://127.0.0.1:8080` before the embed instead, so every copy of the binary shipped pointing at the visitor's own loopback address, and on an https site the request is blocked as mixed content besides. It has been that way since v1.11.0, the first release to embed the frontend, and the earlier binaries embedded none, so the tarball has never served a correctly addressed widget. The placeholder now survives into the embedded copy and the file server fills it with the configured `REMARK_URL` as it serves, which is what the docker image already does to its own copy. The image no longer bakes the loopback address into its embedded copy either, so the fallback it keeps for a missing web root is correct rather than misleading. Substituted in html, js and mjs, the same set `docker-init.sh` rewrites, and the served size is the substituted one so a response is neither truncated nor left hanging. Nothing exercised the marker the frontend build emits wherever the instance url belongs. Every page in the suite sets `remark_config.host` from its own origin, so the compiled-in fallback is never read, and a distribution that stopped substituting would keep the suite green. Two tests. The first reads the served bundles and pages back and asserts the marker is gone from each and that what replaced it is this instance. The second covers what the substitution is for: the widget document carries no host of its own, since `iframe.html` builds its config from a query string the parent never puts one in, so everything it requests is addressed with the compiled-in url. It asserts the widget renders and that the config request went to this instance. The demo pages cannot show the second. Their loader builds the bundle's own script url from `remark_config.host`, so a page without one never gets as far as loading the widget. Verified by disabling both substitution paths, the serve-time one and the docker image's, and rebuilding: both tests fail. Editing the files on disk is not enough, since the file server substitutes as it serves. The served body now depends on remarkURL, but cacheControl builds its etag from version and path only. An operator who notices the widget is addressed to the wrong host, corrects REMARK_URL and restarts the same binary gets 304 on revalidation, so the client keeps a bundle pointing at the old host. Cache-Control is no-cache, so it revalidates every time and never ages out of that state either. That is the exact situation this substitution exists to fix, so the validator has to carry the url. |
||
|
Dmitry VerkhoturovandGitHub
|
23be25d84a |
Fix seven widget defects, including the cookies the separate-domain setup needs (#2197)
* Drop the frontend workspace root and re-resolve the lockfile
`frontend/` carried a `package.json`, a `pnpm-workspace.yaml` and the lockfile
for a workspace of exactly one package. Two manifests meant two places to
declare a version, and the app pin was the one that did not win: `preact` and
`@babel/core` were each written twice, and a bump to the app manifest alone
would have been a silent no-op, since `pnpm.overrides` decides and it lived at
the root.
Everything pnpm reads now lives in `frontend/apps/remark42`: dependencies,
`packageManager`, `engines` and the overrides. `frontend/` keeps `.nvmrc`,
`.husky` and `CLAUDE.md`, none of which pnpm reads. The directory nesting
stays: every path in the repository points at `frontend/apps/remark42`,
including the published contributing docs, so moving the package up would have
rewritten 14 files to no benefit.
Moving the manifest kept the old resolutions verbatim, which left optional peer
subtrees the tree no longer reaches: `ts-node` under jest, `@swc/core` under
webpack, `vitest` under `@testing-library/jest-dom`, `tslib` under
`webpack-dev-server`. None is referenced by any config or source file here.
Re-resolving drops 137 packages and moves 59 to versions already permitted by
the ranges in the manifest, 1446 to 1308, with no direct dependency changing
version: the five that look changed differ only in their peer suffix. Every
file `pnpm build` produces is identical in size before and after.
The frontend-deps stage of the Dockerfile sets `CI=true` so the `prepare`
script skips husky, which has no git repository to install hooks into there.
* Stop markdown-only changes triggering heavy workflows, and check the documented versions
`ci-backend.yml`, `ci-build.yml` and `ci-frontend.yml` all end their path
filters with `!**.md`. The e2e workflow did not, so a change to any markdown
file under `frontend/` or `backend/` matched its `frontend/**` and `backend/**`
entries and started a docker build and the whole browser suite. The release
filter had the same hole and two of its own: it names `README.md` and `LICENSE`
on purpose, since `.goreleaser.yml` packages both, so it now excludes markdown
under `backend/` and `frontend/` only. `CLAUDE.md` and the installation page
were listed as well, and neither is packaged.
`ci-site.yml` goes on matching markdown, which is right, since the site is
built from it. It excludes `CLAUDE.md`, so a future `site/CLAUDE.md` cannot
start a site build, and `site/README.md`, which documents how to build the site
rather than being part of it.
The installation page tells a reader that a source build needs Go 1.25, Node
24+ and PNPM 10. Nothing kept those in step with `backend/go.mod`,
`engines.node`, `packageManager` and `.nvmrc`, and the drift is silent: a wrong
version in the docs builds and tests exactly as well as a right one. `.nvmrc`
is the pin with form here, having sat at 16 through the whole node 20 migration
because nothing red ever pointed at it. The check compares each stated version
against its source and holds `.nvmrc` to `engines.node`, and it fails when the
page states no version at all, so removing the claims cannot turn it into a
check that passes by comparing nothing.
Its own workflow rather than a step in an existing one, since the inputs span
the backend module, the frontend manifest and the site.
* Fix the cookie fallback page, asset path, message senders, auth teardown and cookies
Two defects with the same origin:
|
||
|
Dmitry VerkhoturovandGitHub
|
4d5dae20e2 |
Broaden the e2e suite from 21 cases to 63, and harden its harness (#2196)
* Pin the published /web surface in the e2e suite
#2178 renamed the widget bundles from .js to .mjs and the URLs earlier
releases served under those names stopped resolving. Three were noticed
from the demo site; the rest, including every locale chunk, were found
only by requesting the whole surface of both images over HTTP. #2192
restored them with a server-side alias, and nothing in the suite would
have caught the break or would notice it returning.
Two cases with deliberately different criteria. The documented names are
written out, because the documentation decides that list and not the
build: an operator pastes privacy.html into an OAuth application, the
nginx manual proxies index.html by name, and the integration guides start
from the embed script. Everything else is taken from the build itself, so
whatever the bundler emitted has to serve identical bytes under its
legacy .js name and parse as a classic script, which is the premise
serving one under the other rests on. A third case requests a name that
does not exist, without which a fallback serving one page for everything
would keep the whole table green.
All of them check the content type as well as the bytes: nosniff is set
on every response, so a bundle served as text/plain is as broken as one
that 404s while comparing equal.
On
|
||
 UmputunandGitHub
|
a82dc8d3f1 |
Restore the legacy /web/*.js URLs and fix iframe reuse (#2192)
* Serve the legacy /web/*.js names from their .mjs siblings The build emitted <name>.js alongside <name>.mjs until the two compilations were collapsed into one. Dropping the second compilation was right, but it removed URLs the project itself had published: the v1.16.4 SPA documentation named /web/embed.js directly and its loader snippet requested .js. Pages that hard-coded those names now 404 with no deprecation. webFiles.Open retries a missing .js against the .mjs sibling. The bundles contain no import or export, so the same bytes serve both names. The retry runs only once both sources report the name missing, so a real .js still wins, and an unreadable sibling reports its own error rather than being flattened into the requested file's 404. Related to #2178 * Reuse only the comments iframe embed created createInstance took root.firstElementChild as its iframe, so anything a page left inside #remark42 was adopted instead. A <noscript> fallback became the "iframe", createIframe never ran, and the height messages went to an element that cannot show comments. That also defeats the placeholder support, which promises content in the root is cleared once the iframe reports inited: a text placeholder works, but any element placeholder is mistaken for the iframe, so inited never arrives and the cleanup never runs. The iframe now carries data-remark42-iframe and the lookup is scoped to a direct child, so a second createInstance still reuses it while nothing else in the root can be adopted. Related to #1990 * Assert the backup contents rather than the compressed size TestBackup_MakeBackup and TestBackup_Do pinned the gzip output at 52 bytes, which ties them to the exact output of compress/flate. The same input encodes to 57 bytes on go 1.27, so both fail for anyone building on a toolchain newer than the one CI pins. They now read the backup back and compare it against what the exporter wrote, which is what the tests were reaching for and does not move with the compressor. The payload is a shared constant so the two cannot drift. |
||
|
Dmitry VerkhoturovandGitHub
|
e3d1d0e23e |
Create the e2e trace directory before writing a trace (#2194)
`newPageOn` writes a trace into `traces/` when a test fails, and never created that directory. It is gitignored, so a fresh checkout does not have it. Traces were not in fact being dropped: the driver creates the parent of the trace path itself, checked against the version this module pins rather than assumed. The directory is created here anyway because nothing in the suite states or tests that dependency, and the missing directory has been raised in review on #2180 and again on #2193, each time needing the driver checked before it could be answered. One visible difference on a fresh checkout: the directory now arrives at 0750 rather than the 0755 the driver's own mkdir leaves, both measured. It runs only on a test that has already failed, and logs its error rather than swallowing it, matching the Stop call below it. |
||
|
Dmitry VerkhoturovandGitHub
|
49bf83b09c |
Address the review follow-ups from #2188, #2189 and #2190 (#2193)
* Read the collapsed-threads key through getJsonItem `getFromLocalStorage` parsed the stored string directly, so anything malformed under `__remarkCollapsed` threw out of `restoreCollapsedThreads`. That call sits in `remark.tsx` ahead of the `render`, so the throw took the whole widget with it: the reader was left on the preloader, over a view preference. `getJsonItem` in `common/local-storage.ts` already wraps a parse of a localStorage key and returns null on failure, and null is a shape the check below already reads as empty. The rest of that function is total against whatever the browser holds, and the bare parse was the one way in. * Stop retrying a failed e2e test in CI The suite went in with one gotestsum rerun. It has no failures on record to justify that: 31 CI runs since it landed, all green, and no rerun report has ever been produced. A retry is what turns an intermittent regression into a green build, and while the suite is this young its own failures are the evidence worth keeping. `E2E_RUN_ID` stays. It stamps the threads a run works on with the CI run id, so a thread url in a trace or a log names the run it came from. It carries no data across: the stack is disposable, and a local run under the same id gets those urls on an empty database. * Stop two chooseUnusedPort comments claiming collisions cannot happen All four copies listen on :0, read the assigned port, close the listener and bind later, so nothing holds the number across that gap and another binary can take it. The copies in app/cmd and app/rest/api call a collision very unlikely, which is accurate; the ones in app and the example module said binaries never land on the same number, which is not, and a comment ruling out a port collision is what would send the next person chasing one somewhere else. All four now read the same. Closing the window rather than describing it means the server binding :0 itself and reporting the address it got, which is a larger change. |
||
|
Dmitry VerkhoturovandGitHub
|
0b651dddd4 |
Make backend tests wait on conditions instead of durations (#2190)
* Make backend tests wait on conditions instead of durations The backend workflow has a long tail of runs that fail once and pass on a rerun. Every one of them comes down to a test assuming an operation finishes within some duration rather than waiting for the state it needs. Three were reproducible and each was reproduced against the old code before being changed: TestServerAuthHooks minted a token that lived one second and never tested expiry, so a slow runner turned the first POST into a 401; TestServerApp_AnonMode saw "connection refused" because waitForHTTPServerStart returned silently after three seconds and left a later assertion to fail with something unrelated; TestFsStore_Cleanup slept 200ms against a 300ms ttl that Cleanup widens to 400ms with its commit grace, so roughly 100ms of stall collected an image meant to survive. Fixed sleeps before asserting on asynchronous work are replaced with polls on the condition itself, using require.Eventually and require.EventuallyWithT, and require.Never where the assertion is that something did not happen. Polling closures assert on the CollectT they are handed rather than on t, since testify runs them on another goroutine, and polls that issue HTTP requests stay under the rate limit on the routes they poll through. Where a test needs time to have passed, the clock input is pinned instead: staging ages are stamped with os.Chtimes on both sides of the cleanup boundary right before each call, which also makes the 100ms commit grace an exact case rather than something no assertion reaches, and the RSS tests set store.Comment.Timestamp explicitly rather than racing the wall clock into the first 100ms of a second so pubDate matches. chooseUnusedPort takes a port from the kernel's ephemeral range. Picking at random out of a fixed 10000-port window let two package binaries, which go test ./... runs concurrently, land on the same number between the probe closing and the server binding. The start helpers fail naming the port they waited on, and the SSL tests wait on the redirect port as well as the TLS one. Arbitrary budgets that nothing tests are gone: ten HTTP clients with a one-second timeout against bolt-backed import and export, the "should take about 100msec" assertions, and a one-second bound on noticing an already cancelled context. Shutdown stays bounded at ten seconds so a hang is still caught. Two assertions get stronger. TestServerAuthHooks accepted 403 or 401 from a blocked user, an alternative that existed only because the short token could expire mid-test; it is deterministically 403 now. TestAdmin_BlockedList asserted two users blocked while one carried the same 150ms ttl the next step waits to lapse, so the halves raced each other. goleak stops reporting the regexp2 clock goroutine, which chroma pulls in for syntax highlighting and which lives for up to a second after the last match with a timeout; it ends on its own but a binary finishing inside that window was reported as leaking, and this suite now finishes sooner. The ignore for net/http.(*Server).Shutdown goes the other way: it no longer matches anything, with both packages run fifteen times each under CPU oversubscription to confirm. Two gaps the change would otherwise have opened are covered directly rather than left to the side effects that used to cover them. The one-second token was the only thing exercising the authenticator's ClaimsUpd hook on refresh, so TestServerApp_ClaimsUpd now calls the hook itself and checks admin, blocked, email and restricted-name impersonation, including the two pass-through cases. Lifting the open-route limit removed the last incidental exercise of the rate limiter, so TestRateLimiter drives a burst past the allowance and checks the refusals and that the limit is per client. Both run without a wall clock, and both were confirmed to fail when the behaviour they cover is removed. Production code is untouched. The two sleeps outside test code, the 429 backoff in cmd/cleanup.go and the submit poll in store/image/image.go, are left alone: no CI failure implicates them. Test sleeps drop from 67 to 21, all of them either inside a testing/synctest bubble or a poll interval. The suite runs in about 22 seconds instead of 46, mostly because TestPublic_FindCommentsCtrl_ConsistentCount no longer paces a hundred subtests with an 80ms sleep each to stay under the open route limit. The 300s per-package budget now matches across both workflows, the race_test target and the documented command, and CLAUDE.md records the convention. with '#' will be ignored, and an empty message aborts the commit. # # Date: Sat Aug 22 01:12:31 2026 +0100 # # interactive rebase in progress; onto |
||
|
Dmitry VerkhoturovandGitHub
|
b6975af63c |
Fix collapsed threads not restoring, and the clock skew correction (#2188)
* Fix collapsed threads not restoring, and the clock skew correction
Collapse state was kept as a flat list of `siteID_url_commentID` strings
and read back by splitting on `_`. Any underscore in the url, the site id
or the comment id made the pieces impossible to tell apart, so a page
whose url contains one lost its collapsed threads on every reload, and one
page's entries could be read or deleted as another's: `/post` matched
everything stored for `/post_2`, and a site id of `blog` matched `blog_ru`.
No separator fixes that, since every candidate can occur inside the values,
so the ids are now nested under the site and the url instead. Anything
stored in the old shape reads as empty: collapsed threads are a view
preference, and re-expanding them once is not worth a migration.
The e2e suite had been stripping underscores out of its own thread urls to
work around this, which left its collapse test unable to fail on the bug it
covers. That workaround is gone, and the test now fails without this fix.
`serverClientTimeDiff` was written in seconds and added to an epoch in
milliseconds, so the correction it exists to apply was a thousandth of the
real skew. It is now milliseconds, and named for the unit.
A response with no usable `date` used to fall back to a zero timestamp,
which already made the "skew" about twenty days and would have made it
fifty-five years once the units were right. Nothing is stored now unless
the reading is plausible, since `Date.parse` is lenient enough to turn junk
into a date and let an absurd value through the branch that parses.
The score tooltip reports controversy again when there is any. It has been
dead since the vote component was rewritten in
|
||
|
Dmitry VerkhoturovandGitHub
|
a0879b2336 |
Measure the iframe reveal budgets from inside the page (#2189)
The three reveal tests timed their budgets from before `page.Goto`, so a slow navigation was spent against a window that belongs to the iframe. In `TestIframe_StaysHiddenUntilTheDocumentReportsInited` that made the test vacuous rather than flaky: on a navigation between 2.5 and 5 seconds the loop bounding the visibility assertion had no budget left, ran zero times, and the test passed having asserted nothing. Reproduced by delaying the demo document by three seconds, where the assertion ran 0 times before and runs 23 after. The timeout test had the mirror of it, with navigation counting toward the lower bound that exists to catch a shortened fallback. An init script now records, in the page, when the widget's iframe element enters the document and when its visibility first flips. `create-iframe.ts` arms its fallback a moment earlier, on the detached element, so these read a shade short and every bound is conservative in the same direction. Both bounds were also wider than the thing they guard. The hidden window now runs almost to the fallback rather than half of it, and the lower bound sits just under it rather than at three quarters, which a fallback shortened to four seconds used to clear. CI reruns a failing test once rather than failing the build on the first flake. A browser suite has a floor no amount of care removes, and one flake failing the build is what stops people trusting the suite. Once rather than twice, because a rerun stops at the first pass and each further attempt only widens the window where a real intermittent regression is absorbed. What needed a rerun is written to a report and uploaded with the traces, which are kept whether or not the job went green: a run that recovered on the rerun is exactly the one whose evidence used to be discarded. |
||
 dependabot[bot]andUmputun
|
2bfad021e3 |
chore(deps): bump github.com/mxschmitt/playwright-go
Bumps the go-modules-updates group in /e2e with 1 update: [github.com/mxschmitt/playwright-go](https://github.com/mxschmitt/playwright-go). Updates `github.com/mxschmitt/playwright-go` from 0.6201.0 to 0.6201.1 - [Release notes](https://github.com/mxschmitt/playwright-go/releases) - [Commits](https://github.com/mxschmitt/playwright-go/compare/v0.6201.0...v0.6201.1) --- updated-dependencies: - dependency-name: github.com/mxschmitt/playwright-go dependency-version: 0.6201.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: go-modules-updates ... Signed-off-by: dependabot[bot] <support@github.com> |
||
|
Dmitry VerkhoturovandGitHub
|
ff77f41a3a |
Move the e2e suite to Go and playwright-go (#2180)
* Move the e2e suite to Go and playwright-go The seven playwright tests in `frontend/e2e` become twenty in `e2e/`, a separate Go module driving the same browsers through playwright-go. The npm project, its lockfile entries, its prettier config and `Dockerfile.e2e` go with it, leaving `frontend/` a single-member workspace. The suite covers posting with markdown, replying and the nesting that implies, editing inside the deadline and the backend refusing one outside it, deleting, voting with the optimistic score observed mid-flight and rolled back on failure, changing the sort, collapse persistence across a reload, dev, anonymous and email sign-in end to end, the profile iframe, and the two scripts that render into the host page rather than the widget's own frame. The rendering tests run in chromium, firefox and webkit. The rest sign in, sign-in needs the dev oauth2 provider, and reaching that by name from the host is chromium-only, so they run there alone. `compose-e2e-test.yml` runs remark42, a second instance with a short edit window so that path does not need a five-minute test, and mailpit, which catches the email verification message the suite reads back. Everything binds to the loopback interface: the stack holds a known secret and an admin shared id, and `go test` can start it unattended. The tests run on the host rather than in a container. Three settings there exist for the tests rather than for realism. `REMARK_URL` uses a hostname because the dev oauth2 server binds whatever host it reads out of it, and a loopback bind inside a container cannot be published. `UPDATE_LIMIT` is raised because the default of 0.5/sec rejects any test posting twice in a row. The suite also paces its own `/auth/` calls, which are capped at 2/sec by a bare literal in `rest.go` rather than by a setting. Each test gets its own comment thread from a query string on the demo page, so nothing has to reset the database between runs. CI gains a vet and lint job for the module, since the build tag keeps it out of a plain `go test ./...`, and uploads a browser trace for any test that fails. `e2e/README.md` carries the rest: how to run it, what the stack is for, and the widget behaviour the assertions have to work around. * Update golangci-lint to 2.13.1 in the backend workflow The pin sat three minors behind what the linter installs locally, so CI checked the backend with an older set of rules than anyone running it by hand. 2.10.1 also fetches its config schema over the network on every `config verify`, which is a failure mode with no bearing on the code. Both targets are clean on 2.13.1, `backend/app` and the memory_store example. |