* Make backend tests wait on conditions instead of durations
The backend workflow has a long tail of runs that fail once and pass on
a rerun. Every one of them comes down to a test assuming an operation
finishes within some duration rather than waiting for the state it
needs. Three were reproducible and each was reproduced against the old
code before being changed: TestServerAuthHooks minted a token that lived
one second and never tested expiry, so a slow runner turned the first
POST into a 401; TestServerApp_AnonMode saw "connection refused" because
waitForHTTPServerStart returned silently after three seconds and left a
later assertion to fail with something unrelated; TestFsStore_Cleanup
slept 200ms against a 300ms ttl that Cleanup widens to 400ms with its
commit grace, so roughly 100ms of stall collected an image meant to
survive.
Fixed sleeps before asserting on asynchronous work are replaced with
polls on the condition itself, using require.Eventually and
require.EventuallyWithT, and require.Never where the assertion is that
something did not happen. Polling closures assert on the CollectT they
are handed rather than on t, since testify runs them on another
goroutine, and polls that issue HTTP requests stay under the rate limit
on the routes they poll through.
Where a test needs time to have passed, the clock input is pinned
instead: staging ages are stamped with os.Chtimes on both sides of the
cleanup boundary right before each call, which also makes the 100ms
commit grace an exact case rather than something no assertion reaches,
and the RSS tests set store.Comment.Timestamp explicitly rather than
racing the wall clock into the first 100ms of a second so pubDate
matches.
chooseUnusedPort takes a port from the kernel's ephemeral range. Picking
at random out of a fixed 10000-port window let two package binaries,
which go test ./... runs concurrently, land on the same number between
the probe closing and the server binding. The start helpers fail naming
the port they waited on, and the SSL tests wait on the redirect port as
well as the TLS one.
Arbitrary budgets that nothing tests are gone: ten HTTP clients with a
one-second timeout against bolt-backed import and export, the "should
take about 100msec" assertions, and a one-second bound on noticing an
already cancelled context. Shutdown stays bounded at ten seconds so a
hang is still caught.
Two assertions get stronger. TestServerAuthHooks accepted 403 or 401
from a blocked user, an alternative that existed only because the short
token could expire mid-test; it is deterministically 403 now.
TestAdmin_BlockedList asserted two users blocked while one carried the
same 150ms ttl the next step waits to lapse, so the halves raced each
other.
goleak stops reporting the regexp2 clock goroutine, which chroma pulls
in for syntax highlighting and which lives for up to a second after the
last match with a timeout; it ends on its own but a binary finishing
inside that window was reported as leaking, and this suite now finishes
sooner. The ignore for net/http.(*Server).Shutdown goes the other way:
it no longer matches anything, with both packages run fifteen times each
under CPU oversubscription to confirm.
Two gaps the change would otherwise have opened are covered directly
rather than left to the side effects that used to cover them. The
one-second token was the only thing exercising the authenticator's
ClaimsUpd hook on refresh, so TestServerApp_ClaimsUpd now calls the hook
itself and checks admin, blocked, email and restricted-name
impersonation, including the two pass-through cases. Lifting the
open-route limit removed the last incidental exercise of the rate
limiter, so TestRateLimiter drives a burst past the allowance and checks
the refusals and that the limit is per client. Both run without a wall
clock, and both were confirmed to fail when the behaviour they cover is
removed.
Production code is untouched. The two sleeps outside test code, the 429
backoff in cmd/cleanup.go and the submit poll in store/image/image.go,
are left alone: no CI failure implicates them.
Test sleeps drop from 67 to 21, all of them either inside a
testing/synctest bubble or a poll interval. The suite runs in about 22
seconds instead of 46, mostly because
TestPublic_FindCommentsCtrl_ConsistentCount no longer paces a hundred
subtests with an 80ms sleep each to stay under the open route limit. The
300s per-package budget now matches across both workflows, the race_test
target and the documented command, and CLAUDE.md records the convention.
with '#' will be ignored, and an empty message aborts the commit. # #
Date: Sat Aug 22 01:12:31 2026 +0100 # # interactive rebase in progress;
onto 7c312da1 # Last command done (1 command done): # reword deb6cbf1 #
Make backend tests wait on conditions instead of durations # Next
command to do (1 remaining command): # reword 262e6dc2 # Apply go fix
under Go 1.27 # You are currently editing a commit while rebasing branch
'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed:
.github/workflows/release.yml # modified: CLAUDE.md # modified: Makefile
modified: backend/_example/memory_store/server/rpc_test.go # modified:
backend/app/cmd/import_test.go # modified:
backend/app/cmd/server_test.go # modified: backend/app/main_test.go #
modified: backend/app/rest/api/admin_test.go # modified:
backend/app/rest/api/middleware_test.go # modified:
backend/app/rest/api/migrator_test.go # modified:
backend/app/rest/api/rest_private_test.go # modified:
backend/app/rest/api/rest_public_test.go # modified:
backend/app/rest/api/rest_test.go # modified:
backend/app/rest/api/rss_test.go # modified:
backend/app/rest/proxy/image_test.go # modified:
backend/app/store/image/fs_store_test.go # modified:
backend/app/store/service/service_test.go # modified:
docs/backlog/api-tests-deadlock-on-macos.md #
* Apply go fix under Go 1.27
Go 1.27 extends go fix with the modernizers, so `go fix ./...` now
rewrites patterns the language has since replaced. Running it across all
three modules produces this: legacy sync/atomic calls on plain integers
become the atomic types (notify.Service.closed, image.Service.term and
submitCount, and several test counters), reverse index loops become
slices.Backward, a Split-then-index becomes strings.Cut, counted loops
become range over an int, and interface{} becomes any in the e2e suite.
The example module needed no changes. The e2e module is behind a build
tag, so it only matches with `go fix -tags e2e ./...`.
One knock-on: prealloc can see the bound of a loop once it is written as
range over an int, so the slice it feeds is now preallocated.
with '#' will be ignored, and an empty message aborts the commit. # #
Date: Sat Aug 22 01:32:09 2026 +0100 # # interactive rebase in progress;
onto 7c312da1 # Last commands done (2 commands done): # reword deb6cbf1
262e6dc2 # Apply go fix under Go 1.27 # No commands remaining. # You are
currently editing a commit while rebasing branch
'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed:
backend/app/migrator/native.go # modified: backend/app/notify/notify.go
backend/app/rest/api/rest_private_test.go # modified:
backend/app/store/comment.go # modified:
backend/app/store/image/image.go # modified:
backend/app/store/service/service_test.go # modified:
backend/app/store/service/title_test.go # modified: e2e/e2e_test.go #
modified: e2e/widgets_test.go #
* feat: add configurable SMTP HELO hostname
Allow the SMTP HELO/EHLO hostname to be configured separately from
the SMTP server hostname.
This is useful when the SMTP server requires clients to identify
themselves with a fully qualified hostname different from the server
address.
* chore: remove vendored dependency changes
* Bump go-pkgz/notify to v1.4.0 and document SMTP_HELO_HOST
The HELOHost field lands in go-pkgz/notify v1.4.0, so the branch needs the
bump to compile; v1.3.0 in master has no such field. The example module is
tidied alongside, as any change to backend/go.mod requires.
Documents the parameter in the parameters table and, separately, in the email
setup page: what it does, that leaving it unset keeps the previous `localhost`
greeting, and the case it exists for, a relay refusing the greeting under
Postfix `reject_non_fqdn_helo_hostname`.
Also records the current limit: verification emails for email authentication
go through go-pkgz/auth's own sender, which has no equivalent setting, so the
greeting there is unchanged.
* Bump go-pkgz/auth to v2.2.0 and apply SMTP_HELO_HOST to verification email
The verification email sender had no way to set the greeting, so a relay that
refuses the HELO would accept notifications and still reject sign-in emails.
EmailParams gains HELOHost in go-pkgz/auth v2.2.0, so the same SMTP_HELO_HOST
now drives both paths.
The example module is tidied alongside, as any change to backend/go.mod
requires.
---------
Co-authored-by: oli <someone@somewhere.tld>
Co-authored-by: Dmitry Verkhoturov <paskal.07@gmail.com>
Follow-up to the review notes on #2116:
- warn at startup when --trusted-proxy contains a catch-all (0.0.0.0/0 or ::/0),
which trusts every peer and re-opens the bypass - mirrors the unset-case warning
- realIPMiddleware tests: cover the unparseable-peer and trusted-peer-without-header
branches, and make the observed values per-call so subtests don't share closure locals
- trim the flag description and shorten the startup warning to the terse [WARN] style
Rate limiting and (with --votes-ip) vote de-duplication key on the client IP,
recovered from forwarding headers (X-Real-IP / X-Forwarded-For / CF-Connecting-IP)
when behind a reverse proxy. Those headers were accepted from any client, so a
caller could set them to change its apparent IP.
Add --trusted-proxy / TRUSTED_PROXY (comma-separated CIDR/IP): forwarding headers
are honored only when the direct peer is a trusted proxy; other peers keep their
real socket address. Unset preserves the previous trust-all behavior (with a
startup warning) so existing deployments keep working on upgrade.
Docs: a 'Trusted proxies and client IP' section with per-topology guidance, plus a
note in the nginx manual.
The cleanup command test builds a self-contained mock HTTP server with
only static routes (and {id} patterns read via r.URL.Path, not URLParam),
so chi.NewRouter is unnecessary — http.NewServeMux (Go 1.22 routing) covers
it. Independent of the main router; drops the chi import from app/cmd.
go test -race and golangci-lint clean.
* fix: address parameter docs and --help text inconsistencies
Audit findings from comparing site/src/docs/configuration/parameters/
against the backend flag tags.
Docs (parameters/index.md):
- image.bolt.file default was `/var/pictures.db` (absolute, looks like
a system path); actual default is `./var/pictures.db` (relative,
under the working dir).
- notify.webhook.template default was shown as
`{"text": {{.Text | escapeJSONString}}}` — both the function name
doesn't exist and the unescaped pipe inside the table cell broke the
Description column count for that row. Real default is the literal
`{"text": "{{.Text}}"}`.
- "Custom OAuth2 integration currently supports only one custom
provider at a time" was a free-standing paragraph wedged between two
table rows. kramdown terminated the table on that paragraph and
restarted a new headerless table for the rest of the rows. Moved it
to its own subsection after the table so the table stays contiguous.
Backend --help text (server.go):
- allowed-hosts description ended with a stray double apostrophe in
`CSP 'frame-ancestors''` (typo).
- Deprecated auth.email.{port,passwd,user,tls} flag descriptions were
shuffled — port said "SMTP password", passwd said "SMTP port", user
said "enable TLS", tls said "SMTP TCP connection timeout". Fixed
each to match the flag it's actually describing. Docs already had
the correct descriptions for these deprecated flags.
* fix: webhook template flag default override masking safe fallback
Copilot flagged the audit's "real default" claim and was right. server.go:286
had default:"{\"text\": \"{{.Text}}\"}" — the literal, JSON-unsafe template
that produces invalid JSON if a comment contains a quote or newline. The
notify package (webhook.go:50) has a safer fallback:
if params.Template == "" {
params.Template = webhookDefaultTemplate
}
where webhookDefaultTemplate is {"text": {{.Text | escapeJSONString}}}. But
go-flags applies its default tag at parse time, so the field is never empty
when the user omits --notify.webhook.template, and the safer fallback never
runs.
Drop the unsafe default tag so the webhook package's escapeJSONString-based
default takes effect. Also:
- fix the --help description (was "webhook authentication template", but
it's a payload template, not an auth one; same for headers).
- update parameters/index.md to document the actual safe default
({{.Text | escapeJSONString}}); escape the cell's | as \| so kramdown
doesn't treat it as a column separator.
- typo: "bellow" -> "below" in the headers env-delim comment.
* fix(auth): close OAuth open-redirect by wiring AllowedRedirectHosts
Bump go-pkgz/auth/v2 to master (v2.1.2-0.20260421203319-686683f19cf7)
which carries the `from` redirect validator from go-pkgz/auth#275.
The library default with a nil AllowedRedirectHosts is permissive
(preserves legacy behavior for existing consumers on a dep bump), so
just bumping the dep leaves remark42 vulnerable — a crafted
/auth/<provider>/login?from=https://evil.example.com/... still issues
the 307 to the attacker host after the user completes legitimate
OAuth. Verified end-to-end against a local dev-auth instance before
and after this commit.
Wire Opts.AllowedRedirectHosts in getAuthenticator to the operator's
existing --allowed-hosts config, stripping the CSP "self" sentinel
which is not a real hostname. RemarkURL's own host is always implicit
per the library contract, so a default single-site deployment gains
the protection with no config change. Multi-host embeds work as soon
as their embedding hosts are added to AllowedHosts (they already need
to be there for CSP frame-ancestors).
Refreshed vendor tree to match the new module version.
* chore(lint): suppress G703 false positives on image Save
CI's newer gosec flags os.MkdirAll/os.WriteFile in FileSystem.Save with
G703 because id flows in from the caller. id is validated at the HTTP
layer (safePictureSegment in rest_public.go) and dst is derived via
f.location — not a real traversal. Targeted //nolint with reason.
* fix(auth): normalise AllowedRedirectHosts entries + add unit test
Address Copilot review on PR #2049. The previous closure passed raw
s.AllowedHosts entries straight to the auth library, but --allowed-hosts
holds CSP frame-ancestors source expressions: scheme-prefixed values
(https://blog.example.com), entries with ports, and wildcards
(*.cdn.example.com) are all valid there but the auth library compares
against u.Hostname() and would silently drop them — breaking legitimate
redirects on multi-host deployments.
Extract getAllowedRedirectHosts that:
* trims whitespace, drops empty / 'self' / "self" / wildcard entries
* prepends https:// if scheme missing then url.Parse to extract Hostname
* logs a warning on parse failure rather than poisoning the allowlist
Wire the closure in getAuthenticator to call the helper.
Test_getAllowedRedirectHosts covers all the edge cases Copilot flagged
(scheme stripping, port handling, self spellings, wildcards, empty,
mixed real-world).
* fix(auth): preserve explicit port in AllowedRedirectHosts + clarify fs_store nolint
Address Copilot follow-up on PR #2049:
* getAllowedRedirectHosts stripped explicit ports via u.Hostname(), which
broadened the allowlist. The auth validator checks both Hostname() and
Host, so an entry like admin.example.com:8443 can and should be kept
host:port — allowing only that port, not any. Emit u.Host when
u.Port() != "", u.Hostname() otherwise. Updated tests.
* fs_store Save nolint rationale said "id validated at HTTP layer", but
Save is reached via image.Service.Save and SaveWithID (cache), neither
of which is HTTP validation. id is actually a server-generated hash in
both paths. Updated the comment.
Address all golangci-lint v2.10.1 (CI's version) findings:
* Add http.MaxBytesReader hard cap to ParseMultipartForm sites in
rest_private.savePictureCtrl (32MB) and api/migrator (256MB) — fixes
G120 by bounding total request body before form parsing.
* Suppress G70x in CLI subcommands cmd/{backup,cleanup,import,remap}.go:
all four issue HTTP requests against operator-supplied RemarkURL/CLI
flags, never user input. Each suppression carries a one-line reason.
* Suppress G122 in image fs_store cleanup walk: staging directory tree
is server-only, no untrusted symlinks land there.
CI's golangci-lint v2.10.1 (newer rule set than my local 2.11.4) flags
four G70x cases the previous run missed. All are false positives:
backup.go and cleanup.go drive HTTP requests against the operator's own
RemarkURL from CLI flags (not user input); migrator.go removes a temp
file whose name was returned by os.CreateTemp (server-controlled). Add
targeted //nolint:gosec comments naming the reason at each site.
The image proxy got an ssrfSafeTransport in commit aca0cff3 that resolves
DNS first, blocks any IP in private/reserved CIDRs, then dials by IP to
defeat DNS rebinding. The TitleExtractor used to construct comments'
PostTitle from Locator.URL — a user-supplied field — was missed by that
fix and kept using http.DefaultTransport. The hostname allowlist there
checks the parsed URL host but never the IP it resolves to, so a domain
suffix-matching an allowed host (or 127.0.0.1 itself when AllowedHosts
is empty) reaches the metadata service or any other internal endpoint.
The same gosec rule (G704) was excluded globally in .golangci.yml as part
of aca0cff3, so this gap was not caught by the linter either.
Extract the transport into a new safehttp package so it lives in one
place and can be reused, then pass safehttp.Transport() into the
TitleExtractor's http.Client at construction (cmd/server.go). The image
proxy switches to safehttp.Transport() too — same behaviour, no longer
duplicated.
Reproduction in title_test.go uses the production-style client to hit
an httptest.Server (always 127.0.0.1) and asserts the dialer refuses
even though "127.0.0.1" is in the allowed-domains list. A control case
shows the same setup without safehttp.Transport returns the page —
making the original vulnerability explicit.
matchSiteID guarded most authenticated and admin routes with
`if siteID != "" && user.SiteID != siteID`. Dropping the ?site= query
parameter made the check no-op and any authenticated user passed the
middleware. Downstream handlers fell back to reading site from the JSON
body or just used the empty string, so on email/telegram subscribe
endpoints (which read site from body) a user authenticated to siteA
could perform actions targeting siteB without the cross-site guard
ever firing.
Require ?site= to be present and to match user.SiteID. Body-only site
flows are still supported provided the URL also carries the matching
?site= — both must agree, which removes the bypass and keeps the
declared site visible to the middleware.
Reproduction TestRest_matchSiteID enumerates four cases (matching,
mismatched, missing, empty). Existing test calls that relied on the
implicit pass had to add ?site=remark42 to the URL: the addComment
helper now derives the param from c.Locator.SiteID, picture upload
URL gets the param explicitly, and the email/telegram subscribe table
adds it to every endpoint. The negative cases that previously asserted
StatusBadRequest from the handler now correctly assert StatusForbidden
from the middleware.
Apply go fix ./... analysers (Go 1.26) across backend and examples:
- interface{} → any (type alias, no behaviour change)
- for i := 0; i < N; i++ → for range N / for i := range N
- slices.Contains / slices.ContainsFunc replacing manual loops
- strings.SplitSeq replacing strings.Split in range (avoids allocation)
- strings.CutPrefix replacing HasPrefix+TrimPrefix
- min() replacing manual if/else
- fmt.Appendf replacing []byte(fmt.Sprintf(...))
- strings.Builder replacing string += concatenation
- wg.Go(func(){}) replacing wg.Add(1)/go/wg.Done() pattern
- removed redundant ii := i loop variable copies (unnecessary since Go 1.22)
omitempty on struct-typed JSON fields: go fix removed omitempty from
struct-typed fields (time.Time, PostInfo, UserDetailEntry) because
encoding/json's omitempty never applied to struct types — it was always
a no-op. Kept as bare tags (no omitzero replacement) to preserve the
existing serialisation behaviour.
Replace strings.Split(RemoteAddr, ":") with net.SplitHostPort for correct
IPv6 address extraction in vote deduplication and comment IP tracking.
Harden image proxy: add SSRF-safe transport blocking private/reserved IPs
at connection time with DNS rebinding protection, sanitize error messages
to prevent information leakage, add response size limit via io.LimitReader.
Fix shadowed error variables in BlockedUsers, SetTitle, and Delete methods.
Exclude gosec taint analysis false positives at linter config level.
Add AUTH_MICROSOFT_TENANT env var to allow configuring the Azure AD
tenant for single-tenant Entra ID applications, which cannot use the
default /common endpoint.
Depends on go-pkgz/auth#266
Closes#1998
With AUTH_SEND_JWT_HEADER=true, frontend now properly handles JWT authentication:
- Store JWT token in client-side cookie named 'JWT'
- Extract and store XSRF token from JWT payload
- Set Secure flag automatically when on HTTPS connection
- Update documentation to clarify this behavior
This fixes an issue where login state would be lost after page reload
when using header-based JWT authentication.
This clarifies that the parameter sets CSP 'frame-ancestors'
to limit hosts allowed to embed comments. The commit also improves
the documentation on how to use ALLOWED_HOSTS with AUTH_SAME_SITE
for different setup scenarios.
We might want to change AUTH_SAME_SITE to `strong` in v2.0 as it works
on the subdomain of the same site as well as current Lax option.
`Content-Security-Policy` now restricts resource loading and execution
to enhance security:
- `default-src 'none'`: Disallow all resource loading by default.
- `base-uri 'none'`: Prevents the use of `<base>` tag to change the
base URL for relative URLs.
- `form-action 'none'`: Disallows form submissions.
- `connect-src 'self'`: Restricts the origins that can be connected to
(via XHR, WebSockets, etc.) to the same origin.
- `frame-src 'self'`: Restricts the origins that can be embedded using
`<frame>` and `<iframe>` to the same origin (for `/web/` demo
endpoint).
- `frame-ancestors %s;`: Specifies the origins that are allowed to
embed this content in a frame. If no specific origins are allowed, it
defaults to `*` (any origin). This enhances security by controlling
which sites can embed your content.
- `img-src 'self'`: Allows images to be loaded only from the same
origin. If `imageProxyEnabled` is true, allows images from any origin
(`*`).
- `script-src 'self' 'unsafe-inline'`: Allows scripts to be loaded and
executed only from the same origin and allows inline scripts.
- `style-src 'self' 'unsafe-inline'`: Allows styles to be loaded and
applied only from the same origin and allows inline styles.
- `font-src data:`: Allows fonts to be loaded from data URIs.
- `object-src 'none'`: Disallows the use of `<object>`, `<embed>`, and
`<applet>` tags.
`Permissions-Policy` now restricts the use of certain browser features
which we don't use to enhance user privacy and security:
- `accelerometer=()`: Disables the use of the accelerometer sensor.
- `autoplay=()`: Disables automatic playback of media.
- `camera=()`: Disables the use of the camera.
- `cross-origin-isolated=()`: Disallows the page from being treated as
cross-origin isolated.
- `display-capture=()`: Disables the ability to capture the display.
- `encrypted-media=()`: Disables the use of Encrypted Media Extensions
.
- `fullscreen=()`: Disables the ability to use fullscreen mode.
- `geolocation=()`: Disables the use of geolocation.
- `gyroscope=()`: Disables the use of the gyroscope sensor.
- `keyboard-map=()`: Disables the use of the keyboard map.
- `magnetometer=()`: Disables the use of the magnetometer sensor.
- `microphone=()`: Disables the use of the microphone.
- `midi=()`: Disables the use of the MIDI API.
- `payment=()`: Disables the Payment Request API.
- `picture-in-picture=()`: Disables the use of Picture-in-Picture mode
.
- `publickey-credentials-get=()`: Disables the use of the Web
Authentication API.
- `screen-wake-lock=()`: Disables the ability to prevent the screen
from dimming.
- `sync-xhr=()`: Disables synchronous XMLHttpRequest.
- `usb=()`: Disables the use of the USB API.
- `xr-spatial-tracking=()`: Disables the use of spatial tracking in
WebXR.
- `clipboard-read=()`: Disables the ability to read from the clipboard
.
- `clipboard-write=()`: Disables the ability to write to the clipboard
.
- `gamepad=()`: Disables the use of the Gamepad API.
- `hid=()`: Disables the use of the Human Interface Device API.
- `idle-detection=()`: Disables the ability to detect idle state.
- `interest-cohort=()`: Disables the use of interest cohort tracking.
- `serial=()`: Disables the use of the Serial API.
- `unload=()`: Disables the ability to use the `beforeunload` and
`unload` events.
- `window-management=()`: Disables the ability to use window
management APIs.
Previously, the error printed was just the following:
error response "401 Unauthorized", Unauthorized"
New error:
error response "401 Unauthorized", ensure you have set ADMIN_PASSWD
and provided it to the command you're running: Unauthorized
Allowed domains consist of `REMARK_URL` second-level domain (or whole IP in case it's IP like `127.0.0.1`) and `ALLOWED_HOSTS`. That is needed to prevent Remark42 from asking arbitrary servers and storing the page title as the comment.PostTitle.
Previous behaviour allowed the caller of the API to create a comment
with an arbitrary URL and learn the title of the page, which might be
accessible to the server Remark42 is installed on but not to the user
outside that network (CWE-918).
Allowed domains consist of `REMARK_URL` second-level domain (or whole IP in case it's IP like `127.0.0.1`) and `ALLOWED_HOSTS`. That is needed to prevent Remark42 from asking arbitrary servers and storing the page title as the comment.PostTitle.
Previous behaviour allowed the caller of the API to create a comment
with an arbitrary URL and learn the title of the page, which might be
accessible to the server Remark42 is installed on but not to the user
outside that network (CWE-918).
Dmitry VerkhoturovandGitHub
AlexMa233andGitHub
Mrigank Badola
Tomy HsiehandUmputun
Vladimir DandUmputun