* Assert what the image endpoints promise rather than the compressor's output
Three tests pinned the exact bytes or the exact length of an encoded
image, so they fail on any toolchain whose deflate or png encoder emits
something different. CI pins go 1.25 and passes; go 1.27 fails all three,
while the images themselves are perfectly valid.
TestRest_QR now decodes both the golden file and the response and
compares the pixels, which is the same assertion about the qr code and
none about the encoder. The two resize cases assert the decoded image
fits the box resize was given and touches one of its sides, which is what
fitting to a box means and what the function actually promises.
Resolves#2200.
* Fill the instance URL into the embedded frontend at serve time
The widget falls back to a compiled-in URL whenever a page omits
`remark_config.host`. The bundler cannot know that URL, so it emits
`{% REMARK_URL %}` and each distribution substitutes it: the docker image
rewrites the files under its web root at container start, and the release
binary, which serves the build embedded in itself, had nothing doing it.
`prepare-release-assets.sh` filled the marker with `http://127.0.0.1:8080`
before the embed instead, so every copy of the binary shipped pointing at
the visitor's own loopback address, and on an https site the request is
blocked as mixed content besides.
It has been that way since v1.11.0, the first release to embed the
frontend, and the earlier binaries embedded none, so the tarball has never
served a correctly addressed widget.
The placeholder now survives into the embedded copy and the file server
fills it with the configured `REMARK_URL` as it serves, which is what the
docker image already does to its own copy. The image no longer bakes the
loopback address into its embedded copy either, so the fallback it keeps
for a missing web root is correct rather than misleading.
Substituted in html, js and mjs, the same set `docker-init.sh` rewrites,
and the served size is the substituted one so a response is neither
truncated nor left hanging.
Nothing exercised the marker the frontend build emits wherever the instance
url belongs. Every page in the suite sets `remark_config.host` from its own
origin, so the compiled-in fallback is never read, and a distribution that
stopped substituting would keep the suite green.
Two tests. The first reads the served bundles and pages back and asserts the
marker is gone from each and that what replaced it is this instance. The
second covers what the substitution is for: the widget document carries no
host of its own, since `iframe.html` builds its config from a query string the
parent never puts one in, so everything it requests is addressed with the
compiled-in url. It asserts the widget renders and that the config request
went to this instance.
The demo pages cannot show the second. Their loader builds the bundle's own
script url from `remark_config.host`, so a page without one never gets as far
as loading the widget.
Verified by disabling both substitution paths, the serve-time one and the
docker image's, and rebuilding: both tests fail. Editing the files on disk is
not enough, since the file server substitutes as it serves.
The served body now depends on remarkURL, but cacheControl builds its
etag from version and path only. An operator who notices the widget is
addressed to the wrong host, corrects REMARK_URL and restarts the same
binary gets 304 on revalidation, so the client keeps a bundle pointing
at the old host. Cache-Control is no-cache, so it revalidates every time
and never ages out of that state either.
That is the exact situation this substitution exists to fix, so the
validator has to carry the url.
* Make backend tests wait on conditions instead of durations
The backend workflow has a long tail of runs that fail once and pass on
a rerun. Every one of them comes down to a test assuming an operation
finishes within some duration rather than waiting for the state it
needs. Three were reproducible and each was reproduced against the old
code before being changed: TestServerAuthHooks minted a token that lived
one second and never tested expiry, so a slow runner turned the first
POST into a 401; TestServerApp_AnonMode saw "connection refused" because
waitForHTTPServerStart returned silently after three seconds and left a
later assertion to fail with something unrelated; TestFsStore_Cleanup
slept 200ms against a 300ms ttl that Cleanup widens to 400ms with its
commit grace, so roughly 100ms of stall collected an image meant to
survive.
Fixed sleeps before asserting on asynchronous work are replaced with
polls on the condition itself, using require.Eventually and
require.EventuallyWithT, and require.Never where the assertion is that
something did not happen. Polling closures assert on the CollectT they
are handed rather than on t, since testify runs them on another
goroutine, and polls that issue HTTP requests stay under the rate limit
on the routes they poll through.
Where a test needs time to have passed, the clock input is pinned
instead: staging ages are stamped with os.Chtimes on both sides of the
cleanup boundary right before each call, which also makes the 100ms
commit grace an exact case rather than something no assertion reaches,
and the RSS tests set store.Comment.Timestamp explicitly rather than
racing the wall clock into the first 100ms of a second so pubDate
matches.
chooseUnusedPort takes a port from the kernel's ephemeral range. Picking
at random out of a fixed 10000-port window let two package binaries,
which go test ./... runs concurrently, land on the same number between
the probe closing and the server binding. The start helpers fail naming
the port they waited on, and the SSL tests wait on the redirect port as
well as the TLS one.
Arbitrary budgets that nothing tests are gone: ten HTTP clients with a
one-second timeout against bolt-backed import and export, the "should
take about 100msec" assertions, and a one-second bound on noticing an
already cancelled context. Shutdown stays bounded at ten seconds so a
hang is still caught.
Two assertions get stronger. TestServerAuthHooks accepted 403 or 401
from a blocked user, an alternative that existed only because the short
token could expire mid-test; it is deterministically 403 now.
TestAdmin_BlockedList asserted two users blocked while one carried the
same 150ms ttl the next step waits to lapse, so the halves raced each
other.
goleak stops reporting the regexp2 clock goroutine, which chroma pulls
in for syntax highlighting and which lives for up to a second after the
last match with a timeout; it ends on its own but a binary finishing
inside that window was reported as leaking, and this suite now finishes
sooner. The ignore for net/http.(*Server).Shutdown goes the other way:
it no longer matches anything, with both packages run fifteen times each
under CPU oversubscription to confirm.
Two gaps the change would otherwise have opened are covered directly
rather than left to the side effects that used to cover them. The
one-second token was the only thing exercising the authenticator's
ClaimsUpd hook on refresh, so TestServerApp_ClaimsUpd now calls the hook
itself and checks admin, blocked, email and restricted-name
impersonation, including the two pass-through cases. Lifting the
open-route limit removed the last incidental exercise of the rate
limiter, so TestRateLimiter drives a burst past the allowance and checks
the refusals and that the limit is per client. Both run without a wall
clock, and both were confirmed to fail when the behaviour they cover is
removed.
Production code is untouched. The two sleeps outside test code, the 429
backoff in cmd/cleanup.go and the submit poll in store/image/image.go,
are left alone: no CI failure implicates them.
Test sleeps drop from 67 to 21, all of them either inside a
testing/synctest bubble or a poll interval. The suite runs in about 22
seconds instead of 46, mostly because
TestPublic_FindCommentsCtrl_ConsistentCount no longer paces a hundred
subtests with an 80ms sleep each to stay under the open route limit. The
300s per-package budget now matches across both workflows, the race_test
target and the documented command, and CLAUDE.md records the convention.
with '#' will be ignored, and an empty message aborts the commit. # #
Date: Sat Aug 22 01:12:31 2026 +0100 # # interactive rebase in progress;
onto 7c312da1 # Last command done (1 command done): # reword deb6cbf1 #
Make backend tests wait on conditions instead of durations # Next
command to do (1 remaining command): # reword 262e6dc2 # Apply go fix
under Go 1.27 # You are currently editing a commit while rebasing branch
'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed:
.github/workflows/release.yml # modified: CLAUDE.md # modified: Makefile
modified: backend/_example/memory_store/server/rpc_test.go # modified:
backend/app/cmd/import_test.go # modified:
backend/app/cmd/server_test.go # modified: backend/app/main_test.go #
modified: backend/app/rest/api/admin_test.go # modified:
backend/app/rest/api/middleware_test.go # modified:
backend/app/rest/api/migrator_test.go # modified:
backend/app/rest/api/rest_private_test.go # modified:
backend/app/rest/api/rest_public_test.go # modified:
backend/app/rest/api/rest_test.go # modified:
backend/app/rest/api/rss_test.go # modified:
backend/app/rest/proxy/image_test.go # modified:
backend/app/store/image/fs_store_test.go # modified:
backend/app/store/service/service_test.go # modified:
docs/backlog/api-tests-deadlock-on-macos.md #
* Apply go fix under Go 1.27
Go 1.27 extends go fix with the modernizers, so `go fix ./...` now
rewrites patterns the language has since replaced. Running it across all
three modules produces this: legacy sync/atomic calls on plain integers
become the atomic types (notify.Service.closed, image.Service.term and
submitCount, and several test counters), reverse index loops become
slices.Backward, a Split-then-index becomes strings.Cut, counted loops
become range over an int, and interface{} becomes any in the e2e suite.
The example module needed no changes. The e2e module is behind a build
tag, so it only matches with `go fix -tags e2e ./...`.
One knock-on: prealloc can see the bound of a loop once it is written as
range over an int, so the slice it feeds is now preallocated.
with '#' will be ignored, and an empty message aborts the commit. # #
Date: Sat Aug 22 01:32:09 2026 +0100 # # interactive rebase in progress;
onto 7c312da1 # Last commands done (2 commands done): # reword deb6cbf1
262e6dc2 # Apply go fix under Go 1.27 # No commands remaining. # You are
currently editing a commit while rebasing branch
'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed:
backend/app/migrator/native.go # modified: backend/app/notify/notify.go
backend/app/rest/api/rest_private_test.go # modified:
backend/app/store/comment.go # modified:
backend/app/store/image/image.go # modified:
backend/app/store/service/service_test.go # modified:
backend/app/store/service/title_test.go # modified: e2e/e2e_test.go #
modified: e2e/widgets_test.go #
The test slept 200ms, ran Cleanup with a 300ms TTL and then asserted the
second and third staged images survived. Cleanup collects anything older than
the TTL plus a 100ms commit grace, and the second image was already 300ms old
by then, so a runner that stalled ~100ms anywhere in the setup aged it past
the line and the assertion failed with "file on staging".
Age comes from the file's modification time, so the test now sets it with
os.Chtimes on both sides of the boundary immediately before each Cleanup call:
the image meant to be collected is backdated an hour, the ones meant to
survive are stamped at now. That leaves no window for a stall to age a file
into the wrong bucket, and drops 600ms of sleeping.
Verified by injecting a stall into the setup: 250ms reproduces the failure on
the current code, while the version here survives 2s.
alterComment issued two engine.Flag calls (Blocked, then Verified) for
every comment, so a listing of N comments triggered up to 2N BoltDB read
transactions even when many comments shared the same author. Find,
FindSince, User and Last all funnel through it.
Add a userFlagCache that memoises blocked/verified results by site and
user for the duration of a single listing, so repeated authors are
looked up once. alterComment keeps its signature for single-comment
callers (Get) by using a fresh cache; the batch paths share one.
exportCtrl mapped every export failure to 500 Internal Server Error, so
requesting a backup for a non-existent site (e.g. wrong -s/--site) came
back as a misleading 500 instead of a client error — inconsistent with
the rest of the admin/public API, which returns 400 + ErrSiteNotFound
for site-lookup failures.
Add an engine.ErrSiteNotFound sentinel (wrapped at the bolt db-lookup so
the existing "site %q not found" message is unchanged) and map it to 400
+ rest.ErrSiteNotFound in exportCtrl; genuine internal failures (gzip
close/write) still return 500.
The limit() boundary used >=, so a subtree that fit the page exactly was
treated as overflow and dropped — under-filling the final page (e.g.
limit=5 over subtrees 3,2 returned only the first subtree, 3 comments,
instead of both). Change it to > so an exact-fit subtree is included;
the first node is still always returned in full and larger subtrees
still overflow to the next page.
Adds table tests for MakeTree's limit/offset pagination and countReplies,
and updates the /find consistent-count expectations for the corrected
boundary.
notify/email.go accumulated multi-recipient errors with
multierror.Append(fmt.Errorf(...)) instead of
multierror.Append(result, ...), so the accumulator was overwritten each
iteration and only the last failing recipient's error survived; earlier
failures were silently dropped. The telegram notifier did it correctly.
Replace hashicorp/go-multierror with the stdlib errors.Join everywhere
it was used (notify/email.go, notify/telegram.go, rest/api/rest_private.go,
store/service/service.go, store/image/image.go and store/engine/bolt.go),
which fixes the bug and drops the direct dependency. It stays indirect
because go-pkgz/lcw/v2 still imports it. A regression test in
email_test.go now sends two failing recipients and asserts both errors
are reported.
deleteUser now succeeds for a user who has no comments (e.g. one who only logged
in) instead of failing on the missing per-user bucket. In hard mode the per-user
bucket is deleted, tolerating bbolt's ErrBucketNotFound so a bucket left behind by
an earlier partial removal is still removed; the comment-deletion failure path now
wraps the actual error.
Because the engine cannot distinguish a valid login-only user from a never-existed
one, deletion is idempotent: /admin/deleteme returns 200 for an unknown (but validly
signed) token rather than 400. The deleteme test is updated to this contract, engine
tests cover hard and soft deletion of login-only and unknown users, and the API docs
note the idempotent behaviour.
* fix(security): reject non-image content-types in image proxy and /picture/ to prevent stored XSS
The /api/v1/img proxy and /api/v1/picture/{user}/{id} endpoints emitted
http.DetectContentType on the served bytes as the response Content-Type. A
controlled upstream serving Content-Type: image/png with an HTML body passed
the upstream check (only the response header was inspected, not the body),
and the body bytes then sniffed back to text/html — so the proxy served the
attacker's HTML from the remark42 origin. Browsers honoured the declared
text/html and executed the response as a document with access to cookies and
CSRF tokens. Affected from v1.6.0 (April 2020) through v1.15.0; verified live
via published docker images.
Layered defense applied to both handlers:
- rest.SafeImgContentType (in backend/app/rest/) validates sniffed content
against a strict allowlist: image/png, image/jpeg, image/gif, image/webp,
image/bmp, image/x-icon. Anything else (HTML, XML, SVG, plain text,
octet-stream, or any future image type the stdlib sniffer may learn) is
rejected with no body echo. SVG is implicitly excluded — it sniffs as
text/xml or text/plain, never image/svg+xml, and SVG can execute scripts
when navigated to top-level. The previous octet-stream → image/* fallback
is gone.
- Per-endpoint Content-Security-Policy override sets
"default-src 'none'; sandbox; frame-ancestors 'none'" on every response
(success, 304, or error). Sandbox neuters scripts even if Content-Type
ever regresses. The same policy is also applied to all /api/v1/* via
apiCSPMiddleware as defense-in-depth.
- Content-Disposition: inline; filename="image" frames the response as a
file rather than a renderable document.
- /picture/ rejection paths set Cache-Control: no-store so 4xx responses
are never cached.
The defense headers and the strict ETag matcher are extracted as
rest.SetImageDefenseHeaders and rest.EtagMatches in the shared rest package
(consumed by both proxy/image and api/rest_public — no package cycle).
The /api/v1/img path additionally bumps the ETag to a versioned `"v2:..."`
so revalidating clients (top-level navigation, Ctrl+R, intermediaries) get
a fresh 200 instead of a 304 against poisoned pre-fix cached HTML.
DELIBERATE TRADEOFF: Cache-Control on /api/v1/img success responses remains
max-age=2592000 (30 days), unchanged from before. An aggressive "force
revalidate on every reuse" policy was prototyped during review but reverted
because the perf cost (a server round-trip on every image view, even with
304 saving the body bytes) outweighed the corner-case mitigation. The
realistic exposure of cache carryover is narrow: cache carryover only
affects users who navigated top-level to an attacker URL pre-fix and still
have it in their local cache — the normal <img> embed path cached text/html
but never executed it. Local browser caches that hold pre-fix bytes
continue to serve them until their 30-day TTL expires or are evicted under
memory pressure. The ETag bump reaches all clients that DO revalidate
during the cached lifetime (Ctrl+R, intermediaries, post-expiry use); for
the rest, exposure self-limits via cache expiry. Operators running a
CDN/edge cache in front of remark42 should purge /api/v1/img after deploy.
The /api/v1/img handler short-circuits on a matching current-version
If-None-Match before any store Load or upstream fetch, returning a bodyless
304 with the defense headers set. Safe because the 304 carries no body and
the client's cached bytes came from a prior validated 200; an attacker
fabricating an etag value can only short-circuit fetches for URLs they
themselves crafted. This avoids upstream DoS amplification when clients
revalidate on hot comment pages.
The /api/v1/img route was moved from the "open routes" group (which uses
middleware.NoCache, stripping If-None-Match from incoming requests) to the
"open routes, cached" group alongside /picture/ and /qr/telegram so the
304 revalidation path is no longer broken upstream of the handler.
The /picture/{user}/{id} endpoint does not need the v2 etag prefix. Upload
validates input format via readAndValidateImage and the serve path
re-validates the stored bytes via rest.SafeImgContentType. Bytes within
the resize dimension limits are preserved verbatim, so the browser defense
relies on the response headers (validated Content-Type + nosniff + strict
CSP + Content-Disposition: inline), not on byte normalization.
Global CSP: font-src data: → font-src 'none'. Audit confirmed no @font-face,
no base64 fonts, no icon-font library in the bundle. Drops an unnecessary
attack surface; no behavioural change.
Tests: TestImage_ContentTypeHandling table-tests a real PNG and attack
shapes (HTML claimed as image/png, image/jpeg, image/gif, image/svg+xml,
image/webp; svg with onload; html fragment; polyglot PNG+HTML), proving
the defense holds across arbitrary upstream Content-Type variation.
Polyglot case is intentionally served as image/png — the browser cannot
execute the trailing HTML when the response type is image/png with nosniff.
TestImage_ContentTypeHandling_CacheHit exercises the cache-hit branch with
attacker bytes preloaded into the store. TestImage_PerRequestRevalidation
alternates upstream PNG/HTML across four proxy calls to prove no trust
accumulates between requests. TestImage_RoutesUsingCachedImage asserts
cache-poisoning is caught at serve time. TestImage_EtagVersioned asserts
the v2 prefix invalidates pre-fix etags AND that the revalidation 304
triggers no store Load. TestImage_RevalidationSkipsIO proves the
short-circuit works even with no upstream reachable. TestSafeImgContentType
covers the allowlist directly. TestRest_LoadPictureDefenseHeaders and
TestRest_LoadPictureRejectsNonImage exercise the /picture/ endpoint.
TestRest_apiCSP covers the strict CSP middleware on JSON API + RSS routes;
TestRest_securityHeaders confirms /web/ HTML pages keep the global CSP.
Verified end-to-end against the dev docker image: the original demo URL
(arbitrary HTML claimed as image/png) now returns 415 application/json with
CSP/nosniff/Content-Disposition set, no XSS in the browser.
* fix(security): set Cache-Control: no-store on image-proxy error paths, sync stale route comment
Addresses two review comments on #2067:
1. Cache-Control: max-age=2592000 and Etag were set before the
load/download/validation block, so 404/400/415 error responses inherited
the 30-day cache TTL and the versioned etag — a transient failure (or an
intentionally triggered 415) would be pinned in browser/intermediary
caches for that TTL, keeping users locked out even after the underlying
cause was resolved. Now: etag is computed but not set as a header until
after validation succeeds; error paths route through sendImageProxyError
which sets Cache-Control: no-store and never sets Etag. The 304
short-circuit still sets both because that path serves the same validated
content the client already has cached.
2. The comment at rest.go:282 still described the prototyped
no-cache/must-revalidate Cache-Control policy that was reverted before
the PR landed. Updated to match the actual 30-day max-age behavior.
Tests: TestImage_ContentTypeHandling now asserts reject paths carry
Cache-Control: no-store and have no Etag header, and accept paths carry
the max-age=2592000 + v2: etag.
readAndValidateImage caps the byte size of incoming images but the resize()
helper that follows still called image.Decode unconditionally, allocating
pixel memory proportional to the *declared* image dimensions. A ~100 KB
compressed PNG or GIF that declares 65535x65535 px forces image.Decode to
allocate ~17 GB of raster, OOMing the service on a single comment upload
(or on the proxy's CacheExternal path when caching a malicious upstream).
Hardening:
- maxImagePixels = 16 MP constant. Covers any realistic image (~4096x4096)
while bounding peak allocation.
- resize() now runs image.DecodeConfig first (cheap, no pixel allocation)
to read declared width/height before any full decode.
- Multiplication of width × height uses int64 to defeat 32-bit overflow
(GOARCH=386, 32-bit arm): on those targets, int(cfg.Width)*int(cfg.Height)
could wrap below maxImagePixels and bypass the cap. GIF's 16-bit logical
screen and JPEG's 16-bit SOF dimensions both reach this if int-multiplied.
- Bytes exceeding the cap, or non-image input that fails DecodeConfig,
return nil. prepareImage propagates the rejection as a clear error
instead of storing the malformed/oversized data verbatim.
- The no-resize-needed path returns the validated original bytes verbatim
so animated GIFs round-trip without being flattened to a single frame.
The DecodeConfig precheck applies even when MaxWidth/MaxHeight are 0
(resize disabled) — the dimension cap is unconditional defense-in-depth.
Two adjacent fixes surfaced by the new resize contract:
1. readAndValidateImage previously did `data[:512]` without a bounds check,
panicking on any body shorter than 512 bytes. Now bounded with min().
2. image/webp was listed as an allowed format but no WebP decoder was
registered, so DecodeConfig would refuse legitimate WebP uploads. Added
`_ "golang.org/x/image/webp"` (already in go.mod via x/image/draw) so
the registered decoders match the allowlist.
Tests:
- TestService_resizeRejectsDecompressionBomb builds a 14-byte GIF87a header
declaring 65535x65535 and asserts resize() refuses it both at the unit
level and through SaveWithID end-to-end (no store write).
- TestService_SaveWithIDShortPayload regression-tests the short-body panic.
- TestService_SaveWithIDWebP regression-tests WebP round-trip through
prepareImage with the new DecodeConfig requirement.
- TestService_resize subtests updated to assert non-image bytes are now
refused (previously the helper fell back to returning the raw bytes
verbatim, letting malformed content reach the store).
* fix(auth): close OAuth open-redirect by wiring AllowedRedirectHosts
Bump go-pkgz/auth/v2 to master (v2.1.2-0.20260421203319-686683f19cf7)
which carries the `from` redirect validator from go-pkgz/auth#275.
The library default with a nil AllowedRedirectHosts is permissive
(preserves legacy behavior for existing consumers on a dep bump), so
just bumping the dep leaves remark42 vulnerable — a crafted
/auth/<provider>/login?from=https://evil.example.com/... still issues
the 307 to the attacker host after the user completes legitimate
OAuth. Verified end-to-end against a local dev-auth instance before
and after this commit.
Wire Opts.AllowedRedirectHosts in getAuthenticator to the operator's
existing --allowed-hosts config, stripping the CSP "self" sentinel
which is not a real hostname. RemarkURL's own host is always implicit
per the library contract, so a default single-site deployment gains
the protection with no config change. Multi-host embeds work as soon
as their embedding hosts are added to AllowedHosts (they already need
to be there for CSP frame-ancestors).
Refreshed vendor tree to match the new module version.
* chore(lint): suppress G703 false positives on image Save
CI's newer gosec flags os.MkdirAll/os.WriteFile in FileSystem.Save with
G703 because id flows in from the caller. id is validated at the HTTP
layer (safePictureSegment in rest_public.go) and dst is derived via
f.location — not a real traversal. Targeted //nolint with reason.
* fix(auth): normalise AllowedRedirectHosts entries + add unit test
Address Copilot review on PR #2049. The previous closure passed raw
s.AllowedHosts entries straight to the auth library, but --allowed-hosts
holds CSP frame-ancestors source expressions: scheme-prefixed values
(https://blog.example.com), entries with ports, and wildcards
(*.cdn.example.com) are all valid there but the auth library compares
against u.Hostname() and would silently drop them — breaking legitimate
redirects on multi-host deployments.
Extract getAllowedRedirectHosts that:
* trims whitespace, drops empty / 'self' / "self" / wildcard entries
* prepends https:// if scheme missing then url.Parse to extract Hostname
* logs a warning on parse failure rather than poisoning the allowlist
Wire the closure in getAuthenticator to call the helper.
Test_getAllowedRedirectHosts covers all the edge cases Copilot flagged
(scheme stripping, port handling, self spellings, wildcards, empty,
mixed real-world).
* fix(auth): preserve explicit port in AllowedRedirectHosts + clarify fs_store nolint
Address Copilot follow-up on PR #2049:
* getAllowedRedirectHosts stripped explicit ports via u.Hostname(), which
broadened the allowlist. The auth validator checks both Hostname() and
Host, so an entry like admin.example.com:8443 can and should be kept
host:port — allowing only that port, not any. Emit u.Host when
u.Port() != "", u.Hostname() otherwise. Updated tests.
* fs_store Save nolint rationale said "id validated at HTTP layer", but
Save is reached via image.Service.Save and SaveWithID (cache), neither
of which is HTTP validation. id is actually a server-generated hash in
both paths. Updated the comment.
Go 1.25's testing/synctest package (GA) provides a fake clock bubble
for deterministic goroutine and timer testing. Convert tests that
waited on real-time durations to use synctest, removing most wall-clock
time.Sleep workarounds.
Converted (11 tests, 9 files):
- notify/notify_test.go — all tests, replaced 17 time.Sleep(110ms) with synctest.Wait()
- store/service/service_test.go — VoteSameIPWithDuration, UserReplies, submitImages,
ResubmitStagingImages, deleteImagesOnCommentDelete
- store/image/{image,bolt_store}_test.go — Cleanup, Submit, SubmitDelay
- store/engine/bolt_test.go — FlagListBlocked
- providers/telegram_test.go — DispatchTelegramUpdates
- migrator/backup_test.go — TestBackup_Do
- _example/memory_store/accessor/data_test.go — FlagListBlocked
Simplifications along the way:
- notify/notify_mock.go: dropped the 10ms time.After delay and
ctx.Done select in MockDest — the artificial I/O simulation is
pointless and blocked synctest.Wait from draining the queue
- Removed three dead-code time.Sleep(1s) calls in EditCommentDurationFailed,
EditCommentAdmin, and Info tests: prepopulated comments from 2017
already exceed any EditDuration/ReadOnlyAge under real clock, making
the sleeps meaningless
- UserReplies: replaced the Eventually+Sleep+mutex polling with a
direct time.Sleep under fake clock
Skipped (incompatible with synctest):
- fs_store_test.go: relies on OS file mtime (real wall clock)
- rss_test.go: needs real wall-clock second boundary for pubDate
- admin/rest_private/rest_public tests: httptest network I/O
- cmd/server_test.go: real HTTP server startup polling
Notes on quirks encountered:
- synctest.Wait() does NOT advance fake time, contrary to what one
might expect. It only returns once all other bubble goroutines are
durably blocked. To advance the fake clock, the test goroutine must
itself call time.Sleep
- BoltDB keys the "last" bucket by comment.Timestamp nanosecond string.
Rapid b.Create calls under frozen fake time produce identical keys
and overwrite each other. TestService_UserReplies adds
time.Sleep(time.Nanosecond) between Creates to advance the clock
- Bolt image Cleanup uses strict age > ttl. Under fake time the
age-ttl delta is exactly zero at the boundary, so subtract 1ms from
the passed ttl to stay strictly under
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: umputun <535880+umputun@users.noreply.github.com>
Address all golangci-lint v2.10.1 (CI's version) findings:
* Add http.MaxBytesReader hard cap to ParseMultipartForm sites in
rest_private.savePictureCtrl (32MB) and api/migrator (256MB) — fixes
G120 by bounding total request body before form parsing.
* Suppress G70x in CLI subcommands cmd/{backup,cleanup,import,remap}.go:
all four issue HTTP requests against operator-supplied RemarkURL/CLI
flags, never user input. Each suppression carries a one-line reason.
* Suppress G122 in image fs_store cleanup walk: staging directory tree
is server-only, no untrusted symlinks land there.
The image proxy got an ssrfSafeTransport in commit aca0cff3 that resolves
DNS first, blocks any IP in private/reserved CIDRs, then dials by IP to
defeat DNS rebinding. The TitleExtractor used to construct comments'
PostTitle from Locator.URL — a user-supplied field — was missed by that
fix and kept using http.DefaultTransport. The hostname allowlist there
checks the parsed URL host but never the IP it resolves to, so a domain
suffix-matching an allowed host (or 127.0.0.1 itself when AllowedHosts
is empty) reaches the metadata service or any other internal endpoint.
The same gosec rule (G704) was excluded globally in .golangci.yml as part
of aca0cff3, so this gap was not caught by the linter either.
Extract the transport into a new safehttp package so it lives in one
place and can be reused, then pass safehttp.Transport() into the
TitleExtractor's http.Client at construction (cmd/server.go). The image
proxy switches to safehttp.Transport() too — same behaviour, no longer
duplicated.
Reproduction in title_test.go uses the production-style client to hit
an httptest.Server (always 127.0.0.1) and asserts the dialer refuses
even though "127.0.0.1" is in the allowed-domains list. A control case
shows the same setup without safehttp.Transport returns the page —
making the original vulnerability explicit.
The store tests stored timestamps with time.Local in their fixtures and
asserted equality against returned values that the engine round-trips
through UTC. assert.Equal compares zone identity, so on UTC machines
(CI, most cloud envs) Local==UTC and the tests passed; on a developer
machine in any other timezone (here BST, UTC+1) TestService_Put,
TestService_List, TestBoltDB_InfoPost, TestBoltDB_InfoList and several
others would fail with same wall-clock numbers but mismatched zones.
Replace time.Local with time.UTC across store/comment_test.go,
store/formatter_test.go, store/service/service_test.go,
store/engine/bolt_test.go, store/engine/engine_test.go. Production code
is untouched.
Apply go fix ./... analysers (Go 1.26) across backend and examples:
- interface{} → any (type alias, no behaviour change)
- for i := 0; i < N; i++ → for range N / for i := range N
- slices.Contains / slices.ContainsFunc replacing manual loops
- strings.SplitSeq replacing strings.Split in range (avoids allocation)
- strings.CutPrefix replacing HasPrefix+TrimPrefix
- min() replacing manual if/else
- fmt.Appendf replacing []byte(fmt.Sprintf(...))
- strings.Builder replacing string += concatenation
- wg.Go(func(){}) replacing wg.Add(1)/go/wg.Done() pattern
- removed redundant ii := i loop variable copies (unnecessary since Go 1.22)
omitempty on struct-typed JSON fields: go fix removed omitempty from
struct-typed fields (time.Time, PostInfo, UserDetailEntry) because
encoding/json's omitempty never applied to struct types — it was always
a no-op. Kept as bare tags (no omitzero replacement) to preserve the
existing serialisation behaviour.
Replace strings.Split(RemoteAddr, ":") with net.SplitHostPort for correct
IPv6 address extraction in vote deduplication and comment IP tracking.
Harden image proxy: add SSRF-safe transport blocking private/reserved IPs
at connection time with DNS rebinding protection, sanitize error messages
to prevent information leakage, add response size limit via io.LimitReader.
Fix shadowed error variables in BlockedUsers, SetTitle, and Delete methods.
Exclude gosec taint analysis false positives at linter config level.
When EditDuration is zero or negative, cleanupTTL becomes zero,
causing time.After(0) to fire immediately in a tight loop.
Block on ctx.Done() instead when edit duration is disabled. Fixes#1991
`format=tree` pagination provides top-level comments with all replies
and returns the last top-level comment as `last_comment` to be used
as `offset` for the next page. If comments and replies overflow
the limit, the one stepping out of the limit will not be returned.
If the first comment and its replies after the given offset overflow
the limit, it will be returned with all the replies.
`format=plain` pagination works by providing all comments and returning
the last comment as `last_comment` to be used as `offset`
for the next page.
MakeTree calculated Info locally for historical reasons,
and the results were consistent with the dataService.Info call
but calculated differently.
That change fixes that, ensuring that Info is requested
in the same manner.
Previously, images were deleted only from comments deleted
before EditDuration expiration. After this change, any deletion
of the comment deletes images if they are not used elsewhere
in comments under the same page.
Previously, only the first one was returned for site-wide requests,
and now all returned information will be correctly aggregated,
and the PostInfo.URL and PostInfo.ReadOnly parameters will be dropped.
Allowed domains consist of `REMARK_URL` second-level domain (or whole IP in case it's IP like `127.0.0.1`) and `ALLOWED_HOSTS`. That is needed to prevent Remark42 from asking arbitrary servers and storing the page title as the comment.PostTitle.
Previous behaviour allowed the caller of the API to create a comment
with an arbitrary URL and learn the title of the page, which might be
accessible to the server Remark42 is installed on but not to the user
outside that network (CWE-918).
Allowed domains consist of `REMARK_URL` second-level domain (or whole IP in case it's IP like `127.0.0.1`) and `ALLOWED_HOSTS`. That is needed to prevent Remark42 from asking arbitrary servers and storing the page title as the comment.PostTitle.
Previous behaviour allowed the caller of the API to create a comment
with an arbitrary URL and learn the title of the page, which might be
accessible to the server Remark42 is installed on but not to the user
outside that network (CWE-918).
Previously, we stripped unsafe HTML tags but left some,
but it's not expected to have a link in a title or username,
so the new behaviour is stripping everything.
Previously, proxied and local images were checked for presence in the
storage before previewing or posting the comment. That logic resulted in
an inability to post with an image when a proxy for images is enabled,
as proxied images are not downloaded to disk before the first time
someone loads them, which could only happen after the user either
previews or posts the message.
After this change, preview and post only checks the local images'
presence and ignore the proxied ones.
1) Current implementation simply removes the last word, without truncating up to limit length.
2) In case if even the first word (magnet link or some base64?) is too long don't add extra space.
(url) is a text inserted by default and never an intended URL.
That additional validation will ensure that users won't post relative
links because they are rarely intended.
I haven't found a linter for these, so I had to catch these manually.
I found #757 to fix one of these, and I thought it would be good
to fix everything at once.
Previously, the cache kept the entry and deletion of the parent comment
after child deletion was not possible for the rest
of cache life (5m) duration. Now it's possible to delete
a parent comment after the deletion of the child comment
by a user or admin.
Resolves#1481
