* Read the collapsed-threads key through getJsonItem
`getFromLocalStorage` parsed the stored string directly, so anything
malformed under `__remarkCollapsed` threw out of `restoreCollapsedThreads`.
That call sits in `remark.tsx` ahead of the `render`, so the throw took the
whole widget with it: the reader was left on the preloader, over a view
preference.
`getJsonItem` in `common/local-storage.ts` already wraps a parse of a
localStorage key and returns null on failure, and null is a shape the check
below already reads as empty. The rest of that function is total against
whatever the browser holds, and the bare parse was the one way in.
* Stop retrying a failed e2e test in CI
The suite went in with one gotestsum rerun. It has no failures on record to
justify that: 31 CI runs since it landed, all green, and no rerun report has
ever been produced. A retry is what turns an intermittent regression into a
green build, and while the suite is this young its own failures are the
evidence worth keeping.
`E2E_RUN_ID` stays. It stamps the threads a run works on with the CI run id, so
a thread url in a trace or a log names the run it came from. It carries no data
across: the stack is disposable, and a local run under the same id gets those
urls on an empty database.
* Stop two chooseUnusedPort comments claiming collisions cannot happen
All four copies listen on :0, read the assigned port, close the listener
and bind later, so nothing holds the number across that gap and another
binary can take it. The copies in app/cmd and app/rest/api call a collision
very unlikely, which is accurate; the ones in app and the example module
said binaries never land on the same number, which is not, and a comment
ruling out a port collision is what would send the next person chasing one
somewhere else. All four now read the same.
Closing the window rather than describing it means the server binding :0
itself and reporting the address it got, which is a larger change.
* Make backend tests wait on conditions instead of durations
The backend workflow has a long tail of runs that fail once and pass on
a rerun. Every one of them comes down to a test assuming an operation
finishes within some duration rather than waiting for the state it
needs. Three were reproducible and each was reproduced against the old
code before being changed: TestServerAuthHooks minted a token that lived
one second and never tested expiry, so a slow runner turned the first
POST into a 401; TestServerApp_AnonMode saw "connection refused" because
waitForHTTPServerStart returned silently after three seconds and left a
later assertion to fail with something unrelated; TestFsStore_Cleanup
slept 200ms against a 300ms ttl that Cleanup widens to 400ms with its
commit grace, so roughly 100ms of stall collected an image meant to
survive.
Fixed sleeps before asserting on asynchronous work are replaced with
polls on the condition itself, using require.Eventually and
require.EventuallyWithT, and require.Never where the assertion is that
something did not happen. Polling closures assert on the CollectT they
are handed rather than on t, since testify runs them on another
goroutine, and polls that issue HTTP requests stay under the rate limit
on the routes they poll through.
Where a test needs time to have passed, the clock input is pinned
instead: staging ages are stamped with os.Chtimes on both sides of the
cleanup boundary right before each call, which also makes the 100ms
commit grace an exact case rather than something no assertion reaches,
and the RSS tests set store.Comment.Timestamp explicitly rather than
racing the wall clock into the first 100ms of a second so pubDate
matches.
chooseUnusedPort takes a port from the kernel's ephemeral range. Picking
at random out of a fixed 10000-port window let two package binaries,
which go test ./... runs concurrently, land on the same number between
the probe closing and the server binding. The start helpers fail naming
the port they waited on, and the SSL tests wait on the redirect port as
well as the TLS one.
Arbitrary budgets that nothing tests are gone: ten HTTP clients with a
one-second timeout against bolt-backed import and export, the "should
take about 100msec" assertions, and a one-second bound on noticing an
already cancelled context. Shutdown stays bounded at ten seconds so a
hang is still caught.
Two assertions get stronger. TestServerAuthHooks accepted 403 or 401
from a blocked user, an alternative that existed only because the short
token could expire mid-test; it is deterministically 403 now.
TestAdmin_BlockedList asserted two users blocked while one carried the
same 150ms ttl the next step waits to lapse, so the halves raced each
other.
goleak stops reporting the regexp2 clock goroutine, which chroma pulls
in for syntax highlighting and which lives for up to a second after the
last match with a timeout; it ends on its own but a binary finishing
inside that window was reported as leaking, and this suite now finishes
sooner. The ignore for net/http.(*Server).Shutdown goes the other way:
it no longer matches anything, with both packages run fifteen times each
under CPU oversubscription to confirm.
Two gaps the change would otherwise have opened are covered directly
rather than left to the side effects that used to cover them. The
one-second token was the only thing exercising the authenticator's
ClaimsUpd hook on refresh, so TestServerApp_ClaimsUpd now calls the hook
itself and checks admin, blocked, email and restricted-name
impersonation, including the two pass-through cases. Lifting the
open-route limit removed the last incidental exercise of the rate
limiter, so TestRateLimiter drives a burst past the allowance and checks
the refusals and that the limit is per client. Both run without a wall
clock, and both were confirmed to fail when the behaviour they cover is
removed.
Production code is untouched. The two sleeps outside test code, the 429
backoff in cmd/cleanup.go and the submit poll in store/image/image.go,
are left alone: no CI failure implicates them.
Test sleeps drop from 67 to 21, all of them either inside a
testing/synctest bubble or a poll interval. The suite runs in about 22
seconds instead of 46, mostly because
TestPublic_FindCommentsCtrl_ConsistentCount no longer paces a hundred
subtests with an 80ms sleep each to stay under the open route limit. The
300s per-package budget now matches across both workflows, the race_test
target and the documented command, and CLAUDE.md records the convention.
with '#' will be ignored, and an empty message aborts the commit. # #
Date: Sat Aug 22 01:12:31 2026 +0100 # # interactive rebase in progress;
onto 7c312da1 # Last command done (1 command done): # reword deb6cbf1 #
Make backend tests wait on conditions instead of durations # Next
command to do (1 remaining command): # reword 262e6dc2 # Apply go fix
under Go 1.27 # You are currently editing a commit while rebasing branch
'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed:
.github/workflows/release.yml # modified: CLAUDE.md # modified: Makefile
modified: backend/_example/memory_store/server/rpc_test.go # modified:
backend/app/cmd/import_test.go # modified:
backend/app/cmd/server_test.go # modified: backend/app/main_test.go #
modified: backend/app/rest/api/admin_test.go # modified:
backend/app/rest/api/middleware_test.go # modified:
backend/app/rest/api/migrator_test.go # modified:
backend/app/rest/api/rest_private_test.go # modified:
backend/app/rest/api/rest_public_test.go # modified:
backend/app/rest/api/rest_test.go # modified:
backend/app/rest/api/rss_test.go # modified:
backend/app/rest/proxy/image_test.go # modified:
backend/app/store/image/fs_store_test.go # modified:
backend/app/store/service/service_test.go # modified:
docs/backlog/api-tests-deadlock-on-macos.md #
* Apply go fix under Go 1.27
Go 1.27 extends go fix with the modernizers, so `go fix ./...` now
rewrites patterns the language has since replaced. Running it across all
three modules produces this: legacy sync/atomic calls on plain integers
become the atomic types (notify.Service.closed, image.Service.term and
submitCount, and several test counters), reverse index loops become
slices.Backward, a Split-then-index becomes strings.Cut, counted loops
become range over an int, and interface{} becomes any in the e2e suite.
The example module needed no changes. The e2e module is behind a build
tag, so it only matches with `go fix -tags e2e ./...`.
One knock-on: prealloc can see the bound of a loop once it is written as
range over an int, so the slice it feeds is now preallocated.
with '#' will be ignored, and an empty message aborts the commit. # #
Date: Sat Aug 22 01:32:09 2026 +0100 # # interactive rebase in progress;
onto 7c312da1 # Last commands done (2 commands done): # reword deb6cbf1
262e6dc2 # Apply go fix under Go 1.27 # No commands remaining. # You are
currently editing a commit while rebasing branch
'fix/backend-test-flakiness' on '7c312da1'. # # Changes to be committed:
backend/app/migrator/native.go # modified: backend/app/notify/notify.go
backend/app/rest/api/rest_private_test.go # modified:
backend/app/store/comment.go # modified:
backend/app/store/image/image.go # modified:
backend/app/store/service/service_test.go # modified:
backend/app/store/service/title_test.go # modified: e2e/e2e_test.go #
modified: e2e/widgets_test.go #
Dependabot updates `backend/` only, so the example module that replaces
it with `../../` keeps the old versions as indirect entries and the
`test examples` job fails with `go: updates to go.mod needed`.
Beyond testify itself this picks up the yaml module move, from
`gopkg.in/yaml.v3` to `go.yaml.in/yaml/v3`, and drops two indirect
entries nothing needs any more.
* feat: add configurable SMTP HELO hostname
Allow the SMTP HELO/EHLO hostname to be configured separately from
the SMTP server hostname.
This is useful when the SMTP server requires clients to identify
themselves with a fully qualified hostname different from the server
address.
* chore: remove vendored dependency changes
* Bump go-pkgz/notify to v1.4.0 and document SMTP_HELO_HOST
The HELOHost field lands in go-pkgz/notify v1.4.0, so the branch needs the
bump to compile; v1.3.0 in master has no such field. The example module is
tidied alongside, as any change to backend/go.mod requires.
Documents the parameter in the parameters table and, separately, in the email
setup page: what it does, that leaving it unset keeps the previous `localhost`
greeting, and the case it exists for, a relay refusing the greeting under
Postfix `reject_non_fqdn_helo_hostname`.
Also records the current limit: verification emails for email authentication
go through go-pkgz/auth's own sender, which has no equivalent setting, so the
greeting there is unchanged.
* Bump go-pkgz/auth to v2.2.0 and apply SMTP_HELO_HOST to verification email
The verification email sender had no way to set the greeting, so a relay that
refuses the HELO would accept notifications and still reject sign-in emails.
EmailParams gains HELOHost in go-pkgz/auth v2.2.0, so the same SMTP_HELO_HOST
now drives both paths.
The example module is tidied alongside, as any change to backend/go.mod
requires.
---------
Co-authored-by: oli <someone@somewhere.tld>
Co-authored-by: Dmitry Verkhoturov <paskal.07@gmail.com>
rest.CORS refuses "*" together with credentials since go-pkgz/rest#52, so the
bump and the option have to land together: the option does not exist in v1.22.0
and the panic fires at construction, inside routes(), which makes it a startup
failure rather than a request-time one.
The wildcard stays. The comment widget is embedded on arbitrary third-party
sites, so the set of origins is not knowable, which is why the escape hatch was
asked for upstream instead of accepting the panic. What it costs is unchanged
and now written next to the call: any site a signed-in user visits can read
authenticated responses, so state-changing requests have to keep being protected
by something other than the origin, X-XSRF-Token today.
The example module is tidied in the same commit, as it reaches go-pkgz/rest
through the replace directive and its indirect graph would otherwise keep the
old pin and fail the readonly module check in CI.
The bump also carries testify to v1.12.0, which drops go-spew and go-difflib
from the module graph.
Excessive memory allocation during VP8L decoding, reachable from
app/store/image/image.go:333 where image.Decode runs on uploaded data.
The existing DecodeConfig dimension guard doesn't cover it, since the
over-allocation happens during decode rather than from declared dimensions.
Tidies backend/_example/memory_store in the same commit: it carries the
backend's deps as indirect entries and would otherwise fail the example CI step.
notify/email.go accumulated multi-recipient errors with
multierror.Append(fmt.Errorf(...)) instead of
multierror.Append(result, ...), so the accumulator was overwritten each
iteration and only the last failing recipient's error survived; earlier
failures were silently dropped. The telegram notifier did it correctly.
Replace hashicorp/go-multierror with the stdlib errors.Join everywhere
it was used (notify/email.go, notify/telegram.go, rest/api/rest_private.go,
store/service/service.go, store/image/image.go and store/engine/bolt.go),
which fixes the bug and drops the direct dependency. It stays indirect
because go-pkgz/lcw/v2 still imports it. A regression test in
email_test.go now sends two failing recipients and asserts both errors
are reported.
v1.22.0 includes the preflight Vary fix (https://github.com/go-pkgz/rest/pull/44):
rest.CORS now adds Vary: Access-Control-Request-Method and
Access-Control-Request-Headers on preflight itself, making the local wrapper
that added them redundant. corsMiddleware now returns rest.CORS directly;
TestCorsMiddleware still asserts those preflight Vary headers, now supplied
upstream.
Also tidies the _example/memory_store module for the new version.
Update all Go modules in backend/ and backend/_example/memory_store/ to
their latest versions (chroma 2.27, go-redis 9.21, bbolt 1.5, slack 0.27,
golang.org/x/* and others); re-tidy and re-vendor, keep the example module
in sync.
Hold github.com/go-chi/chi/v5 at v5.2.5: v5.3.0 deprecates
middleware.RealIP (IP-spoofing advisories). Switching off RealIP changes
how the client IP is derived for rate limiting and votes, which is a
security decision better made on its own rather than inside a dependency
bump.
go test -race, go vet, golangci-lint and govulncheck all clean on both
modules.
Go 1.25's testing/synctest package (GA) provides a fake clock bubble
for deterministic goroutine and timer testing. Convert tests that
waited on real-time durations to use synctest, removing most wall-clock
time.Sleep workarounds.
Converted (11 tests, 9 files):
- notify/notify_test.go — all tests, replaced 17 time.Sleep(110ms) with synctest.Wait()
- store/service/service_test.go — VoteSameIPWithDuration, UserReplies, submitImages,
ResubmitStagingImages, deleteImagesOnCommentDelete
- store/image/{image,bolt_store}_test.go — Cleanup, Submit, SubmitDelay
- store/engine/bolt_test.go — FlagListBlocked
- providers/telegram_test.go — DispatchTelegramUpdates
- migrator/backup_test.go — TestBackup_Do
- _example/memory_store/accessor/data_test.go — FlagListBlocked
Simplifications along the way:
- notify/notify_mock.go: dropped the 10ms time.After delay and
ctx.Done select in MockDest — the artificial I/O simulation is
pointless and blocked synctest.Wait from draining the queue
- Removed three dead-code time.Sleep(1s) calls in EditCommentDurationFailed,
EditCommentAdmin, and Info tests: prepopulated comments from 2017
already exceed any EditDuration/ReadOnlyAge under real clock, making
the sleeps meaningless
- UserReplies: replaced the Eventually+Sleep+mutex polling with a
direct time.Sleep under fake clock
Skipped (incompatible with synctest):
- fs_store_test.go: relies on OS file mtime (real wall clock)
- rss_test.go: needs real wall-clock second boundary for pubDate
- admin/rest_private/rest_public tests: httptest network I/O
- cmd/server_test.go: real HTTP server startup polling
Notes on quirks encountered:
- synctest.Wait() does NOT advance fake time, contrary to what one
might expect. It only returns once all other bubble goroutines are
durably blocked. To advance the fake clock, the test goroutine must
itself call time.Sleep
- BoltDB keys the "last" bucket by comment.Timestamp nanosecond string.
Rapid b.Create calls under frozen fake time produce identical keys
and overwrite each other. TestService_UserReplies adds
time.Sleep(time.Nanosecond) between Creates to advance the clock
- Bolt image Cleanup uses strict age > ttl. Under fake time the
age-ttl delta is exactly zero at the boundary, so subtract 1ms from
the passed ttl to stay strictly under
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: umputun <535880+umputun@users.noreply.github.com>
Bump Go dependencies in both backend/ and backend/_example/memory_store.
Notable updates:
- github.com/go-pkgz/lgr v0.12.1 -> v0.12.3
- github.com/klauspost/compress v1.18.2 -> v1.18.5
- github.com/PuerkitoBio/goquery v1.11.0 -> v1.12.0
- github.com/montanaflynn/stats v0.7.1 -> v0.9.0
- github.com/redis/go-redis/v9 v9.17.2 -> v9.18.0
- github.com/slack-go/slack v0.17.3 -> v0.21.1
- go.mongodb.org/mongo-driver v1.17.6 -> v1.17.9
- golang.org/x/crypto v0.48.0 -> v0.50.0
- golang.org/x/net v0.49.0 -> v0.53.0
- golang.org/x/image v0.36.0 -> v0.39.0
- golang.org/x/sys v0.41.0 -> v0.43.0
- golang.org/x/{oauth2,sync,text} minor bumps
Key markdown/sanitisation libs (bluemonday v1.0.27,
alecthomas/chroma/v2 v2.23.1, russross/blackfriday/v2 v2.1.0,
Depado/bfchroma/v2 v2.0.0) are already at the latest available
versions and were not bumped.
Verified the Chroma span-class allowlist regex in
backend/app/store/comment.go:128-131 is still fully in sync with
chroma/v2 types.go StandardTypes map (86 classes, byte-equal after
sorting). The inline comment references commit c263f6f which is
stale (Chroma is at v2 now), but the class list content is current.
Ran `go mod tidy` + `go mod vendor` + full race test suite on both
modules. All green. Added a reminder in CLAUDE.md that updating
backend/ Go modules also requires `go mod tidy` in
backend/_example/memory_store since the example module uses a
local replace directive and inherits indirect deps from the main
module.
Apply go fix ./... analysers (Go 1.26) across backend and examples:
- interface{} → any (type alias, no behaviour change)
- for i := 0; i < N; i++ → for range N / for i := range N
- slices.Contains / slices.ContainsFunc replacing manual loops
- strings.SplitSeq replacing strings.Split in range (avoids allocation)
- strings.CutPrefix replacing HasPrefix+TrimPrefix
- min() replacing manual if/else
- fmt.Appendf replacing []byte(fmt.Sprintf(...))
- strings.Builder replacing string += concatenation
- wg.Go(func(){}) replacing wg.Add(1)/go/wg.Done() pattern
- removed redundant ii := i loop variable copies (unnecessary since Go 1.22)
omitempty on struct-typed JSON fields: go fix removed omitempty from
struct-typed fields (time.Time, PostInfo, UserDetailEntry) because
encoding/json's omitempty never applied to struct types — it was always
a no-op. Kept as bare tags (no omitzero replacement) to preserve the
existing serialisation behaviour.
* migrate golangci-lint to v2 and update go version
- migrated .golangci.yml to version 2 format
- updated go.mod from 1.23.0 to 1.24
- removed deprecated run.timeout configuration
* update to go 1.25 and baseimage v1.17.0
- updated go.mod to go 1.25
- updated Dockerfile to use buildgo-v1.17.0 (go 1.25.0)
- updated Dockerfile to use app-v1.17.0
* fix flaky tests with proper synchronization
- use assert.Eventually instead of fixed sleep in TestService_Many
- wait for webhook before shutdown in TestMain_WithWebhook
- fixes race conditions exposed by Go 1.25 scheduler changes
* fix data race in MockDest.closed field
- add IsClosed() method with proper locking
- add locking to String() method
- use IsClosed() in tests instead of direct field access
- fixes race condition detected by go test -race
* update example go.mod to go 1.25
* update golangci-lint to v2.6.0 for go 1.25 support
* fix linter issue and update CLAUDE.md
- merge conditional assignment in example accessor/data.go
- add reminder in CLAUDE.md to always test and lint examples before committing
* update example Dockerfile to baseimage v1.17.0 for go 1.25
This commit replaces all references to `umputun/remark42` Docker images
on Docker Hub with `ghcr.io/umputun/remark42` from the GitHub
Container Registry. It updates various Docker Compose files,
documentation, and the Makefile to use the new image location.
It also updates the kubernetes example to use the latest version.
Docker Hub is going to kill free pulls for too long by now.
Previously, images were deleted only from comments deleted
before EditDuration expiration. After this change, any deletion
of the comment deletes images if they are not used elsewhere
in comments under the same page.
Without this option, the aud is ignored.
It works only with RPC admin storage.
The shared key returned for all requests with the default shared admin
storage, so enabling that option does not affect it.
Dmitry VerkhoturovandGitHub
dependabot[bot]andDmitry Verkhoturov
Vladimir DandUmputun