Commit Graph
40 Commits
Author SHA1 Message Date
Dmitry VerkhoturovandGitHub ba7c3aed94 refactor: modernise Go code with go fix and manual improvements (#2027)
Apply go fix ./... analysers (Go 1.26) across backend and examples:
- interface{} → any (type alias, no behaviour change)
- for i := 0; i < N; i++ → for range N / for i := range N
- slices.Contains / slices.ContainsFunc replacing manual loops
- strings.SplitSeq replacing strings.Split in range (avoids allocation)
- strings.CutPrefix replacing HasPrefix+TrimPrefix
- min() replacing manual if/else
- fmt.Appendf replacing []byte(fmt.Sprintf(...))
- strings.Builder replacing string += concatenation
- wg.Go(func(){}) replacing wg.Add(1)/go/wg.Done() pattern
- removed redundant ii := i loop variable copies (unnecessary since Go 1.22)

omitempty on struct-typed JSON fields: go fix removed omitempty from
struct-typed fields (time.Time, PostInfo, UserDetailEntry) because
encoding/json's omitempty never applied to struct types — it was always
a no-op. Kept as bare tags (no omitzero replacement) to preserve the
existing serialisation behaviour.
2026-03-25 16:42:37 -05:00
Umputun aca0cff399 fix: IPv6 address truncation and image proxy SSRF vulnerabilities
Replace strings.Split(RemoteAddr, ":") with net.SplitHostPort for correct
IPv6 address extraction in vote deduplication and comment IP tracking.

Harden image proxy: add SSRF-safe transport blocking private/reserved IPs
at connection time with DNS rebinding protection, sanitize error messages
to prevent information leakage, add response size limit via io.LimitReader.

Fix shadowed error variables in BlockedUsers, SetTitle, and Delete methods.
Exclude gosec taint analysis false positives at linter config level.
2026-02-28 04:13:07 -06:00
Dmitry Verkhoturov edfc5b9d76 Automatic fix of errors reported by golangci-lint v2
- Use strings.ReplaceAll
- Remove redundant internal structure names
2025-03-24 22:46:43 +01:00
Dmitry Verkhoturov 82a0888c42 add pagination to GET /api/v1/find endpoint
`format=tree` pagination provides top-level comments with all replies
and returns the last top-level comment as `last_comment` to be used
as `offset` for the next page. If comments and replies overflow
the limit, the one stepping out of the limit will not be returned.
 If the first comment and its replies after the given offset overflow
 the limit, it will be returned with all the replies.

`format=plain` pagination works by providing all comments and returning
the last comment as `last_comment` to be used as `offset`
for the next page.
2024-12-10 14:26:55 +00:00
Dmitry VerkhoturovandUmputun 618c267370 combine multiple post info in DataStore.Info instead of returning first
Previously, only the first one was returned for site-wide requests,
and now all returned information will be correctly aggregated,
and the PostInfo.URL and PostInfo.ReadOnly parameters will be dropped.
2023-11-04 12:49:40 -05:00
Dmitry VerkhoturovandUmputun 7a71d47556 remove all HTML tags from comment title and username
Previously, we stripped unsafe HTML tags but left some,
but it's not expected to have a link in a title or username,
so the new behaviour is stripping everything.
2023-10-10 12:41:26 -05:00
Yury KotovandUmputun add01455fb Fix snippet generation
1) Current implementation simply removes the last word, without truncating up to limit length.
2) In case if even the first word (magnet link or some base64?) is too long don't add extra space.
2023-07-04 13:32:37 -05:00
Dmitry VerkhoturovandUmputun 26f82ad95c don't allow relative links in comments
(url) is a text inserted by default and never an intended URL.

That additional validation will ensure that users won't post relative
links because they are rarely intended.
2023-04-09 23:30:21 -05:00
Dmitry VerkhoturovandUmputun 596861a594 don't remove the twitter-tweet class from blockquote
This is needed to format the Twitter blockquotes as tweets.
2023-01-09 03:20:54 -06:00
Dmitry Verkhoturov 243c8356e7 do not sanitise the original comment markdown
Previously it was sanitised using the HTML sanitiser,
but it had proven troublesome and unnecessary.
Remark42 rendered the markdown into proper HTML, but then some pieces
of it (like cited HTML code inside the code block, marked by backticks)
were cut out, which then showed the incorrect markdown to a user when
they were editing the comment.

For example, the comment "`foo<bar>`" became "foo" after sanitising,
and despite the proper render user saw only "foo" when editing
the comment.

After this change, the initial comment markdown is preserved unaltered.
It could contain dangerous HTML with JS, which I assume shouldn't
be a problem as it's never rendered as HTML but instead supposed
to be converted to HTML by the interpreter. In Remark42, it's stored
in a comment.Text field and sanitised and thus safe.

I've left information about the potential danger of rendering
the original markdown as-is without an interpreter in
all relevant places I could find.
2022-07-22 01:08:08 +02:00
Dmitry VerkhoturovandUmputun 9c1a827685 update alecthomas/chroma and Depado/bfchroma to v2 2022-07-13 20:29:46 -05:00
Umputun 598d767791 resolves #415 and the similar issue with title 2022-04-29 10:52:05 -05:00
Dmitry VerkhoturovandUmputun 90e537358d update golangci-lint to 1.43.0, fix found issues 2021-11-23 15:00:40 -06:00
Umputun 91441d1160 add controversy and imported to untrusted list 2021-09-01 14:21:52 -05:00
Umputun e90dae2b94 sanitize Title on find level as well 2021-09-01 14:14:12 -05:00
Umputun a7b44eee1a sanitize PostTitle 2021-09-01 13:56:06 -05:00
Umputun aa14069f8b lint: fix func comment 2021-03-26 16:05:41 -05:00
Umputun ddbceb5375 post-sanitize location's url in case if stored already #927
this is done via service.alterComment the same way as we handle
leftovers from prev issues with results stored already
2021-03-26 16:00:50 -05:00
Umputun 5ed4c7100c shorter match condition 2021-03-26 15:06:39 -05:00
Umputun e762ea7b91 sanitize both locator.URL and user.Picture 2021-03-26 15:06:39 -05:00
Umputun 1048355b7b strip voted_ip from response 2021-01-18 15:44:32 -06:00
Umputun 5296074dae fix sanitize policy to allow loading-lazy for images #855 2021-01-18 15:36:15 -06:00
vdimirandUmputun 9957ab642b Add Imported field to comment, do not wait image submission for such comments 2020-07-09 12:43:07 -05:00
Dmitry VerkhoturovandUmputun 513c8f353d sticter golangci config, fix discovered issues 2020-04-19 16:21:09 -05:00
1e548eb85f Code colors (backend support) (#590)
* testing code highlight with bfchroma render

* add bfchroma vendoring

* render code htm with classes

* adjust bfchroma test cases inside formatter

* go mod tidy

* fix markdown code render tests

* adjust comment text sanitizer to permit code HTML classes

* go mod tidy

* add chroma classes list regexp

* unify gitignore entries style

Co-authored-by: Umputun <umputun@gmail.com>
2020-02-08 21:11:12 -06:00
Umputun 8d109901b8 Merge branch 'master' of github.com:umputun/remark 2019-08-20 00:07:19 -05:00
Umputun 07b16c46dc unescape quotes from user name #415 2019-08-20 00:07:13 -05:00
UmputunandGitHub 2e90b6172b Votes ip (#416)
* change vote params to request

* limit voting for the same ip

* limit same ip vote duration

* add same ip vote check for directions

* wire RestrictVoteIP and duration

* add votes-ip and votes-ip-time description
2019-08-19 14:04:12 -05:00
Umputun dfaf4d13d3 add parent summary to rss description #339 2019-07-14 00:00:02 -05:00
Umputun 2e4dfe3891 move vote setter to service level 2019-04-07 14:05:19 -05:00
Umputun 446cb486a0 extend test fo votes 2019-04-07 13:47:22 -05:00
Umputun 3b5a1a62de add vote for the current user, hide list of other votes #297 2019-03-25 16:14:10 -05:00
Umputun 7128f66374 add controversy value (votes metric) #274 2019-02-10 15:53:29 -06:00
Umputun 690a0aa763 add post title to comment 2019-01-05 04:24:49 -06:00
UmputunandGitHub 30ff1dacde feature/artifact (#190)
* embed assets for web with separate dockerfile

* add artifact dockerfile

* add artifacts upload to github release

* fix statik params and add make for local builds

* fix docker image name to match compose

* fix typos
2018-08-12 14:24:46 -05:00
UmputunandGitHub 2ff955edf1 feature/comment-formater (#186)
* extract comment formatter functuionality to allow md in #156

* add abilty to pass multiple convertors to formater
2018-08-10 19:35:45 -05:00
UmputunandGitHub de013c7d5a feature/mongo (#165)
* WIP: start mongo engine

* WIP: mongo accessor and admin

* integrate mongo store to main

* disable mongo tests by default, only in CI

* connection with constructor

* add mongo buffered writer

* buffered mongo writer

* fix nil responses on an empty list from mongo

* missing mongo index for scores

* cancelable store

* add gridfs implementation of avatar store

* fix race on mongo session copy

* gridfs avatars without tmp files

* move avatar store

* minor comments and refactoring for avatar store

* merged from current master

* simplify gridfs reader

* lint: fix minor warns

* test mongo against env defined url

* pass MONGO_REMARK_TEST to docker and travis

* set dockerfile env for mongo test url

* increase connect timeout in mongo tests

* pass MONGO_REMARK_TEST to drone build

* add MONGO_REMARK_TEST to branch stage of drone

* mass mongo test url via build_args_from_env

* populate mongo IP to docker build hosts

* test env

* pass mongo ip via .mongo

* remove .mongo temp from git

* add .mongo -> env to linter step

* allow more time to autoflush writer test

* default mongo tests to "mongo" if not in env

* merge fresh master into

* add test for mongo cleanup

* msg for a failed test

* lazy fix for failed test

* add an ability to skip all mongo tests

* add backend dev instructions

* remove unused code from mongo server

* move mongo testing to connection_test

* restore testing.go

* lint: minor warns for testing code
2018-07-13 12:27:11 -05:00
UmputunandGitHub 7a6eca56b1 feature/jwtcleanup (#113)
* remove extra dmin check, add blocking check via jwt and auth middleware
* short jwt and refresh for expired
* lint: missing comment on UserFlager
* simplify user management in auth refresh
* allow custom max cookie age
* test blocked user
* reset cookie for blocked user
* move admin perm detection to data service
* customizable ttl with opts as a part of auth group
* add local auth provider dev
* main minimal test for dev auth mode
* add comments and update docs with current params
* add admin and auth_dev flags
* comments for dev compose
* lint: shadow err
2018-06-27 01:43:13 -05:00
UmputunandGitHub c316b27fe1 feature/block ttl (#108)
* add ttl to blocking #88

* remove unused parsing code

* add comments explaining TTL for blocking
2018-06-24 15:43:32 -05:00
UmputunandGitHub 7aee7add08 Feature/backend (#101)
* move backend code #99

* adjust drone build for backend subdir

* move .rest to backend
2018-06-23 12:31:19 -05:00