Commit Graph
317 Commits
Author SHA1 Message Date
Dmitry Verkhoturov 82a0888c42 add pagination to GET /api/v1/find endpoint
`format=tree` pagination provides top-level comments with all replies
and returns the last top-level comment as `last_comment` to be used
as `offset` for the next page. If comments and replies overflow
the limit, the one stepping out of the limit will not be returned.
 If the first comment and its replies after the given offset overflow
 the limit, it will be returned with all the replies.

`format=plain` pagination works by providing all comments and returning
the last comment as `last_comment` to be used as `offset`
for the next page.
2024-12-10 14:26:55 +00:00
Dmitry VerkhoturovandUmputun 7604548035 Update go to 1.23, golangci-lint to 1.61, baseimage to 1.14 2024-11-08 16:19:58 -06:00
Dmitry VerkhoturovandUmputun 661f042cb4 pin golangci-lint version to latest available, fix reported errors 2024-05-09 22:32:00 -05:00
Dmitry VerkhoturovandUmputun e5743185b0 collect /find Info for tree and plain types consistently
MakeTree calculated Info locally for historical reasons,
and the results were consistent with the dataService.Info call
but calculated differently.

That change fixes that, ensuring that Info is requested
in the same manner.
2024-03-16 12:49:45 -05:00
Dmitry VerkhoturovandUmputun 1313dee829 update to lcw v2 with generic types 2024-02-20 14:15:35 -06:00
Dmitry VerkhoturovandUmputun 532573fb34 fix problems reported by golangci-lint 2024-02-20 12:10:39 -06:00
Dmitry VerkhoturovandUmputun 81c30e01f8 cleanup images from deleted comments
Previously, images were deleted only from comments deleted
before EditDuration expiration. After this change, any deletion
of the comment deletes images if they are not used elsewhere
in comments under the same page.
2024-01-20 13:29:06 -06:00
Dmitry VerkhoturovandUmputun d143932924 add MIN_COMMENT_SIZE parameter 2023-12-02 12:16:26 -06:00
Dmitry Verkhoturov dcc7613409 allow disabling fancy HTML formatting
It might be necessary if the comments should preserve
original quotes instead of replacing them with angled ones.
2023-11-26 09:13:53 +01:00
Dmitry VerkhoturovandUmputun cd481d401d add tests for admin Store and DataService 2023-11-04 12:49:40 -05:00
Dmitry VerkhoturovandUmputun 618c267370 combine multiple post info in DataStore.Info instead of returning first
Previously, only the first one was returned for site-wide requests,
and now all returned information will be correctly aggregated,
and the PostInfo.URL and PostInfo.ReadOnly parameters will be dropped.
2023-11-04 12:49:40 -05:00
Dmitry VerkhoturovandUmputun 307866f7f5 simplify BoltDB.Info code
The new code does the same as the old one but doesn't call the checkFlag
in case ReadOnly is already set based on age.
2023-11-04 12:46:38 -05:00
Dmitry VerkhoturovandUmputun efceed6f68 limit TitleExtractor to allow only Remark42 whitelisted domains
Allowed domains consist of `REMARK_URL` second-level domain (or whole IP in case it's IP like `127.0.0.1`) and `ALLOWED_HOSTS`. That is needed to prevent Remark42 from asking arbitrary servers and storing the page title as the comment.PostTitle.

Previous behaviour allowed the caller of the API to create a comment
with an arbitrary URL and learn the title of the page, which might be
accessible to the server Remark42 is installed on but not to the user
outside that network (CWE-918).
2023-10-10 23:34:35 -05:00
Dmitry VerkhoturovandUmputun f4358173c7 limit TitleExtractor to allow only Remark42 whitelisted domains
Allowed domains consist of `REMARK_URL` second-level domain (or whole IP in case it's IP like `127.0.0.1`) and `ALLOWED_HOSTS`. That is needed to prevent Remark42 from asking arbitrary servers and storing the page title as the comment.PostTitle.

Previous behaviour allowed the caller of the API to create a comment
with an arbitrary URL and learn the title of the page, which might be
accessible to the server Remark42 is installed on but not to the user
outside that network (CWE-918).
2023-10-10 23:34:35 -05:00
Dmitry VerkhoturovandUmputun 7a71d47556 remove all HTML tags from comment title and username
Previously, we stripped unsafe HTML tags but left some,
but it's not expected to have a link in a title or username,
so the new behaviour is stripping everything.
2023-10-10 12:41:26 -05:00
Umputun 9ad4f0b75e lint: remove deprecated rand.Seed from tests 2023-08-07 13:09:38 -05:00
Dmitry VerkhoturovandUmputun c72f30eabb remove proxied images from sanity check
Previously, proxied and local images were checked for presence in the
storage before previewing or posting the comment. That logic resulted in
 an inability to post with an image when a proxy for images is enabled,
 as proxied images are not downloaded to disk before the first time
 someone loads them, which could only happen after the user either
 previews or posts the message.

After this change, preview and post only checks the local images'
presence and ignore the proxied ones.
2023-07-23 12:10:52 -05:00
Yury KotovandUmputun add01455fb Fix snippet generation
1) Current implementation simply removes the last word, without truncating up to limit length.
2) In case if even the first word (magnet link or some base64?) is too long don't add extra space.
2023-07-04 13:32:37 -05:00
Dmitry VerkhoturovandUmputun 26f82ad95c don't allow relative links in comments
(url) is a text inserted by default and never an intended URL.

That additional validation will ensure that users won't post relative
links because they are rarely intended.
2023-04-09 23:30:21 -05:00
Dmitry VerkhoturovandUmputun 1b90604b2d update go modules, fix Apple auth redirect
Previously the redirect after successful authentication didn't work.
2023-04-02 03:47:35 -05:00
Dmitry VerkhoturovandUmputun a03c002df4 bump golangci-lint to latest 2023-04-02 01:21:33 -05:00
Dmitry VerkhoturovandUmputun c70a66a1c5 bump go modules 2023-01-21 13:30:44 -06:00
Dmitry VerkhoturovandUmputun 596861a594 don't remove the twitter-tweet class from blockquote
This is needed to format the Twitter blockquotes as tweets.
2023-01-09 03:20:54 -06:00
Dmitry VerkhoturovandUmputun d7e9be99f9 make Close() calls idempotent
Previously, few of them resulted in panics when called more than once.
2023-01-03 01:41:26 -06:00
Dmitry VerkhoturovandUmputun c86bff8811 remove duplicate type definitions from function signatures
I haven't found a linter for these, so I had to catch these manually.
I found #757 to fix one of these, and I thought it would be good
to fix everything at once.
2022-11-13 11:51:51 -06:00
Dmitry VerkhoturovandUmputun d2ea572abf fix "it's" used in place of "its" 2022-10-20 14:18:21 -05:00
Dmitry VerkhoturovandUmputun dd1ba9b518 clean up comment HasReplies cache on child comment deletion
Previously, the cache kept the entry and deletion of the parent comment
after child deletion was not possible for the rest
of cache life (5m) duration. Now it's possible to delete
a parent comment after the deletion of the child comment
by a user or admin.

Resolves #1481
2022-10-03 03:28:29 -05:00
Dmitry VerkhoturovandUmputun 050f1b7941 migrate from mockery to moq 2022-10-02 21:21:28 -05:00
Dmitry VerkhoturovandUmputun 2ecc80e18c fix RPC engine work with ListFlags method
Fixes the following conversion problem for BlockedUser:

```
panic: interface conversion: interface {} is map[string]interface {},
not store.BlockedUser [recovered]
```

Resolves #1475.
2022-09-29 11:56:12 -05:00
Dmitry VerkhoturovandUmputun 0728b28856 bump golangci-lint, fix discovered problems
Also, improve the goveralls installation method.
2022-09-28 18:11:09 -05:00
Dmitry VerkhoturovandUmputun a6a9270f63 apply pngcrush, advpng and optipng to all png images
Commands used:

```
find . -type f -iname "*.png" -exec advpng -z4 {} \;
find . -type f -iname "*.png" -exec optipng -o7 -preserve {} \;
find . -type f -iname "*.png" -exec pngcrush -rem allb -brute -reduce -ow {} \;
```
2022-09-28 16:05:03 -05:00
Dmitry VerkhoturovandUmputun 372429a9f5 call admin store with a proper key
Previously it was set to fixed strings, likely a test artefact.

Resolves #1499.
2022-09-28 16:04:19 -05:00
Dmitry VerkhoturovandUmputun e77dc33333 bump mockery from v1.1.2 to v2.14.0
Command-line params changed their names,
and old ones won't work anymore.
2022-09-12 17:22:34 -05:00
Dmitry Verkhoturov 86d059bf99 move templates from rakyll/statik to go:embed
There is no need for the rakyll/statik package starting with Go 1.16,
which provides us with tools for embedding files
without third-party libraries.
2022-07-29 19:07:49 +02:00
Dmitry VerkhoturovandUmputun 4b4c749756 remove mod=vendor from go build directives as it's no longer actual
It's a default in the presence of the vendor folder since Go 1.14,
https://go.dev/ref/mod
2022-07-27 22:09:14 -05:00
Dmitry Verkhoturov 243c8356e7 do not sanitise the original comment markdown
Previously it was sanitised using the HTML sanitiser,
but it had proven troublesome and unnecessary.
Remark42 rendered the markdown into proper HTML, but then some pieces
of it (like cited HTML code inside the code block, marked by backticks)
were cut out, which then showed the incorrect markdown to a user when
they were editing the comment.

For example, the comment "`foo<bar>`" became "foo" after sanitising,
and despite the proper render user saw only "foo" when editing
the comment.

After this change, the initial comment markdown is preserved unaltered.
It could contain dangerous HTML with JS, which I assume shouldn't
be a problem as it's never rendered as HTML but instead supposed
to be converted to HTML by the interpreter. In Remark42, it's stored
in a comment.Text field and sanitised and thus safe.

I've left information about the potential danger of rendering
the original markdown as-is without an interpreter in
all relevant places I could find.
2022-07-22 01:08:08 +02:00
Dmitry VerkhoturovandUmputun 2d2f2ab02a return docker build for frontend and backend code outside in CI
Previously we built a Docker image just for the test,
but the introduction of multi-arch build in 9fbf0952
build also meant the push of the image, so it was
restricted only to the master branch.
This change re-introduces the Docker image build
outside the master branch, which is helpful
in pull requests.

We recently had a few frontend PRs which broke
the Docker image build silently, and that change
prevents it from happening.
2022-07-19 19:25:54 -05:00
Dmitry VerkhoturovandUmputun 9c1a827685 update alecthomas/chroma and Depado/bfchroma to v2 2022-07-13 20:29:46 -05:00
UmputunandGitHub 7cdb006f81 Links rune (#1344)
* shorten links with non-latin properly

* lint: unneeded conversion
2022-04-30 12:03:52 -05:00
Umputun 598d767791 resolves #415 and the similar issue with title 2022-04-29 10:52:05 -05:00
Dmitry VerkhoturovandUmputun 0c3053d4ad CloseIdleConnections on http clients
Without this, go.uber.org/goleak reports
leaking goroutine caused by HTTP client
on many tests when ran one by one.
2022-04-29 10:51:49 -05:00
Dmitry VerkhoturovandUmputun ba86db1263 replace errors package with fmt.Errorf
https://gist.github.com/Peltoche/60b8b81dfbf70164d0e2b88988003229
was used for it, thanks to @Peltoche for publishing it.
2022-04-26 00:25:09 -05:00
Dmitry VerkhoturovandUmputun 2a7966b9e6 replace numbers with proper HTTP status codes in tests
Also, remove unneeded whitespaces using whitespace
linter for golangci-lint.
2022-04-17 19:42:45 -05:00
Dmitry VerkhoturovandUmputun cfb7361adf optimise images for smaller size 2022-02-24 16:18:28 -06:00
Dmitry VerkhoturovandUmputun 8d42d0714f bump backend dependencies
Also, switch from fork github.com/umputun/go-flags back to original
github.com/jessevdk/go-flags.
2022-01-31 14:24:33 -06:00
Dmitry VerkhoturovandUmputun 90e537358d update golangci-lint to 1.43.0, fix found issues 2021-11-23 15:00:40 -06:00
Dmitry VerkhoturovandUmputun ca7cbedea4 regenerate mocks and use require.NoError in place of Nil 2021-10-25 16:47:15 -05:00
romanilchyshyn df0d4d27fa patreon auth 2021-10-13 00:30:14 +03:00
Dmitry VerkhoturovandUmputun 5abeab4008 stop Ticker after use to prevent the memory leak
https://github.com/golang/go/wiki/CodeReviewConcurrency#ticker-stop
2021-10-03 16:09:30 -06:00
Dmitry VerkhoturovandUmputun 3f0ede560c bump go modules in the project 2021-09-08 11:34:48 -05:00