Compare commits

..
299 Commits
Author SHA1 Message Date
Umputun 26e6e57949 add cmd/var to ignore 2023-08-07 13:33:04 -05:00
Umputun b572966bc4 remove cmd/var 2023-08-07 13:32:44 -05:00
Umputun bbfa4f1043 update base images for backend docker build 2023-08-07 13:12:42 -05:00
Umputun 9ad4f0b75e lint: remove deprecated rand.Seed from tests 2023-08-07 13:09:38 -05:00
Umputun 2093f4ece2 update go to 1.20, bump deps 2023-08-07 13:03:16 -05:00
Alexander GusmanandUmputun 7bc7703dc2 build: use pnpm instead of npm to install deps for size checks 2023-07-24 12:29:55 -05:00
Alexander GusmanandUmputun 0ed7452e77 feat(ui): telegram user subscriptions 2023-07-24 12:29:55 -05:00
Alexander GusmanandUmputun 366cc19c1b fix(rest): correct status code for telegram check token 2023-07-24 12:29:55 -05:00
Dmitry VerkhoturovandUmputun c72f30eabb remove proxied images from sanity check
Previously, proxied and local images were checked for presence in the
storage before previewing or posting the comment. That logic resulted in
 an inability to post with an image when a proxy for images is enabled,
 as proxied images are not downloaded to disk before the first time
 someone loads them, which could only happen after the user either
 previews or posts the message.

After this change, preview and post only checks the local images'
presence and ignore the proxied ones.
2023-07-23 12:10:52 -05:00
Justin HawkinsandGitHub 235f0dade0 Update documentation to support Caddy V2 (#1657)
* Update index.md

Change Caddy example config to the current V2 config.

* Simplify config

* Include legacy config as well
2023-07-20 21:12:11 -05:00
Dmitry VerkhoturovandUmputun 9c718cbc5f clarify SharedSecret usage by comment
It's not used aside from `server` but it was decided to keep it in place
for backwards compatibility in PR #1544.
2023-07-16 20:50:03 -05:00
Dmitry VerkhoturovandUmputun 02de92afc7 update Google auth setup instructions
Update based on my experience going through them.
2023-07-16 18:14:38 -05:00
Yury KotovandUmputun add01455fb Fix snippet generation
1) Current implementation simply removes the last word, without truncating up to limit length.
2) In case if even the first word (magnet link or some base64?) is too long don't add extra space.
2023-07-04 13:32:37 -05:00
EugeneandGitHub 497f3ce47f Skip confirmation step on email subscription (#1646)
* Email subscription params in request body

* Email subscription params in request body

fix tests

* Skip confirm step on email sub

When user logged in with the same email he tries to subscribe

* Skip confirm step on email sub

set autoConfirm param to make it work

* Update size-limit

* Handle 409: already subscribed

* refactor: prevStep to justSubscribed

prevStep is not used anywhere else and because of it influences output text (haveSubscribed), have changed it to more intuitive justSubscribed variable

* Test case for http error 409
2023-06-29 14:15:59 -05:00
LoneExileandUmputun 64188e5713 docs: added Astro w/ React/Preact Components Integration 2023-06-29 13:21:42 -05:00
Eugene OrlovandUmputun 33a6d6da97 Email subscription params in request body
fix tests
2023-06-29 10:52:36 -05:00
Eugene OrlovandUmputun 6410e3be85 Email subscription params in request body 2023-06-29 10:52:36 -05:00
Dmitry VerkhoturovandUmputun 136d7e8215 switch from telegram_bot_username to telegram_notifications
Bot username is returned as an answer to subscribe request,
so knowing it in advance is unnecessary.
2023-06-29 10:52:00 -05:00
Eugene OrlovandUmputun d3fdd7b0d8 Fix color var usage 2023-06-27 22:16:38 -05:00
Eugene OrlovandUmputun 329fcc204c Update size-limit 2023-06-27 22:16:38 -05:00
Eugene OrlovandUmputun ba2c7894a8 Telegram QR styling
- 1/1 aspect ratio
- white border around (for easier scan)
2023-06-27 22:16:38 -05:00
Eugene OrlovandUmputun 07667c8881 fix: Gap between buttons and markdown tip 2023-06-12 09:44:04 -05:00
Eugene OrlovandUmputun 68504a70a0 [site] fix: anchor offset
use different header size on mobiles and desktops
2023-06-06 15:22:02 -05:00
Eugene OrlovandUmputun 32073b3d66 [site] fix: anchor offset
make it use tailwind styles
2023-06-06 15:22:02 -05:00
Eugene OrlovandUmputun d1c1664a38 [site] fix: anchor offset
when navigating through an anchor
2023-06-06 15:22:02 -05:00
Vasilii BlazhnovandUmputun 8cbcff98ec Fixed wrong markdown 2023-05-08 14:35:30 -05:00
Dmitry VerkhoturovandUmputun 26f82ad95c don't allow relative links in comments
(url) is a text inserted by default and never an intended URL.

That additional validation will ensure that users won't post relative
links because they are rarely intended.
2023-04-09 23:30:21 -05:00
Dmitry VerkhoturovandUmputun 1b90604b2d update go modules, fix Apple auth redirect
Previously the redirect after successful authentication didn't work.
2023-04-02 03:47:35 -05:00
Dmitry VerkhoturovandUmputun a03c002df4 bump golangci-lint to latest 2023-04-02 01:21:33 -05:00
Dmitry VerkhoturovandUmputun 1ce9415d34 ping pnmp version to 7
Previously version 8 was installed once it became available, breaking
the pipelines and docker build.
2023-04-01 14:45:44 -05:00
SimonHaasandUmputun cc842901b3 updated deprecated link 2023-03-17 10:59:45 -05:00
Paul MineevandUmputun 23d7e4cdbb fix indentation for apps/remark42 2023-03-10 17:46:41 -06:00
Dejavu MoeandUmputun b48f8fca31 Fix spell errors 2023-03-10 17:21:04 -06:00
DejavuMoeandUmputun a4da93326e Improve Simplified Chinese translation
modified:frontend/apps/remark42/app/locales/zh.json
2023-03-10 17:20:45 -06:00
konstantin krivleniaandUmputun 8bd5c0d163 #1605 fix loading th locale 2023-03-10 17:20:25 -06:00
DejavuMoeandUmputun 972ab87247 Update translation of the Simplified Chinese 2023-03-08 17:22:06 -06:00
dmitry.konchalenkovandUmputun d1ea664b41 Update translations 2023-02-28 11:52:29 -06:00
Dmitry Verkhoturov a55fadd53a make docker build fail on backend test failure
Previously, the commands were combined incorrectly, and the failure
of backend tests was ignored.
2023-02-05 21:11:12 +01:00
Dmitry VerkhoturovandUmputun c70a66a1c5 bump go modules 2023-01-21 13:30:44 -06:00
Dmitry VerkhoturovandUmputun 8357846818 add test JWT token generation instructions 2023-01-15 12:54:44 -06:00
Matt JacksonandUmputun 31ea91afb8 docs: added Astro w/Svelte Components Integration 2023-01-14 19:12:38 -06:00
Dmitry Verkhoturov 6616541f65 improve frontend documentation
Variables were documented in the documentation but not in the code,
and max_last_comments needed to be documented.
2023-01-10 23:53:26 +01:00
Paul MineevandUmputun 01695822bb fix: calculate correct size when no_footer=true 2023-01-10 11:27:44 -06:00
Paul MineevandUmputun f0186d1aab fix: fix no footer param 2023-01-10 11:27:44 -06:00
Dmitry VerkhoturovandUmputun 41a3359085 add the ability to set the JWS aud per site_id
Without this option, the aud is ignored.
It works only with RPC admin storage.

The shared key returned for all requests with the default shared admin
storage, so enabling that option does not affect it.
2023-01-10 11:24:41 -06:00
Dmitry Verkhoturov d6cce8df2c cleanup of the frontend code
- replace undocumented `substr` with `substring`
- remove unused code
- inline a few variables
- simplify ifs when possible
- improve saveCollapsedComments documentation
- cleanup the unused imports
- remove unused variables and types
2023-01-09 22:16:35 +01:00
Dmitry VerkhoturovandUmputun 596861a594 don't remove the twitter-tweet class from blockquote
This is needed to format the Twitter blockquotes as tweets.
2023-01-09 03:20:54 -06:00
Dmitry VerkhoturovandUmputun 385ea800a4 don't verify subscription email once more for email users
Previous behaviour is preserved for query parameters way of requesting
the subscription. The new behaviour with the possibility to confirm
the email right away without a separate /email/confirm call is enabled
only with request params sent in the request body, which was not a thing
before 27fc339e, which was merged just now and is not part
of any tagged version yet.
2023-01-09 03:17:25 -06:00
Dmitry Verkhoturov 27fc339e36 use the request body for email subscription endpoints
Previously, the endpoints were using query parameters.
After this change, the body is tried to be parsed.
2023-01-08 23:26:06 +01:00
Dmitry VerkhoturovandUmputun 61e2173f25 add e2e tests to makefile
Also, add missing entries to .dockerignore.
2023-01-08 14:31:36 -06:00
Dmitry VerkhoturovandUmputun 13a3fc3d1b move remark42 frontend nvmrc to /frontend/ 2023-01-08 14:31:15 -06:00
Dmitry VerkhoturovandUmputun 6a1b515ea9 add anti-spam documentation
It describes how anti-spam works now and its future.
2023-01-08 14:30:55 -06:00
Dmitry VerkhoturovandUmputun 82f27e6b63 fix typos in frontend code 2023-01-08 12:30:18 -06:00
Paul MineevandUmputun 6ac75031ad fix and optimize apple icon 2023-01-07 18:16:21 -06:00
Dmitry VerkhoturovandUmputun 8b7f1331ee add Apple auth provider frontend support
With distinct logos for light and dark theme from
https://devimages-cdn.apple.com/design/resources/download/Logo-Sign-in-with-Apple.dmg
2023-01-07 18:16:21 -06:00
Dmitry VerkhoturovandUmputun 099aad8475 add apple bad key test, fix key location
Previously, default location was outside of container mount.
2023-01-04 03:54:38 -06:00
Dmitry VerkhoturovandUmputun c1b3fba344 add backend support for Apple auth provider
It's a bit different from other OAuth providers and requires a
different set of options and a private key file.
2023-01-03 23:47:42 -06:00
Dmitry VerkhoturovandUmputun d7e9be99f9 make Close() calls idempotent
Previously, few of them resulted in panics when called more than once.
2023-01-03 01:41:26 -06:00
Dmitry Verkhoturov 067a8bcb21 make the email token tooltip more informative
Previously it said just "Token", but now it will provide more explicit
instructions about copying and pasting the token received by email.

Resolves #1339
2023-01-03 10:52:17 +04:00
Umputun f5569a62f1 add local analytic support 2022-12-25 18:45:31 -06:00
Jakub FridrichandUmputun 1ab1ed8a82 Added cs lang 2022-12-16 12:00:57 -06:00
Pavel MineevandUmputun 8d95baa70f chore: add resize observer mock 2022-12-05 12:03:36 -06:00
Pavel MineevandUmputun 050bce163a refac: use resize observer instead mutation observer in order to resize iframe 2022-12-05 12:03:36 -06:00
Pavel MineevandUmputun 68b84683d0 fix: update iframe size on textarea size change 2022-12-03 12:08:00 -06:00
Pavel MineevandUmputun 6fe373d540 fix: update fe contributing documentation and get dev commands in order 2022-11-30 01:19:51 -06:00
Pavel MineevandUmputun ba19bcc729 fix: update iframe size on mount 2022-11-29 18:12:14 -06:00
Dmitry VerkhoturovandUmputun f1b65db2c7 prefer unused PreferServerCipherSuites param
This is not used since 1.17,
https://github.com/golang/go/commit/9d0819b27ca248f9949e7cf6bf7cb9fe7cf574e8
2022-11-13 12:02:40 -06:00
Dmitry VerkhoturovandUmputun f5f287ef06 bump actions/setup-go to v3 to resolve node12 deprecation note 2022-11-13 12:00:54 -06:00
Igor KlipachandUmputun f161e6033c Removed duplicated linter 'megacheck' 2022-11-13 11:54:26 -06:00
Dmitry VerkhoturovandUmputun c86bff8811 remove duplicate type definitions from function signatures
I haven't found a linter for these, so I had to catch these manually.
I found #757 to fix one of these, and I thought it would be good
to fix everything at once.
2022-11-13 11:51:51 -06:00
Dmitry VerkhoturovandUmputun 596b1045bd allow dots in site id during email user validation 2022-11-03 16:31:00 -05:00
Dmitry VerkhoturovandUmputun 907ca2b590 remove the only generic logger usage
github.com/go-pkgz/lgr should be used instead for consistency
with the rest of the code
2022-11-03 16:30:10 -05:00
Dmitry VerkhoturovandUmputun f1b5469b83 bump github actions versions
Old ones produce warnings due to deprecation of NodeJS 12
2022-11-03 15:20:30 -05:00
Denis HananeinandUmputun 984fbde540 Support no_footer option 2022-10-25 21:42:08 -05:00
Dmitry VerkhoturovandUmputun 2bdc05dd47 remove unused entrypoint.sh
It should be removed after changes in 141c75401.
2022-10-23 12:55:34 -05:00
Dmitry VerkhoturovandUmputun d2ea572abf fix "it's" used in place of "its" 2022-10-20 14:18:21 -05:00
Dmitry VerkhoturovandUmputun 8d5c4cd578 make docker build work on tag again 2022-10-20 14:16:38 -05:00
Dmitry VerkhoturovandUmputun dd1ba9b518 clean up comment HasReplies cache on child comment deletion
Previously, the cache kept the entry and deletion of the parent comment
after child deletion was not possible for the rest
of cache life (5m) duration. Now it's possible to delete
a parent comment after the deletion of the child comment
by a user or admin.

Resolves #1481
2022-10-03 03:28:29 -05:00
Dmitry VerkhoturovandUmputun cebe929118 bump go modules, enable LoginAuth option for email
That option is needed for outlook.com and Office 365, resolves #1472.
2022-10-03 03:26:34 -05:00
Dmitry VerkhoturovandUmputun 3eccf01f1f bump docker-compose.yml for the site to work with dev properly
Previously, an image built for the `build` service was then used
for `server`, and changes were invisible to the user
before the container rebuild.

After that change, the useless static `build` service is deleted,
Dockerfile is only used in the CI pipeline, and only the `server`
service is left in docker-compose for the user to test
and see documentation changes locally in real-time.

Resolves #1178
2022-10-02 23:52:19 -05:00
Dmitry VerkhoturovandUmputun 050f1b7941 migrate from mockery to moq 2022-10-02 21:21:28 -05:00
Pavel MineevandUmputun 53cc370727 admin: fix block period 2022-10-02 13:40:58 -05:00
Paul MineevandUmputun 363e05d580 fix iframe resize on auth dropdown opening 2022-10-01 18:34:33 -05:00
Dmitry VerkhoturovandUmputun 2ecc80e18c fix RPC engine work with ListFlags method
Fixes the following conversion problem for BlockedUser:

```
panic: interface conversion: interface {} is map[string]interface {},
not store.BlockedUser [recovered]
```

Resolves #1475.
2022-09-29 11:56:12 -05:00
Dmitry VerkhoturovandUmputun 0728b28856 bump golangci-lint, fix discovered problems
Also, improve the goveralls installation method.
2022-09-28 18:11:09 -05:00
Dmitry VerkhoturovandUmputun a6a9270f63 apply pngcrush, advpng and optipng to all png images
Commands used:

```
find . -type f -iname "*.png" -exec advpng -z4 {} \;
find . -type f -iname "*.png" -exec optipng -o7 -preserve {} \;
find . -type f -iname "*.png" -exec pngcrush -rem allb -brute -reduce -ow {} \;
```
2022-09-28 16:05:03 -05:00
Dmitry VerkhoturovandUmputun 372429a9f5 call admin store with a proper key
Previously it was set to fixed strings, likely a test artefact.

Resolves #1499.
2022-09-28 16:04:19 -05:00
Umputun 4733ad8eb1 Revert "fix iframe resize on auth dropdown opening"
This reverts commit 8e96aa26b4.
2022-09-21 15:57:41 -05:00
Paul MineevandUmputun 8e96aa26b4 fix iframe resize on auth dropdown opening 2022-09-21 10:56:34 -05:00
Dmitry VerkhoturovandUmputun c0e8520d31 fix makefile and dockerfile
Here are multiple changes to commands run by Makefile based on my
attempts to run it on an Oracle Linux machine with the latest Docker.
2022-09-17 13:05:56 -05:00
Paul MineevandUmputun 345f80d90d add contribution links in readme 2022-09-14 14:43:58 -05:00
Paul MineevandUmputun ffcef2fe99 fix selected item 2022-09-13 10:54:13 -05:00
Dmitry VerkhoturovandUmputun e77dc33333 bump mockery from v1.1.2 to v2.14.0
Command-line params changed their names,
and old ones won't work anymore.
2022-09-12 17:22:34 -05:00
kaikunzandUmputun 5fee19f6c7 Update supportedLocales.json
add th
2022-09-12 17:20:51 -05:00
kaikunzandUmputun 659c623240 th.json
th
2022-09-12 17:19:14 -05:00
Dmitry VerkhoturovandUmputun 695d0ea13a allow dashes in site ID when validating email auth request
As discovered in #1477, dashes are expected to work in the site ID
and do work everywhere but in email auth. That change makes
the behaviour consistent: site ID now allows dashes.
2022-09-12 17:16:22 -05:00
Pavel MineevandDmitry Verkhoturov 0f7ac514fb add e2e tests 2022-09-10 11:00:34 +02:00
Dmitry Verkhoturov 9ad3be2e97 bump go modules, make auth dev hostname customisable
After this commit, dev auth would start working with the `REMARK_URL`
hostname instead of the previously hardcoded 127.0.0.1.

Breaks development setup where `REMARK_URL` was set
to a non-standard value and dev auth was running on 127.0.0.1
and working, as, after that change, it would stop working.
2022-08-26 23:35:57 +02:00
Dmitry Verkhoturov 2c36fab8aa add compose files to dockerignore, remove defaults from frontend compose
Before that change, docker would create a new image
on docker-compose file changes.

Frontend docker-compose file change removes options set
to the same values as their default values.
2022-08-26 23:35:40 +02:00
Avinal KumarandUmputun 3b7a4b6e52 Fix directory name in CODEOWNERS file
Signed-off-by: Avinal Kumar <avinal.xlvii@gmail.com>
2022-08-25 01:10:14 -05:00
Paul MineevandUmputun 499302c48e fix broken icons 2022-08-24 11:37:30 -05:00
Dmitry VerkhoturovandUmputun 28fbe76547 remove duplicated docker-compose.yml from parameters doc
Also, clarify the wording on TIME_ZONE parameter a little.
2022-08-23 22:39:02 -05:00
Paul MineevandUmputun fabb31275d FE codeowners 2022-08-23 22:18:04 -05:00
Dmitry VerkhoturovandUmputun 96b75af027 proper grammar for ADMIN_PASSWD parameter notes 2022-08-23 21:22:32 -05:00
Pavel MineevandPaul Mineev dc048ef047 fix: anon cannot request data removal 2022-08-22 11:42:15 -07:00
Pavel MineevandUmputun d6b2960a4a fix: save choosen color scheme 2022-08-21 22:36:59 -05:00
Pavel MineevandUmputun b47b6c4f56 add api sdk 2022-08-20 12:58:40 -05:00
Umputun 75427df7de make translated lines in ru shorter as a workaround for ugly formatting 2022-08-19 14:24:54 -05:00
Umputun cb98885e1f add warn log on rejected email auth validation https://github.com/umputun/remark42/discussions/1139#discussioncomment-3409701 2022-08-16 17:10:09 -04:00
Paul Mineev 7e0445cc94 disable color-scheme on remark iframe 2022-08-10 10:20:31 -07:00
Dmitry VerkhoturovandUmputun a9836aaea0 move static web files from rakyll/statik to go:embed
There is no need for the rakyll/statik package starting with Go 1.16,
which provides us with tools for embedding files
without third-party libraries.
2022-08-03 15:41:02 -05:00
crazyandUmputun 27b28dba0d add lose commit Traditional Chinese locale file 2022-08-01 10:53:39 -05:00
Dmitry Verkhoturov 86d059bf99 move templates from rakyll/statik to go:embed
There is no need for the rakyll/statik package starting with Go 1.16,
which provides us with tools for embedding files
without third-party libraries.
2022-07-29 19:07:49 +02:00
Paul MineevandUmputun 76f5ce32ca add turborepo 2022-07-29 11:05:21 -05:00
Dmitry VerkhoturovandUmputun ff9eab998a bump buildgo to 1.9.2 to be consistent with other baseimage usages 2022-07-28 16:41:15 -05:00
Dmitry VerkhoturovandUmputun ac3a36eb13 rewrite email documentation, elaborate on how to replace templates
I've run Grammarly over that text to find and fix some flaws
and added more detailed instructions on replacing a built-in template
with your own.
2022-07-28 13:28:27 -05:00
Dmitry VerkhoturovandUmputun 4b4c749756 remove mod=vendor from go build directives as it's no longer actual
It's a default in the presence of the vendor folder since Go 1.14,
https://go.dev/ref/mod
2022-07-27 22:09:14 -05:00
Dmitry VerkhoturovandUmputun 4777f4059f hardcode dockerhub username and ghcr.io repo for site docker push
Currently, such a build most likely has access to secrets but
fails due to the wrong username logging with DockerHub when
rebase is done by anyone but @umputun.

I've missed that last case in 5e5b3e0 and ad5d555.
2022-07-27 22:08:10 -05:00
Dmitry Verkhoturov ad5d555ac8 hardcode dockerhub username and ghcr.io repo for site docker push
Currently, such a build most likely has access to secrets but
fails due to the wrong username logging with DockerHub when
rebase is done by anyone but @umputun.
2022-07-28 01:34:21 +02:00
Dmitry Verkhoturov fc7540fc9b do not run size limit job outside of PRs
Currently, this step emits the
"Error: No PR found. Only pull_request workflows are supported."
message when run on the master branch commits (after the merge),
so the change prevents it from being run there.
2022-07-28 01:34:05 +02:00
Dmitry Verkhoturov 63a2bdea48 get rid of getstarted.html mentions
umputun introduced that reference in 70649b271,
but I can't find any references to it or the file itself now.
2022-07-27 23:22:08 +02:00
Dmitry Verkhoturov 41d47fdb3e don't enable telegram user notifications when they are disabled
Before that change, enabling Telegram auth
also enabled Telegram user notifications.
2022-07-27 23:21:09 +02:00
Paul Mineev 86dae37c5d docs: improve frontend dev instructions 2022-07-27 10:39:10 -07:00
Dmitry Verkhoturov 2a97d9379e fix unsubscribe template filename 2022-07-27 19:02:34 +02:00
orangesobeautifulandUmputun ba0060316c add Traditional Chinese (zh-tw) translations 2022-07-27 11:43:45 -05:00
Paul MineevandUmputun 41f18a23e5 docs: link backend dev docs to readme.md 2022-07-25 16:39:11 -05:00
Paul MineevandUmputun bf26ad4acc link readme for frontend dev 2022-07-25 16:39:11 -05:00
Paul MineevandUmputun acc56ad42b run dev env for site in docker 2022-07-25 13:29:33 -05:00
Paul MineevandUmputun 922e779118 docs: reorg contributing section 2022-07-25 13:23:36 -05:00
Dmitry VerkhoturovandUmputun f104a6e1b6 move common options to cmd.go
Timeout, admin password and site id are set in many commands,
and we need to take care of synchronising the descriptions
and flags between them.

This change moves these standard options to cmd.go importing them
in the same manner CommonOpts imported by all commands already.
2022-07-25 12:42:13 -05:00
Dmitry VerkhoturovandUmputun 0aa6052eba update backup documentation
Few things here:
1. Merge automatic and manual backup to a single page
2. State that ADMIN_PASSWD must be enabled for backup or restore to work
3. Remove unneeded usage of --admin-passwd from commands
  inside the container
4. Make the main backups page (not clickable through the interface,
  available only in search) redirect to information about backups
  instead of displaying text
5. Add HTTPS port to canonical docker-compose.yml
6. Clarify build option in the canonical docker-compose.yaml
2022-07-25 12:09:42 -05:00
RobinandUmputun f25b34d5e2 docs(manuals): add ts version of react component
Some time ago Gatsby started to support Typescript natively. Also the frontend of remark42 is written in Typescript. So I thought it's a good idea to add a Typescript version of the component.
2022-07-25 12:02:08 -05:00
Dmitry VerkhoturovandUmputun 5e5b3e0830 hardcode dockerhub username and ghcr.io repo for docker build
Currently, such a build most likely has access to secrets but
fails due to the wrong username logging with DockerHub when
rebase is done by anyone but @umputun.
2022-07-24 16:43:38 -05:00
Dmitry Verkhoturov f1163139d7 add docker image labels, add docker image badge to readme
Specification for labels:
https://github.com/opencontainers/image-spec/blob/main/annotations.md
2022-07-24 23:06:42 +02:00
Paul MineevandUmputun 5db6e4364a fix simple mode 2022-07-21 22:24:32 -05:00
Paul MineevandUmputun 3d1a3fd7cf update ci workflows 2022-07-21 19:52:48 -05:00
Paul MineevandUmputun 6625face50 update dev docs 2022-07-21 18:27:33 -05:00
Paul MineevandUmputun d9764c251d workspaces 2022-07-21 18:27:33 -05:00
Dmitry Verkhoturov 243c8356e7 do not sanitise the original comment markdown
Previously it was sanitised using the HTML sanitiser,
but it had proven troublesome and unnecessary.
Remark42 rendered the markdown into proper HTML, but then some pieces
of it (like cited HTML code inside the code block, marked by backticks)
were cut out, which then showed the incorrect markdown to a user when
they were editing the comment.

For example, the comment "`foo<bar>`" became "foo" after sanitising,
and despite the proper render user saw only "foo" when editing
the comment.

After this change, the initial comment markdown is preserved unaltered.
It could contain dangerous HTML with JS, which I assume shouldn't
be a problem as it's never rendered as HTML but instead supposed
to be converted to HTML by the interpreter. In Remark42, it's stored
in a comment.Text field and sanitised and thus safe.

I've left information about the potential danger of rendering
the original markdown as-is without an interpreter in
all relevant places I could find.
2022-07-22 01:08:08 +02:00
Dmitry VerkhoturovandUmputun 2d2f2ab02a return docker build for frontend and backend code outside in CI
Previously we built a Docker image just for the test,
but the introduction of multi-arch build in 9fbf0952
build also meant the push of the image, so it was
restricted only to the master branch.
This change re-introduces the Docker image build
outside the master branch, which is helpful
in pull requests.

We recently had a few frontend PRs which broke
the Docker image build silently, and that change
prevents it from happening.
2022-07-19 19:25:54 -05:00
Dmitry VerkhoturovandUmputun 50785e0577 bump didip/tollbooth from v6 to v7 2022-07-19 17:13:02 -05:00
Dmitry VerkhoturovandUmputun 9c1a827685 update alecthomas/chroma and Depado/bfchroma to v2 2022-07-13 20:29:46 -05:00
Paul MineevandUmputun 8c658b7eda fix frontend test/build in docker 2022-07-13 14:35:54 -05:00
Paul MineevandUmputun 26d8d3daee use pnpm lock 2022-07-13 14:00:48 -05:00
Dmitry VerkhoturovandUmputun 26476db95d update go-pkgz/rest, stretchr/testify, three stdlib modules 2022-07-13 12:40:30 -05:00
Pavel Mineev 3c90f6ae61 increase size-limit 2022-07-12 13:34:44 -07:00
Pavel Mineev 4889afdf0c use size-limit/file 2022-07-12 13:34:44 -07:00
Paul MineevandPaul Mineev 2e777ea752 switch to pnpm 2022-07-11 20:13:39 -07:00
Paul MineevandPaul Mineev fddf1e21f3 prettier 2022-07-11 16:21:44 -07:00
Paul MineevandPaul Mineev d6013b10e6 separate persist logic from compnennt 2022-07-11 16:21:44 -07:00
Pavel MineevandUmputun aac6af40cc update height when an image is loaded 2022-07-11 12:54:48 -05:00
Dmitry VerkhoturovandUmputun 1f96a0e4d3 update go-pkgz/auth module to fix dev provider work
Fix for error introduced in the following commit:
https://github.com/go-pkgz/auth/commit/06e72788bcbb23d958c60655b42892b95457477e

After text/template was replaced with the html/template,
the dev provider started escaping parameters
which were not supposed to be escaped.
2022-07-10 10:49:41 -05:00
Paul MineevandUmputun 99716984ad up node version, add nvmrc, recommend nvm 2022-07-02 11:48:22 -05:00
Paul MineevandUmputun 24e9404a6f Fix email autofill in subscription popup 2022-07-01 12:33:15 -05:00
Paul MineevandUmputun 1f1adba5fd up eslint 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun b907354746 update size limit for remark chunk 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun 9a08d4a412 fix problems after update 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun 3a08e6dd55 bump prod deps 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun 02f782a5ab use ts for jest config, use transformIgnorePatterns, jsdom env by default 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun 30d68b2d1e update eslint config 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun 8974cde582 bump dev deps 2022-07-01 12:22:57 -05:00
Paul MineevandUmputun 5f3206a4e8 preact preset already includes rules for jest 2022-07-01 11:13:28 -05:00
Paul MineevandUmputun 40e38d225e fix broken deps 2022-07-01 11:13:28 -05:00
Dmitry VerkhoturovandUmputun a73072c8fb add documentation on running remark42 on a separate domain 2022-06-30 19:12:00 -05:00
Dmitry VerkhoturovandGitHub 6a5c5a4c08 add missing env_delim to ALLOWED_HOSTS parameter (#1395) 2022-06-29 22:32:48 -05:00
Dzung DoandGitHub fa7d5cee87 Translate some string into Vietnamese (#1381)
* Finish Vietnamese translation and editing some typo missing
* Update vi.json
2022-06-07 10:45:41 -05:00
Paul MineevandUmputun fe4db30e6d show subscription buttons in simple view, add ability to hide rss button 2022-06-06 10:00:24 -05:00
Paul MineevandUmputun 6936268fd2 fix basepath for oauth icons 2022-06-05 17:46:31 -05:00
Umputun e55f6ffdf3 go mod tidy for examples 2022-06-05 12:53:34 -05:00
Umputun e182e3c776 switch to master version of auth
for https://github.com/go-pkgz/auth/pull/119
2022-06-05 12:49:30 -05:00
UmputunandGitHub 6309443d1f removes .git from build layer, emulates GH build (#1375)
* removes .git from build layer, emulates GH build #269

* rundev target with git version passed in
2022-06-05 12:36:18 -05:00
Umputun 6f81bf00e8 add validation for email and site 2022-06-05 11:57:51 -05:00
Umputun 86a1f5ee5d exact match on email login path 2022-06-05 11:57:51 -05:00
Umputun b3e460eebd sleep in anon test to prevent limiter 2022-06-05 11:57:51 -05:00
Umputun 5121c48c31 reduce max limiter for /auth to 2r/s 2022-06-05 11:57:51 -05:00
Umputun 5f8e16cbe2 add email auth validation with middleware 2022-06-05 11:57:51 -05:00
Ruslan NagimovandUmputun 12e4f283fe typo 2022-06-02 10:47:06 -05:00
Dzung DoandUmputun 20ca0896a6 Finish Vietnamese translation
and editing some typo missing
2022-05-30 13:39:45 -05:00
Dmitry VerkhoturovandUmputun 3b5f44da46 bump go modules, fix StartTLS email notifications
In #1359, we discovered that StartTLS was not working\
due to the wrong host passed. This bumps the library for the fix.

Also, after a switch to go-pkgz/notify MailGun email sending
broke due to the difference in the destination email parsing,
the fix is also applied after this commit.
2022-05-20 16:00:30 -05:00
Dmitry VerkhoturovandUmputun 9049d7a616 add missing RPC type of image storage to the documentation 2022-05-15 15:56:53 -05:00
Dmitry VerkhoturovandUmputun adb77d9a11 sync server parameters with their description 2022-05-12 11:41:52 -05:00
Paul MineevandUmputun 18cc34535f fix lint and test 2022-05-10 14:47:34 -05:00
Paul MineevandUmputun 9053668cfe fix editing mode 2022-05-10 14:06:02 -05:00
Alena MaslovaandUmputun b82ed825f7 add comment about custom ID user generation 2022-05-10 12:59:49 -05:00
Alena MaslovaandUmputun ac492180e8 use custom UserIDFunc for anonymous provider 2022-05-10 12:59:49 -05:00
dmitry.konchalenkovandUmputun 168a6d36c0 Update translations 2022-05-09 02:52:06 -05:00
UmputunandGitHub 7cdb006f81 Links rune (#1344)
* shorten links with non-latin properly

* lint: unneeded conversion
2022-04-30 12:03:52 -05:00
dependabot[bot]andUmputun 5a6ca7a82b Bump ejs from 3.1.6 to 3.1.7 in /site
Bumps [ejs](https://github.com/mde/ejs) from 3.1.6 to 3.1.7.
- [Release notes](https://github.com/mde/ejs/releases)
- [Changelog](https://github.com/mde/ejs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mde/ejs/compare/v3.1.6...v3.1.7)

---
updated-dependencies:
- dependency-name: ejs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-29 13:34:05 -05:00
Dmitry VerkhoturovandUmputun a980bdbad2 switch to go-pkgz/notify package: email 2022-04-29 13:32:15 -05:00
Dmitry VerkhoturovandUmputun 0d9c80aec7 switch to go-pkgz/notify package: slack 2022-04-29 13:32:15 -05:00
Dmitry VerkhoturovandUmputun 59fb68ab2d switch to go-pkgz/notify package: telegram 2022-04-29 13:32:15 -05:00
Dmitry VerkhoturovandUmputun ad0ac693de switch to go-pkgz/notify package: webhook 2022-04-29 13:32:15 -05:00
Dmitry VerkhoturovandUmputun 0560a893cf switch to go-pkgz/notify package: modules 2022-04-29 13:32:15 -05:00
Umputun 598d767791 resolves #415 and the similar issue with title 2022-04-29 10:52:05 -05:00
Dmitry VerkhoturovandUmputun 0c3053d4ad CloseIdleConnections on http clients
Without this, go.uber.org/goleak reports
leaking goroutine caused by HTTP client
on many tests when ran one by one.
2022-04-29 10:51:49 -05:00
Dmitry VerkhoturovandUmputun 91b9324080 cleanup test files 2022-04-29 10:51:49 -05:00
Dmitry VerkhoturovandUmputun ba86db1263 replace errors package with fmt.Errorf
https://gist.github.com/Peltoche/60b8b81dfbf70164d0e2b88988003229
was used for it, thanks to @Peltoche for publishing it.
2022-04-26 00:25:09 -05:00
Paul MineevandUmputun 21d0339d3d restore low_score comment state 2022-04-24 20:19:13 -05:00
Paul MineevandUmputun 9e30e3bd2c enable downvoting for only positive 2022-04-24 20:19:13 -05:00
Paul MineevandUmputun bf6a08bdcd Move back last comments css file 2022-04-21 17:43:35 -05:00
Umputun 3f3bfaffc6 Merge remote-tracking branch 'origin/master' 2022-04-18 00:03:59 -05:00
Umputun 9efcc9aba3 add removal info to policy 2022-04-18 00:03:53 -05:00
Dmitry VerkhoturovandUmputun 2a7966b9e6 replace numbers with proper HTTP status codes in tests
Also, remove unneeded whitespaces using whitespace
linter for golangci-lint.
2022-04-17 19:42:45 -05:00
Umputun 9ab27da343 update deps on example 2022-04-15 20:29:14 -05:00
Umputun f4856c86d7 update deps 2022-04-15 12:50:05 -05:00
dependabot[bot]andUmputun 53df70bcef Bump async from 2.6.3 to 2.6.4 in /frontend
Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4.
- [Release notes](https://github.com/caolan/async/releases)
- [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md)
- [Commits](https://github.com/caolan/async/compare/v2.6.3...v2.6.4)

---
updated-dependencies:
- dependency-name: async
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-15 12:40:55 -05:00
Paul MineevandUmputun 2d3677d629 clean testid in production build 2022-04-15 12:39:26 -05:00
Paul MineevandUmputun 58b0841360 update comment actions block 2022-04-15 10:32:01 -05:00
Paul MineevandUmputun cd7a2a3c73 fix light bg on dark theme 2022-04-13 22:06:12 -05:00
Paul MineevandUmputun c58e6471ff recover translation for controversion string 2022-04-13 12:54:55 -05:00
Paul MineevandUmputun a840dd9081 increce size limit for last-comments widget 2022-04-13 12:51:38 -05:00
Paul MineevandUmputun 0e4ae6e050 rewrite vote component 2022-04-13 12:51:38 -05:00
Paul MineevandUmputun a290d906ee fix copy for latest safari 2022-04-13 12:15:31 -05:00
Paul MineevandUmputun 000d64d78f fix broken verify icon on admin view and optimize the icon 2022-04-13 03:40:39 -05:00
Paul MineevandUmputun c987513886 add icon for paid patreon sub 2022-04-13 03:40:39 -05:00
Paul MineevandUmputun 001b994e73 replace ts-jest and babel-jest to swc/jest 2022-04-13 03:40:19 -05:00
Paul MineevandUmputun 5ed9e9d41b update size-limit and move to app preset metrics 2022-04-12 12:18:56 -05:00
Paul MineevandUmputun 5de9544337 update max size limit for counter.js 2022-04-10 15:35:31 -05:00
Paul MineevandUmputun a320ae03ae Prevent code execution from the query 2022-04-10 15:35:31 -05:00
Paul MineevandUmputun fe6b119254 fix xss from iframe name 2022-04-10 15:35:09 -05:00
Paul MineevandUmputun 145b4c474c use default system font 2022-04-10 15:34:54 -05:00
Paul MineevandUmputun f7bddd757d Improve frontend docs 2022-04-10 11:18:15 -05:00
Paul MineevandUmputun d89c9a1d1c make content width on docs pages wider 2022-04-09 19:22:25 -05:00
Dmitry VerkhoturovandUmputun 0eac76ce6c improve notifications description in parameters doc 2022-04-07 18:27:39 -05:00
Dmitry VerkhoturovandUmputun 977e5f018b renew Google auth documentation 2022-04-07 17:14:56 -05:00
Dmitry VerkhoturovandUmputun 38a680ec4c use remark42.mysite.com consistently in documentation 2022-04-07 17:14:56 -05:00
Dmitry VerkhoturovandUmputun 7af30852db add variables into auth instructions 2022-04-07 17:14:56 -05:00
Dmitry VerkhoturovandUmputun 0047fa057f add missing main component to frontend config examples 2022-04-06 18:40:27 -05:00
Dmitry VerkhoturovandUmputun 7d4ba01409 improve getting started installation instructions 2022-04-06 17:33:34 -05:00
Umputun 65e3a82d8b add no-signature mode suppressing app info middleware #1305 2022-04-05 12:08:44 -05:00
UmputunandGitHub 89dc8ac6dd Go 1.17 (#1306)
* change go mod to 1.17

* update go-pgkz and transitive deps

* bump examples to go-1.17

* bump deps
2022-04-05 11:50:45 -05:00
konstantin krivleniaandUmputun 5ed7f27d5e update stylelint 2022-04-03 12:03:01 -05:00
dependabot[bot]andUmputun 4e0c41cf86 Bump minimist from 1.2.5 to 1.2.6 in /site
Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6.
- [Release notes](https://github.com/substack/minimist/releases)
- [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6)

---
updated-dependencies:
- dependency-name: minimist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-01 13:18:30 -05:00
itsmejoeeeyandUmputun 86b0e8d57c Remove emoji from warning in Twitter section in auth docs 2022-03-25 23:09:39 -05:00
itsmejoeeeyandUmputun d3fe35bdf2 Add warning to Twitter section in docs Authorization page 2022-03-25 23:09:39 -05:00
Dmitry VerkhoturovandUmputun 39c141c98d increase timeout for backup and restore and remap 15m->60m
Resolves #1297
2022-03-25 15:55:37 -05:00
dependabot[bot]andPaul Mineev d46c5c0c24 Bump nanoid from 3.1.23 to 3.2.0 in /frontend
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.23 to 3.2.0.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.23...3.2.0)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-25 21:29:03 -06:00
dependabot[bot]andUmputun 0ba8ecdf82 Bump prismjs from 1.26.0 to 1.27.0 in /site
Bumps [prismjs](https://github.com/PrismJS/prism) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/PrismJS/prism/releases)
- [Changelog](https://github.com/PrismJS/prism/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PrismJS/prism/compare/v1.26.0...v1.27.0)

---
updated-dependencies:
- dependency-name: prismjs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-25 20:16:27 -06:00
mstfelgandUmputun 08903ac6f6 add arabic locale 2022-02-25 14:20:14 -06:00
Dmitry VerkhoturovandUmputun 932c0e8af0 optimise images for smaller size 2022-02-24 16:54:03 -06:00
Dmitry VerkhoturovandUmputun d47133c004 add admin UI documentation, fix title for no-subdomain doc 2022-02-24 16:54:03 -06:00
Dmitry VerkhoturovandUmputun cfb7361adf optimise images for smaller size 2022-02-24 16:18:28 -06:00
Dmitry Verkhoturov d4481d145f fix remark_config reference in the installation documentation 2022-02-23 09:12:21 +01:00
Dmitry VerkhoturovandUmputun 2ef1cfe1ea fix logic of detecting deprecated notify type params 2022-02-19 02:43:34 -06:00
Pavel MineevandUmputun 9216d4189e add telegram translation string 2022-02-18 15:59:29 -06:00
Paul MineevandUmputun 286e4c8a43 update telegram qr size 2022-02-15 18:37:24 -06:00
Dmitry VerkhoturovandUmputun f03fcf3fcf make QR for telegram borderless 2022-02-15 18:33:51 -06:00
Dmitry VerkhoturovandUmputun f448208475 allow skipping frontend build in Docker
That option allows having backend-only build,
skipping the long frontend build and test step.
Frontend developers run NodeJS locally and usually
don't need to have frontend built inside the docker image.
2022-02-15 18:30:38 -06:00
dependabot[bot]andUmputun 214e183b07 Bump follow-redirects from 1.14.7 to 1.14.8 in /site
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.7 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.7...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-15 00:19:36 -06:00
dependabot[bot]andPaul Mineev 338059e507 Bump follow-redirects from 1.14.7 to 1.14.8 in /frontend
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.7 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.7...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-15 00:18:45 -06:00
konstantin krivleniaandUmputun 188e3974bc add support node 17 2022-02-13 23:33:41 -06:00
Dmitry VerkhoturovandUmputun b5f34b8777 remove notice about user notifications in telegram 2022-02-12 03:52:16 -06:00
Umputun 9d8985e616 change update to silent curl 2022-02-11 12:00:55 -06:00
Paul MineevandUmputun 9bae155e79 Fixes for telegram
- fix type for an error in fetcher
- use is-object func for checks
- better error handling for auth requests
2022-02-11 10:59:19 -06:00
Umputun 7b47bd2f05 return 200 and empty list on user comments if nothing #1265 2022-02-10 15:31:09 -06:00
UmputunandGitHub 74b47d3a8a Merge pull request #1107 from Ksinia/master
Add telegram auth to frontend
2022-02-09 11:05:44 -06:00
Dmitry VerkhoturovandUmputun 6fd730655a bump Go modules 2022-02-09 11:04:13 -06:00
Pavel Mineev c9b1fb4737 simplify click outside 2022-02-07 22:47:00 -06:00
Pavel Mineev 38414c4628 move error hook to hooks file 2022-02-07 22:36:36 -06:00
Pavel Mineev b97b1f9462 telegram adjustments and tests
- get rid of redux store in favor of local state
- add tests for telegram happy path
- lift auth handling on Auth level
2022-02-07 22:28:19 -06:00
dependabot[bot]andPavel Mineev 7395198843 Bump nanoid from 3.1.23 to 3.2.0 in /frontend
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.23 to 3.2.0.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.23...3.2.0)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 22:28:07 -06:00
dependabot[bot]andUmputun 988391bc9d Bump nanoid from 3.1.23 to 3.2.0 in /frontend
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.23 to 3.2.0.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.23...3.2.0)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 19:52:35 -06:00
Ksinia 5c91b8764e Switch QR to backend rendering 2022-02-08 00:28:34 +01:00
Ksinia 7c6b44a5bb Get rid of isTelegramShown state 2022-02-07 23:33:32 +01:00
Ksinia 0f4ac6348c Fix function name 2022-02-07 23:33:32 +01:00
Pavel MineevandKsinia 0ea91bb99f update tg icon 2022-02-07 23:33:32 +01:00
Pavel MineevandKsinia 3e12ca4074 fix tests 2022-02-07 23:33:32 +01:00
Ksinia 69b110e36b Add telegram auth to frontend 2022-02-07 23:33:32 +01:00
Dmitry VerkhoturovandUmputun 9dbc36e426 remove explicit /srv/ usage from docker-compose files
We have plenty of paths used in the application, but two of them
are hardcoded in examples all over the code for historical reasons.

I found that by default in Docker, the path would resolve to the value
we are setting it explicitly to, so it doesn't make sense to set
a few variables we are setting now explicitly.
2022-02-07 03:54:54 -06:00
JoeyZhouAusandUmputun e0980d4700 Fix typo in index.md
The text in the backend development section should be backend instead of frontend
2022-02-06 21:30:01 -06:00
Dmitry VerkhoturovandUmputun f58073fa57 add proper default values for two boolean variables 2022-02-06 16:37:47 -06:00
Dmitry VerkhoturovandUmputun 6fe83fb6b8 bump auth module
Follow-up for https://github.com/go-pkgz/auth/pull/107
2022-02-06 14:22:27 -06:00
Dmitry VerkhoturovandUmputun 49d2552a78 fix frontend documentation typo 2022-02-06 13:55:52 -06:00
RoganikandUmputun 78be64900b Add tips to url-migration docs 2022-02-06 12:42:32 -06:00
Dmitry VerkhoturovandUmputun 603decabf6 add QR API endpoint for Telegram auth and notifications
Telegram authentication requires you to open a chat on the phone.
It's convenient to have a QR code for the case when you want to
log in on the computer but have Telegram only on your phone
and would be able to scan the QR instead of copy-pasting the link
from the computer to the phone any other way.

Originally we thought of generating QR on the client but found
backend-generated QR a better alternative because we avoid adding
one more JavaScript dependency to the frontend that way.
2022-01-31 14:28:18 -06:00
Dmitry VerkhoturovandUmputun 8d42d0714f bump backend dependencies
Also, switch from fork github.com/umputun/go-flags back to original
github.com/jessevdk/go-flags.
2022-01-31 14:24:33 -06:00
Dmitry VerkhoturovandUmputun 3a9806a368 improve doc on backend run without docker 2022-01-30 19:05:57 -06:00
Dmitry Verkhoturov 31af19e456 make FindDeprecatedFlagsCollisions private method 2022-01-31 00:40:03 +03:00
Dmitry Verkhoturov 8689b11e7c log when deprecated and new args are set at the same time
For example, when notify.telegram.token and telegram.token
are both set but to different values, user might see
"access denied" error in log on attempt to send telegram
notification, thinking that notify.telegram.token value
is used, when in fact it is ignored and only telegram.token
is used.

New behavior is the same, ignoring the old param when new
one is set, but issuing the error log message which
explicitly tells the user about that.

Resolves #1218.
2022-01-31 00:40:03 +03:00
Dmitry VerkhoturovandUmputun ffa0d34ed1 improve backend and frontend development docs
Remove generic development documentation, make frontend
and backend pages more specific and self-sufficient,
as previously you had to read development and frontend
pages in order to understand how to properly develop
frontend.
2022-01-30 12:25:07 -06:00
Umputun dc9411e2b9 add work-in-progress note about tg auth docs 2022-01-30 04:09:45 -06:00
Dmitry VerkhoturovandUmputun bd42c75a84 improve telegram notifications documentation 2022-01-24 15:06:50 -06:00
Umputun 6af938051a drop drone legacy magic from artifact build 2022-01-23 16:25:24 -06:00
Umputun 33b96a4263 the same buildplatform only for node deps step 2022-01-23 16:04:14 -06:00
Umputun 2d033de224 build node things only once, on builplatform only 2022-01-23 16:02:46 -06:00
Umputun c5d56a89cd pass GITHUB_REF into docker build 2022-01-23 15:17:13 -06:00
Umputun e8bd5dfc6e add missing args 2022-01-23 15:12:19 -06:00
Umputun 077b2d297c switch to versions.sh instead of legacy git-rev.sh 2022-01-23 14:34:45 -06:00
Umputun 45c633776f skip legacy drone check 2022-01-23 14:06:26 -06:00
Dmitry VerkhoturovandUmputun ddba4059b0 add "invite your tg bot" note to documentation
Pointed out in #1221 originally.
2022-01-23 13:20:14 -06:00
Ivan UhalinandUmputun 434f0f5f3d update docs link 2022-01-20 02:23:53 -06:00
Paul MineevandUmputun aabee8312d update eleventy config 2022-01-18 13:09:30 -06:00
Paul MineevandUmputun 5d1f021667 update site lockfile 2022-01-18 12:59:17 -06:00
Umputun 8ef308e6a2 Revert "update site lockfile"
This reverts commit a5d1be80d7.
2022-01-18 12:59:03 -06:00
Paul MineevandUmputun a5d1be80d7 update site lockfile 2022-01-18 12:41:23 -06:00
Pavel MineevandUmputun dd06500573 bump frontend deps 2022-01-18 01:38:30 -06:00
Pavel MineevandUmputun 249dd5d2b2 move gatsby integration to manuals 2022-01-18 00:10:13 -06:00
Pavel MineevandUmputun ac610db7e3 bump site deps 2022-01-18 00:02:27 -06:00
a748e26166 Integrating Remark42 with Gatsby (documentation) (#1222)
* add sample Gatsby/React component with comments md

* fix typo

* remove semicolons

* add link to gatsby doc in nav.json

* fix typo

* make comment actually a comment in the return

* improve comment syntax

Co-authored-by: Ben <BenRoe@users.noreply.github.com>
2022-01-15 12:04:40 -06:00
2594 changed files with 202259 additions and 93725 deletions
+18 -3
View File
@@ -1,15 +1,30 @@
/logs/
/target/
/var/
/frontend/node_modules/
/frontend/public/
/.vscode/
/.idea/
/bin/
/.git/
# frontend files not needed in docker image
/frontend/node_modules/
/frontend/apps/remark42/node_modules/
/frontend/apps/remark42/public/
# e2e tests arficats
/frontend/e2e/playwright-report/
/frontend/e2e/playwright/.cache/
/frontend/e2e/test-results/
# source files
docker-compose.yml
compose-dev-backend.yml
compose-dev-frontend.yml
compose-private-backend.yml
compose-private-frontend.yml
compose-e2e-test.yml
compose-private.yml
rest-client.env.json
Makefile
# generated files
*.cov
@@ -21,4 +36,4 @@ debug.test
*.test
remark42
/backend/var/
compose-private-backend.yml
/playwright-report/
+3 -1
View File
@@ -1,3 +1,5 @@
root = true
[*]
indent_style = tab
insert_final_newline = true
@@ -6,6 +8,6 @@ insert_final_newline = true
indent_style = space
trim_trailing_whitespace = false
[*.yml]
[*.{yml,json}]
indent_size = 2
indent_style = space
+2 -1
View File
@@ -2,4 +2,5 @@
# Unless a later match takes precedence, @umputun will be requested for
# review when someone opens a pull request.
* @umputun
* @umputun
frontend/* @akellbl4 @Mavrin
@@ -1,4 +1,4 @@
name: test_backend
name: backend
on:
push:
@@ -10,17 +10,19 @@ on:
- "!backend/scripts/**"
- "!**.md"
pull_request:
types: [opened, reopened]
paths:
- ".github/workflows/ci-test-backend.yml"
- "backend/**"
- "!backend/scripts/**"
- "!**.md"
jobs:
test-backend:
test:
name: Test & Coverage
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: debug if needed
run: if [[ "$DEBUG" == "true" ]]; then env; fi
@@ -28,37 +30,44 @@ jobs:
DEBUG: ${{secrets.DEBUG}}
- name: install go
uses: actions/setup-go@v2
uses: actions/setup-go@v3
with:
go-version: 1.16
go-version: "1.20"
- name: install golangci-lint and goveralls
run: |
curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh| sh -s -- -b $GITHUB_WORKSPACE v1.43.0
go get -u github.com/mattn/goveralls
- name: test and lint backend
- name: test and build backend
run: |
go test -race -timeout=60s -covermode=atomic -coverprofile=$GITHUB_WORKSPACE/profile.cov_tmp ./...
cat $GITHUB_WORKSPACE/profile.cov_tmp | grep -v "_mock.go" > $GITHUB_WORKSPACE/profile.cov
$GITHUB_WORKSPACE/golangci-lint --config ${GITHUB_WORKSPACE}/backend/.golangci.yml run --out-format=github-actions ./...
go build -race ./...
working-directory: backend/app
env:
GOFLAGS: "-mod=vendor"
TZ: "America/Chicago"
- name: test and lint examples
- name: test examples
run: |
go version
$GITHUB_WORKSPACE/golangci-lint version
go test -race ./...
$GITHUB_WORKSPACE/golangci-lint --config ${GITHUB_WORKSPACE}/backend/.golangci.yml run --out-format=github-actions ./...
go build -race ./...
working-directory: backend/_example/memory_store
env:
TZ: "America/Chicago"
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: latest
working-directory: backend/app
- name: golangci-lint on example directory
uses: golangci/golangci-lint-action@v3
with:
version: latest
args: --config ../../.golangci.yml
working-directory: backend/_example/memory_store
- name: submit coverage
run: $(go env GOPATH)/bin/goveralls -service="github" -coverprofile=$GITHUB_WORKSPACE/profile.cov
run: |
go install github.com/mattn/goveralls@latest
goveralls -service="github" -coverprofile=$GITHUB_WORKSPACE/profile.cov
working-directory: backend
env:
COVERALLS_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+29 -13
View File
@@ -7,16 +7,17 @@ on:
paths:
- ".github/workflows/ci-build.yml"
- "backend/**"
- "frontend/**"
- "frontend/apps/**"
- ".dockerignore"
- "docker-init.sh"
- "Dockerfile"
- "!**.md"
- "!frontend/packages/**"
pull_request:
paths:
- ".github/workflows/ci-build.yml"
- "backend/**"
- "frontend/**"
- "frontend/apps/**"
- ".dockerignore"
- "docker-init.sh"
- "Dockerfile"
@@ -24,21 +25,36 @@ on:
jobs:
build-images:
name: Build Docker images
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
- name: set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
- name: available platforms
run: echo ${{ steps.buildx.outputs.platforms }}
- name: build docker image without pushing (only outside master)
if: ${{ github.ref != 'refs/heads/master' }}
run: |
docker buildx build \
--build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true \
--platform linux/amd64 .
- name: build example docker image without pushing (only outside master)
if: ${{ github.ref != 'refs/heads/master' }}
run: |
docker buildx build \
--build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true \
--platform linux/amd64 -f backend/_example/memory_store/Dockerfile .
- name: build and deploy master image to ghcr.io and dockerhub
if: ${{ github.ref == 'refs/heads/master' }}
env:
@@ -49,14 +65,14 @@ jobs:
GITHUB_REF: ${{ github.ref}}
run: |
ref="$(echo ${GITHUB_REF} | cut -d'/' -f3)"
echo GITHUB_REF - $ref
echo "GITHUB_REF=${GITHUB_REF}, GITHUB_SHA=${GITHUB_SHA}, GIT_BRANCH=${ref}"
echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
echo ${DOCKER_HUB_TOKEN} | docker login -u ${USERNAME} --password-stdin
echo ${DOCKER_HUB_TOKEN} | docker login -u umputun --password-stdin
docker buildx build --push \
--build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true --build-arg CI=github \
--build-arg GITHUB_SHA=${GITHUB_SHA} --build-arg GIT_BRANCH=${ref} \
--build-arg GITHUB_SHA=${GITHUB_SHA} --build-arg GIT_BRANCH=${ref} --build-arg GITHUB_REF=${GITHUB_REF} \
--platform linux/amd64,linux/arm/v7,linux/arm64 \
-t ghcr.io/${USERNAME}/remark42:${ref} -t ${USERNAME}/remark42:${ref} .
-t ghcr.io/umputun/remark42:${ref} -t umputun/remark42:${ref} .
- name: deploy tagged (latest) to ghcr.io and dockerhub
if: ${{ startsWith(github.ref, 'refs/tags/') }}
@@ -68,12 +84,12 @@ jobs:
GITHUB_REF: ${{ github.ref}}
run: |
ref="$(echo ${GITHUB_REF} | cut -d'/' -f3)"
echo "GITHUB_REF=$ref, GITHUB_SHA=${GITHUB_SHA}"
echo "GITHUB_REF=${GITHUB_REF}, GITHUB_SHA=${GITHUB_SHA}, GIT_BRANCH=${ref}"
echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
echo ${DOCKER_HUB_TOKEN} | docker login -u ${USERNAME} --password-stdin
echo ${DOCKER_HUB_TOKEN} | docker login -u umputun --password-stdin
docker buildx build --push \
--build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true --build-arg CI=github \
--build-arg GITHUB_SHA=${GITHUB_SHA} --build-arg GIT_BRANCH=${ref} \
--build-arg GITHUB_SHA=${GITHUB_SHA} --build-arg GIT_BRANCH=${ref} --build-arg GITHUB_REF=${GITHUB_REF} \
--platform linux/amd64,linux/arm/v7,linux/arm64 \
-t ghcr.io/umputun/remark42:${ref} -t ghcr.io/umputun/remark42:latest \
-t umputun/remark42:${ref} -t umputun/remark42:latest .
@@ -82,4 +98,4 @@ jobs:
if: ${{ github.ref == 'refs/heads/master' }}
env:
UPDATER_KEY: ${{ secrets.UPDATER_KEY }}
run: curl https://jess.umputun.com/update/remark42-core/${UPDATER_KEY}
run: curl -s https://jess.umputun.com/update/remark42-core/${UPDATER_KEY}
+153
View File
@@ -0,0 +1,153 @@
name: "@remark42/api"
on:
push:
branches:
- master
paths:
- ".github/workflows/ci-frontend-api.yml"
- "frontend/packages/**"
- "!**.md"
pull_request:
paths:
- ".github/workflows/ci-frontend-api.yml"
- "frontend/packages/**"
- "!**.md"
jobs:
type-check:
name: Type check
runs-on: ubuntu-latest
strategy:
matrix:
node: [16.15.1]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
version: 7
run_install: false
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
- name: Run type check
run: pnpm type-check:api
working-directory: ./frontend
lint:
name: Lint
runs-on: ubuntu-latest
strategy:
matrix:
node: [16.15.1]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
version: 7
run_install: false
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
- name: Run linters
run: pnpm lint:api
working-directory: ./frontend/
test:
name: Tests & Coverage
runs-on: ubuntu-latest
strategy:
matrix:
node: [16.15.1]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
version: 7
run_install: false
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
- name: Test & Coverage
run: pnpm coverage:api
working-directory: ./frontend
- name: Submit coverage
run: ${{ github.workspace }}/frontend/apps/remark42/node_modules/.bin/codecov
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
@@ -1,20 +0,0 @@
name: frontend
on:
pull_request:
paths:
- ".github/workflows/ci-frontend-size-limit.yml"
- "frontend/**"
- "!**.md"
jobs:
size:
runs-on: ubuntu-latest
env:
CI_JOB_NUMBER: 1
steps:
- uses: actions/checkout@v1
- uses: andresz1/size-limit-action@v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
directory: frontend
+157 -43
View File
@@ -3,105 +3,219 @@ name: frontend
on:
push:
branches:
tags:
- master
paths:
- ".github/workflows/ci-frontend.yml"
- "frontend/**"
- "frontend/apps/remark42/**"
- "!**.md"
pull_request:
paths:
- ".github/workflows/ci-frontend.yml"
- "frontend/**"
- "frontend/apps/remark42/**"
- "!**.md"
jobs:
check-translations:
translations-check:
name: Translations check
runs-on: ubuntu-latest
strategy:
matrix:
node: [14.15]
node: [16.15.1]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- run: npm ci --loglevel warn
working-directory: ./frontend
- uses: actions/cache@v2
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
path: ${{ github.workspace }}/frontend/node_modules/.cache
key: ${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
version: 7
run_install: false
- run: npm run check:translation
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
check-typescript:
- name: Translations check
run: pnpm translation-check
working-directory: ./frontend/apps/remark42
type-check:
name: Type check
runs-on: ubuntu-latest
strategy:
matrix:
node: [14.15]
node: [16.15.1]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- run: npm ci --loglevel warn
working-directory: ./frontend
- uses: actions/cache@v2
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
path: ${{ github.workspace }}/frontend/node_modules/.cache
key: ${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
version: 7
run_install: false
- run: npm run check:types
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
- name: Run type check
run: pnpm type-check
working-directory: ./frontend/apps/remark42
lint:
name: Eslint & Stylelint
runs-on: ubuntu-latest
strategy:
matrix:
node: [14.15]
node: [16.15.1]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- run: npm ci --loglevel warn
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
version: 7
run_install: false
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
- run: npx run-p lint
working-directory: ./frontend
- name: Run linters
run: pnpm lint
working-directory: ./frontend/apps/remark42
size-limit:
name: Size limit
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
env:
CI_JOB_NUMBER: 1
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
version: 7
run_install: false
- name: Check bundle size
uses: andresz1/size-limit-action@dd31dce7dcc72a041fd3e49abf0502b13fc4ce05
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
directory: ./frontend/apps/remark42
package_manager: pnpm
test:
name: Tests & Coverage
runs-on: ubuntu-latest
strategy:
matrix:
node: [14.15]
node: [16.15.1]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
- name: Checkout
uses: actions/checkout@v3
- name: Install node
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- run: npm ci --loglevel warn
- name: Install pnpm
uses: pnpm/action-setup@v2.0.1
id: pnpm-install
with:
version: 7
run_install: false
- name: Get pnpm store directory
id: pnpm-cache
run: |
echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"
- name: Setup pnpm cache
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm i
working-directory: ./frontend
- run: npm run test:coverage
working-directory: ./frontend
- name: Test & Coverage
run: pnpm coverage
working-directory: ./frontend/apps/remark42
- name: submit coverage
run: node ${{ github.workspace }}/frontend/node_modules/.bin/codecov
- name: Submit coverage
run: ${{ github.workspace }}/frontend/apps/remark42/node_modules/.bin/codecov
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+8 -7
View File
@@ -3,6 +3,7 @@ name: site
on:
push:
branches:
- master
tags:
paths:
- ".github/workflows/ci-site.yml"
@@ -13,20 +14,20 @@ on:
- "site/**"
jobs:
build-site:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
- name: set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
- name: available platforms
run: echo ${{ steps.buildx.outputs.platforms }}
@@ -44,7 +45,7 @@ jobs:
echo GITHUB_REF - $ref
echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
docker buildx build --push --no-cache --platform linux/amd64,linux/arm/v7,linux/arm64 \
-t ghcr.io/${USERNAME}/remark24-site:${ref} .
-t ghcr.io/umputun/remark42-site:${ref} .
- name: deploy tagged (latest) to ghcr.io and dockerhub
if: ${{ startsWith(github.ref, 'refs/tags/') }}
@@ -59,7 +60,7 @@ jobs:
echo "GITHUB_REF=$ref, GITHUB_SHA=${GITHUB_SHA}"
echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
docker buildx build --push --no-cache --platform linux/amd64,linux/arm/v7,linux/arm64 \
-t ghcr.io/${USERNAME}/remark24-site:${ref} -t ghcr.io/${USERNAME}/remark24-site:latest .
-t ghcr.io/umputun/remark42-site:${ref} -t ghcr.io/umputun/remark42-site:latest .
- name: remote site deployment from master
if: ${{ github.ref == 'refs/heads/master' }}
+34
View File
@@ -0,0 +1,34 @@
name: e2e
on:
push:
branches: [master]
paths:
- "frontend/apps/remark42/**"
- "frontend/e2e/**"
pull_request:
branches: [master]
paths:
- "frontend/apps/remark42/**"
- "frontend/e2e/**"
jobs:
tests:
name: Tests
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Build & run containers
id: tests
run: COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker compose -f compose-e2e-test.yml up --build --quiet-pull --exit-code-from tests
- uses: actions/upload-artifact@v2
if: always()
with:
name: playwright-report
path: ./playwright-report/
retention-days: 30
+4
View File
@@ -17,8 +17,12 @@ remark42
/bin/
/backend/var/
/backend/app/var/
/backend/app/cmd/web/
/backend/*.html.tmpl
compose-private-backend.yml
compose-private-frontend.yml
compose-private.yml
/backend/_example/*/vendor
http-client.env.json
/playwright-report/
/backend/app/cmd/var
+74 -43
View File
@@ -1,82 +1,113 @@
FROM umputun/baseimage:buildgo-v1.8.0 as build-backend
FROM --platform=$BUILDPLATFORM node:16.15.1-alpine AS frontend-deps
ARG SKIP_FRONTEND_TEST
ARG SKIP_FRONTEND_BUILD
WORKDIR /srv/frontend/
COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml /srv/frontend/
COPY ./frontend/apps/remark42/package.json /srv/frontend/apps/remark42/
RUN \
if [[ -z "$SKIP_FRONTEND_BUILD" || -z "$SKIP_FRONTEND_TEST" ]]; then \
apk add --no-cache --update git && \
npm i -g pnpm@7; \
fi
RUN --mount=type=cache,id=pnpm,target=/root/.pnpm-store/v3 \
if [[ -z "$SKIP_FRONTEND_BUILD" || -z "$SKIP_FRONTEND_TEST" ]]; then \
pnpm i; \
fi
FROM --platform=$BUILDPLATFORM frontend-deps AS build-frontend
ARG SKIP_FRONTEND_TEST
ARG SKIP_FRONTEND_BUILD
ENV CI=true
WORKDIR /srv/frontend/apps/remark42/
COPY ./frontend/apps/remark42/ /srv/frontend/apps/remark42/
RUN \
if [ -z "$SKIP_FRONTEND_TEST" ]; then \
pnpm lint type-check translation-check test; \
else \
echo 'Skip frontend test'; \
fi
RUN \
if [ -z "$SKIP_FRONTEND_BUILD" ]; then \
pnpm build; \
else \
mkdir /srv/frontend/apps/remark42/public; \
echo 'Skip frontend build'; \
fi
FROM umputun/baseimage:buildgo-v1.11.0 as build-backend
ARG CI
ARG DRONE
ARG DRONE_TAG
ARG DRONE_COMMIT
ARG DRONE_BRANCH
ARG DRONE_PULL_REQUEST
ARG GITHUB_REF
ARG GITHUB_SHA
ARG GIT_BRANCH
ARG SKIP_BACKEND_TEST
ARG BACKEND_TEST_TIMEOUT
ADD backend /build/backend
ADD .git/ /build/backend/.git/
# to embed the frontend files statically into Remark42 binary
COPY --from=build-frontend /srv/frontend/apps/remark42/public/ /build/backend/app/cmd/web/
RUN find /build/backend/app/cmd/web/ -regex '.*\.\(html\|js\|mjs\)$' -print -exec sed -i "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \;
WORKDIR /build/backend
ENV GOFLAGS="-mod=vendor"
# install gcc in order to be able to go test package with -race
RUN apk --no-cache add gcc libc-dev
RUN echo go version: `go version`
# run tests
RUN \
cd app && \
if [ -z "$SKIP_BACKEND_TEST" ] ; then \
CGO_ENABLED=1 go test -race -p 1 -timeout="${BACKEND_TEST_TIMEOUT:-300s}" -covermode=atomic -coverprofile=/profile.cov_tmp ./... && \
cat /profile.cov_tmp | grep -v "_mock.go" > /profile.cov ; \
cat /profile.cov_tmp | grep -v "_mock.go" > /profile.cov && \
golangci-lint run --config ../.golangci.yml ./... ; \
else echo "skip backend tests and linter" ; fi
else \
echo "skip backend tests and linter" \
; fi
# if DRONE presented use DRONE_* git env to make version
RUN \
if [ -z "$DRONE" ] ; then echo "runs outside of drone" && version="$(/script/git-rev.sh)" ; \
else version=${DRONE_TAG}${DRONE_BRANCH}${DRONE_PULL_REQUEST}-${DRONE_COMMIT:0:7}-$(date +%Y%m%d-%H:%M:%S) ; fi && \
version="$(/script/version.sh)" && \
echo "version=$version" && \
go build -o remark42 -ldflags "-X main.revision=${version} -s -w" ./app
FROM node:12.16-alpine as build-frontend-deps
FROM umputun/baseimage:app-v1.11.0
ARG CI
ENV HUSKY_SKIP_INSTALL=true
ARG GITHUB_SHA
RUN apk add --no-cache --update git
ADD frontend/package.json /srv/frontend/package.json
ADD frontend/package-lock.json /srv/frontend/package-lock.json
RUN cd /srv/frontend && CI=true npm ci --loglevel warn
FROM node:12.16-alpine as build-frontend
ARG CI
ARG SKIP_FRONTEND_TEST
ARG NODE_ENV=production
COPY --from=build-frontend-deps /srv/frontend/node_modules /srv/frontend/node_modules
ADD frontend /srv/frontend
RUN cd /srv/frontend && \
if [ -z "$SKIP_FRONTEND_TEST" ] ; then npx run-p lint test check; \
else echo "skip frontend tests and lint" ; npm run build ; fi && \
rm -rf ./node_modules
FROM umputun/baseimage:app-v1.8.0
LABEL org.opencontainers.image.authors="Umputun <umputun@gmail.com>" \
org.opencontainers.image.description="Remark42 comment engine" \
org.opencontainers.image.documentation="https://remark42.com/docs/getting-started/" \
org.opencontainers.image.licenses="MIT" \
org.opencontainers.image.source="https://github.com/umputun/remark42.git" \
org.opencontainers.image.title="Remark42" \
org.opencontainers.image.url="https://remark42.com/" \
org.opencontainers.image.revision="${GITHUB_SHA}"
WORKDIR /srv
ADD docker-init.sh /entrypoint.sh
COPY docker-init.sh /srv/init.sh
ADD backend/scripts/backup.sh /usr/local/bin/backup
ADD backend/scripts/restore.sh /usr/local/bin/restore
ADD backend/scripts/import.sh /usr/local/bin/import
RUN chmod +x /entrypoint.sh /usr/local/bin/backup /usr/local/bin/restore /usr/local/bin/import
RUN chmod +x /srv/init.sh /usr/local/bin/backup /usr/local/bin/restore /usr/local/bin/import
COPY --from=build-backend /build/backend/remark42 /srv/remark42
COPY --from=build-backend /build/backend/templates /srv
COPY --from=build-frontend /srv/frontend/public/ /srv/web
COPY docker-init.sh /srv/init.sh
COPY --from=build-frontend /srv/frontend/apps/remark42/public/ /srv/web/
RUN chown -R app:app /srv
RUN ln -s /srv/remark42 /usr/bin/remark42
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s CMD curl --fail http://localhost:8080/ping || exit 1
RUN chmod +x /srv/init.sh
CMD ["/srv/remark42", "server"]
+28 -51
View File
@@ -1,61 +1,41 @@
FROM node:12.16-alpine as build-frontend-deps
FROM node:16.15.1-alpine AS frontend-deps
ARG CI
ARG DRONE
ARG DRONE_TAG
ARG DRONE_COMMIT
ARG DRONE_BRANCH
ENV CI=true
ENV SKIP_FRONTEND_TEST=true
WORKDIR /srv/frontend
RUN apk add --no-cache --update git
ADD frontend/package.json /srv/frontend/package.json
ADD frontend/package-lock.json /srv/frontend/package-lock.json
RUN cd /srv/frontend && CI=true npm ci
COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml /srv/frontend/
COPY ./frontend/apps/remark42/package.json /srv/frontend/apps/remark42/package.json
FROM node:12.16-alpine as build-frontend
RUN apk add --no-cache --update git && npm i -g pnpm@7
RUN --mount=type=cache,id=pnpm,target=/root/.pnpm-store/v3 pnpm i
ARG CI
ARG NODE_ENV=production
ENV SKIP_FRONTEND_TEST=true
ENV HUSKY_SKIP_INSTALL=true
FROM frontend-deps AS build-frontend
COPY --from=build-frontend-deps /srv/frontend/node_modules /srv/frontend/node_modules
ADD frontend /srv/frontend
RUN cd /srv/frontend && \
npm run build && \
rm -rf ./node_modules
ENV NODE_ENV=production
ENV CI=true
FROM umputun/baseimage:buildgo-latest as build-backend
WORKDIR /srv/frontend/apps/remark42/
COPY ./frontend/apps/remark42/ /srv/frontend/apps/remark42/
RUN pnpm build
FROM umputun/baseimage:buildgo-v1.9.2 as build-backend
ARG GITHUB_TOKEN
ENV SKIP_BACKEND_TEST=true
ARG GITHUB_REF
ARG GITHUB_SHA
WORKDIR /build/backend
ADD backend /build/backend
ADD README.md /build/
ADD LICENSE /build/
ADD .git/ /build/backend/.git/
COPY --from=build-frontend /srv/frontend/apps/remark42/public/ /build/backend/app/cmd/web/
COPY --from=build-frontend /srv/frontend/public/ web
RUN find /build/backend/app/cmd/web/ -regex '.*\.\(html\|js\|mjs\)$' -print -exec sed -i "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \;
RUN \
export WEB_ROOT=/build/backend/web && \
find . -regex '.*\.\(html\|js\|mjs\)$' -print -exec sed -i "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \; && \
statik --src=${WEB_ROOT} --dest=/build/backend/app/rest -p api -f && \
statik --src=/build/backend/templates --dest=/build/backend/app -p templates -ns templates -f && \
ls -la /build/backend/app/templates/statik.go && \
ls -la /build/backend/app/rest/api/statik.go && \
ls -la /build/backend/web/
# if DRONE presented use DRONE_* git env to make version
RUN \
if [ -z "$DRONE" ] ; then \
echo "runs outside of drone" && version=$("/script/version.sh"); \
else version=${DRONE_TAG}${DRONE_BRANCH}${DRONE_PULL_REQUEST}-${DRONE_COMMIT:0:7}-$(date +%Y%m%d-%H:%M:%S); fi && \
echo "version=$version" && \
export GOFLAGS="-mod=vendor" && \
version=$("/script/version.sh") && echo "version=${version}" && \
GOOS=linux GOARCH=amd64 go build -o remark42.linux-amd64 -ldflags "-X main.revision=${version} -s -w" ./app && \
GOOS=linux GOARCH=386 go build -o remark42.linux-386 -ldflags "-X main.revision=${version} -s -w" ./app && \
GOOS=linux GOARCH=arm go build -o remark42.linux-arm -ldflags "-X main.revision=${version} -s -w" ./app && \
@@ -66,19 +46,16 @@ RUN \
GOOS=freebsd GOARCH=amd64 go build -o remark42.freebsd-amd64 -ldflags "-X main.revision=${version} -s -w" ./app
RUN \
if [ -z "$DRONE_TAG" ] ; then \
echo "runs outside of drone" && tag=""; \
else tag=_${DRONE_TAG}; fi && \
apk add --no-cache --update zip && \
cp ../LICENSE ./LICENSE && cp ../README.md ./README.md && \
tar cvzf remark42${tag}.linux-amd64.tar.gz remark42.linux-amd64 LICENSE README.md && \
tar cvzf remark42${tag}.linux-386.tar.gz remark42.linux-386 LICENSE README.md && \
tar cvzf remark42${tag}.linux-arm.tar.gz remark42.linux-arm LICENSE README.md && \
tar cvzf remark42${tag}.linux-arm64.tar.gz remark42.linux-arm64 LICENSE README.md && \
tar cvzf remark42${tag}.darwin-amd64.tar.gz remark42.darwin-amd64 LICENSE README.md && \
tar cvzf remark42${tag}.darwin-arm64.tar.gz remark42.darwin-arm64 LICENSE README.md && \
tar cvzf remark42${tag}.freebsd-amd64.tar.gz remark42.freebsd-amd64 LICENSE README.md && \
zip remark42${tag}.windows-amd64.zip remark42.windows-amd64.exe LICENSE README.md
tar cvzf remark42.linux-amd64.tar.gz remark42.linux-amd64 LICENSE README.md && \
tar cvzf remark42.linux-386.tar.gz remark42.linux-386 LICENSE README.md && \
tar cvzf remark42.linux-arm.tar.gz remark42.linux-arm LICENSE README.md && \
tar cvzf remark42.linux-arm64.tar.gz remark42.linux-arm64 LICENSE README.md && \
tar cvzf remark42.darwin-amd64.tar.gz remark42.darwin-amd64 LICENSE README.md && \
tar cvzf remark42.darwin-arm64.tar.gz remark42.darwin-arm64 LICENSE README.md && \
tar cvzf remark42.freebsd-amd64.tar.gz remark42.freebsd-amd64 LICENSE README.md && \
zip remark42.windows-amd64.zip remark42.windows-amd64.exe LICENSE README.md
FROM alpine
+17 -8
View File
@@ -1,5 +1,7 @@
OS=linux
ARCH=amd64
GITHUB_REF=$(shell git rev-parse --symbolic-full-name HEAD)
GITHUB_SHA=$(shell git rev-parse --short HEAD)
bin:
docker build -f Dockerfile.artifacts -t remark42.bin .
@@ -9,15 +11,18 @@ bin:
docker rm -f remark42.bin
docker:
docker build -t umputun/remark42 --build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true .
DOCKER_BUILDKIT=1 docker build -t umputun/remark42 --build-arg GITHUB_REF=$(GITHUB_REF) --build-arg GITHUB_SHA=$(GITHUB_SHA) \
--build-arg CI=true --build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true .
dockerx:
docker buildx build --build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true \
--progress=plain --platform linux/amd64,linux/arm/v7,linux/arm64 \
-t ghcr.io/umputun/remark42:master -t umputun/remark42:master .
docker buildx build --build-arg GITHUB_REF=$(GITHUB_REF) --build-arg GITHUB_SHA=$(GITHUB_SHA) --build-arg CI=true \
--build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true \
--progress=plain --platform linux/amd64,linux/arm/v7,linux/arm64 \
-t ghcr.io/umputun/remark42:master -t umputun/remark42:master .
release:
docker build -f Dockerfile.artifacts --no-cache --pull -t remark42.bin .
docker build -f Dockerfile.artifacts --no-cache --pull --build-arg CI=true \
--build-arg GITHUB_REF=$(GITHUB_REF) --build-arg GITHUB_SHA=$(GITHUB_SHA) -t remark42.bin .
- @docker rm -f remark42.bin 2>/dev/null || exit 0
- @mkdir -p bin
docker run -d --name=remark42.bin remark42.bin
@@ -25,12 +30,13 @@ release:
docker cp remark42.bin:/artifacts/remark42.linux-386.tar.gz bin/remark42.linux-386.tar.gz
docker cp remark42.bin:/artifacts/remark42.linux-arm64.tar.gz bin/remark42.linux-arm64.tar.gz
docker cp remark42.bin:/artifacts/remark42.darwin-amd64.tar.gz bin/remark42.darwin-amd64.tar.gz
docker cp remark42.bin:/artifacts/remark42.darwin-arm64.tar.gz bin/remark42.darwin-arm64.tar.gz
docker cp remark42.bin:/artifacts/remark42.freebsd-amd64.tar.gz bin/remark42.freebsd-amd64.tar.gz
docker cp remark42.bin:/artifacts/remark42.windows-amd64.zip bin/remark42.windows-amd64.zip
docker rm -f remark42.bin
race_test:
cd backend/app && go test -race -mod=vendor -timeout=60s -count 1 ./...
cd backend/app && go test -race -timeout=60s -count 1 ./...
backend:
docker-compose -f compose-dev-backend.yml build
@@ -39,8 +45,11 @@ frontend:
docker-compose -f compose-dev-frontend.yml build
rundev:
docker pull umputun/baseimage:buildgo-latest
SKIP_BACKEND_TEST=true SKIP_FRONTEND_TEST=true docker-compose -f compose-private.yml build
SKIP_BACKEND_TEST=true SKIP_FRONTEND_TEST=true GITHUB_REF=$(GITHUB_REF) GITHUB_SHA=$(GITHUB_SHA) CI=true \
docker-compose -f compose-private.yml build
docker-compose -f compose-private.yml up
e2e:
docker compose -f compose-e2e-test.yml up --build --quiet-pull --exit-code-from tests
.PHONY: bin backend
+10 -4
View File
@@ -1,4 +1,4 @@
# Remark42 [![Build Status](https://github.com/umputun/remark42/workflows/build/badge.svg)](https://github.com/umputun/remark42/actions) [![Go Report Card](https://goreportcard.com/badge/github.com/umputun/remark42)](https://goreportcard.com/report/github.com/umputun/remark42) [![Coverage Status](https://coveralls.io/repos/github/umputun/remark42/badge.svg?branch=master)](https://coveralls.io/github/umputun/remark42?branch=master) [![codecov](https://codecov.io/gh/umputun/remark42/branch/master/graph/badge.svg)](https://app.codecov.io/gh/umputun/remark42)
# Remark42 [![Build Status](https://github.com/umputun/remark42/workflows/build/badge.svg)](https://github.com/umputun/remark42/actions) [![Image Size](https://img.shields.io/docker/image-size/umputun/remark42/master)](https://hub.docker.com/r/umputun/remark42) [![Go Report Card](https://goreportcard.com/badge/github.com/umputun/remark42)](https://goreportcard.com/report/github.com/umputun/remark42) [![Coverage Status](https://coveralls.io/repos/github/umputun/remark42/badge.svg?branch=master)](https://coveralls.io/github/umputun/remark42?branch=master) [![codecov](https://codecov.io/gh/umputun/remark42/branch/master/graph/badge.svg)](https://app.codecov.io/gh/umputun/remark42)
Remark42 is a self-hosted, lightweight and simple (yet functional) comment engine, which doesn't spy on users. It can be embedded into blogs, articles, or any other place where readers add comments.
@@ -23,7 +23,7 @@ Remark42 is a self-hosted, lightweight and simple (yet functional) comment engin
* Clean, lightweight and customizable UI with white and dark themes
* Multi-site mode from a single instance
* Integration with automatic SSL (direct and via [nginx-le](https://github.com/nginx-le/nginx-le))
* [Privacy focused](#privacy)
* [Privacy focused](https://remark42.com/#privacy)
[Demo site](https://remark42.com/demo/) available with all authentication methods, including email auth and anonymous access.
@@ -32,10 +32,16 @@ Remark42 is a self-hosted, lightweight and simple (yet functional) comment engin
Comments example:
![](screenshots/comments.png)
For admin screenshots see [Admin UI wiki](https://github.com/umputun/remark42/wiki/Admin-UI)
For admin screenshots see [Admin UI documentation](https://remark42.com/docs/manuals/admin-interface/)
</details>
All remark42 documentation is available [by the link](https://remark42.com/docs/).
All remark42 documentation is available [by the link](https://remark42.com/docs/getting-started/installation/).
## Contribution
In order to start and work on the project locally in development mode check our contribution documentation for [backend](https://remark42.com/docs/contributing/backend/) and [frontend](https://remark42.com/docs/contributing/frontend/).
If you are interested in adding a new localization please check [these docs](https://remark42.com/docs/contributing/translations/).
## Related projects
+3 -9
View File
@@ -8,8 +8,6 @@ run:
linters-settings:
govet:
check-shadowing: true
golint:
min-confidence: 0.1
maligned:
suggest-new: true
goconst:
@@ -37,18 +35,14 @@ linters:
- revive
- govet
- unconvert
- megacheck
- structcheck
- gas
- gocyclo
- dupl
- misspell
- unparam
- varcheck
- deadcode
- unused
- typecheck
- ineffassign
- varcheck
- stylecheck
- gochecknoinits
- exportloopref
@@ -64,9 +58,9 @@ issues:
- text: "at least one file in a package should have a package comment"
linters:
- stylecheck
- text: "should have a package comment, unless it's in another file for this package"
- text: "package-comments: should have a package comment"
linters:
- golint
- revive
- path: _test\.go
linters:
- gosec
+1
View File
@@ -0,0 +1 @@
../site/src/docs/contributing/backend/index.md
+12 -2
View File
@@ -1,12 +1,22 @@
FROM umputun/baseimage:buildgo-latest as build-backend
FROM umputun/baseimage:buildgo-v1.9.2 as build-backend
ADD backend /build/backend
WORKDIR /build/backend/_example/memory_store
RUN go build -o /build/bin/memory_store -ldflags "-X main.revision=0.0.0 -s -w"
FROM umputun/baseimage:app-v1.9.2
FROM umputun/baseimage:app-latest
ARG GITHUB_SHA
LABEL org.opencontainers.image.authors="Umputun <umputun@gmail.com>" \
org.opencontainers.image.description="Remark42 comment engine example JRPC memory store" \
org.opencontainers.image.documentation="https://github.com/umputun/remark42/tree/master/backend/_example/memory_store" \
org.opencontainers.image.licenses="MIT" \
org.opencontainers.image.source="https://github.com/umputun/remark42.git" \
org.opencontainers.image.title="Remark42 JRPC example memory store" \
org.opencontainers.image.url="https://remark42.com/" \
org.opencontainers.image.revision="${GITHUB_SHA}"
WORKDIR /srv
COPY --from=build-backend /build/bin/memory_store /srv/memory_store
@@ -7,8 +7,9 @@
package accessor
import (
"fmt"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store/admin"
)
@@ -34,7 +35,8 @@ func NewMemAdminStore(key string) *MemAdmin {
return &MemAdmin{data: map[string]AdminRec{}, key: key}
}
// Key executes find by siteID and returns substructure with secret key
// Key supposed to execute find by siteID and returns substructure with secret key,
// but in this case the shared secret is used for all sites
func (m *MemAdmin) Key(_ string) (key string, err error) {
return m.key, nil
}
@@ -43,7 +45,7 @@ func (m *MemAdmin) Key(_ string) (key string, err error) {
func (m *MemAdmin) Admins(siteID string) (ids []string, err error) {
resp, ok := m.data[siteID]
if !ok {
return nil, errors.Errorf("site %s not found", siteID)
return nil, fmt.Errorf("site %s not found", siteID)
}
log.Printf("[DEBUG] admins for %s, %+v", siteID, resp.IDs)
return resp.IDs, nil
@@ -53,7 +55,7 @@ func (m *MemAdmin) Admins(siteID string) (ids []string, err error) {
func (m *MemAdmin) Email(siteID string) (email string, err error) {
resp, ok := m.data[siteID]
if !ok {
return "", errors.Errorf("site %s not found", siteID)
return "", fmt.Errorf("site %s not found", siteID)
}
return resp.Email, nil
@@ -63,7 +65,7 @@ func (m *MemAdmin) Email(siteID string) (email string, err error) {
func (m *MemAdmin) Enabled(siteID string) (ok bool, err error) {
resp, ok := m.data[siteID]
if !ok {
return false, errors.Errorf("site %s not found", siteID)
return false, fmt.Errorf("site %s not found", siteID)
}
return resp.Enabled, nil
}
@@ -72,7 +74,7 @@ func (m *MemAdmin) Enabled(siteID string) (ok bool, err error) {
func (m *MemAdmin) OnEvent(siteID string, ev admin.EventType) error {
resp, ok := m.data[siteID]
if !ok {
return errors.Errorf("site %s not found", siteID)
return fmt.Errorf("site %s not found", siteID)
}
if ev == admin.EvCreate {
resp.CountCreated++ // not a good idea, just for demo
+21 -19
View File
@@ -7,13 +7,12 @@
package accessor
import (
"log"
"fmt"
"sort"
"sync"
"time"
"github.com/pkg/errors"
log "github.com/go-pkgz/lgr"
"github.com/umputun/remark42/backend/app/store"
"github.com/umputun/remark42/backend/app/store/engine"
)
@@ -58,7 +57,7 @@ func NewMemData() *MemData {
func (m *MemData) Create(comment store.Comment) (commentID string, err error) {
if ro, e := m.Flag(engine.FlagRequest{Flag: engine.ReadOnly, Locator: comment.Locator}); e == nil && ro {
return "", errors.Errorf("post %s is read-only", comment.Locator.URL)
return "", fmt.Errorf("post %s is read-only", comment.Locator.URL)
}
m.mu.Lock()
@@ -66,7 +65,7 @@ func (m *MemData) Create(comment store.Comment) (commentID string, err error) {
comments := m.posts[comment.Locator.SiteID]
for _, c := range comments { // don't allow duplicated IDs
if c.ID == comment.ID {
return "", errors.New("dup key")
return "", fmt.Errorf("dup key")
}
}
comments = append(comments, comment)
@@ -161,7 +160,7 @@ func (m *MemData) Count(req engine.FindRequest) (count int, err error) {
})
return len(comments), nil
default:
return 0, errors.Errorf("invalid count request %+v", req)
return 0, fmt.Errorf("invalid count request %+v", req)
}
}
@@ -176,7 +175,7 @@ func (m *MemData) Info(req engine.InfoRequest) (res []store.PostInfo, err error)
return c.Locator == req.Locator
})
if len(comments) == 0 {
return nil, errors.New("not found")
return nil, fmt.Errorf("not found")
}
info := store.PostInfo{
URL: req.Locator.URL,
@@ -235,7 +234,7 @@ func (m *MemData) Info(req engine.InfoRequest) (res []store.PostInfo, err error)
return res, nil
}
return nil, errors.Errorf("invalid info request %+v", req)
return nil, fmt.Errorf("invalid info request %+v", req)
}
// Flag sets and gets flag values
@@ -268,7 +267,7 @@ func (m *MemData) ListFlags(req engine.FlagRequest) (res []interface{}, err erro
return res, nil
case engine.Blocked:
log.Printf("%+v", m.metaUsers)
log.Printf("[INFO] metaUsers: %+v", m.metaUsers)
for _, u := range m.metaUsers {
if u.SiteID == req.Locator.SiteID && u.Blocked && u.BlockedUntil.After(time.Now()) {
res = append(res, store.BlockedUser{ID: u.UserID, Until: u.BlockedUntil})
@@ -277,7 +276,7 @@ func (m *MemData) ListFlags(req engine.FlagRequest) (res []interface{}, err erro
return res, nil
}
return nil, errors.Errorf("flag %s not listable", req.Flag)
return nil, fmt.Errorf("flag %s not listable", req.Flag)
}
// UserDetail sets or gets single detail value, or gets all details fo§r requested site.
@@ -287,7 +286,7 @@ func (m *MemData) UserDetail(req engine.UserDetailRequest) ([]engine.UserDetailE
switch req.Detail {
case engine.UserEmail, engine.UserTelegram:
if req.UserID == "" {
return nil, errors.New("userid cannot be empty in request for single detail")
return nil, fmt.Errorf("userid cannot be empty in request for single detail")
}
m.mu.Lock()
@@ -306,9 +305,9 @@ func (m *MemData) UserDetail(req engine.UserDetailRequest) ([]engine.UserDetailE
defer m.mu.Unlock()
return m.listDetails(req.Locator)
}
return nil, errors.New("unsupported request with userdetail all")
return nil, fmt.Errorf("unsupported request with userdetail all")
default:
return nil, errors.Errorf("unsupported detail %q", req.Detail)
return nil, fmt.Errorf("unsupported detail %q", req.Detail)
}
}
@@ -337,13 +336,13 @@ func (m *MemData) Delete(req engine.DeleteRequest) error {
case req.Locator.SiteID != "" && req.Locator.URL == "" && req.CommentID == "" && req.UserID == "" && req.UserDetail == "": // delete site
if _, ok := m.posts[req.Locator.SiteID]; !ok {
return errors.New("not found")
return fmt.Errorf("not found")
}
m.posts[req.Locator.SiteID] = []store.Comment{}
return nil
}
return errors.Errorf("invalid delete request %+v", req)
return fmt.Errorf("invalid delete request %+v", req)
}
func (m *MemData) deleteComment(loc store.Locator, id string, mode store.DeleteMode) error {
@@ -352,7 +351,7 @@ func (m *MemData) deleteComment(loc store.Locator, id string, mode store.DeleteM
return c.Locator == loc && c.ID == id
})
if len(comments) == 0 {
return errors.New("not found")
return fmt.Errorf("not found")
}
comments[0].SetDeleted(mode)
@@ -430,7 +429,10 @@ func (m *MemData) setFlag(req engine.FlagRequest) (res bool, err error) {
info.ReadOnly = status
m.metaPosts[req.Locator] = info
}
return status, errors.Wrapf(err, "failed to set flag %+v", req)
if err != nil {
return false, fmt.Errorf("failed to set flag %+v: %w", req, err)
}
return status, nil
}
// getUserDetail returns UserDetailEntry with requested userDetail (omitting other details)
@@ -535,7 +537,7 @@ func (m *MemData) get(loc store.Locator, commentID string) (store.Comment, error
return c.Locator == loc && c.ID == commentID
})
if len(comments) == 0 {
return store.Comment{}, errors.New("not found")
return store.Comment{}, fmt.Errorf("not found")
}
return comments[0], nil
}
@@ -557,7 +559,7 @@ func (m *MemData) updateComment(comment store.Comment) error {
m.posts[comment.Locator.SiteID] = comments
return nil
}
return errors.New("not found")
return fmt.Errorf("not found")
}
func (m *MemData) match(comments []store.Comment, fn func(c store.Comment) bool) (res []store.Comment) {
@@ -624,6 +624,7 @@ func TestMemData_DeleteComment(t *testing.T) {
func TestMemData_Close(t *testing.T) {
b := prepMem(t)
assert.NoError(t, b.Close())
assert.NoError(t, b.Close(), "second call should not result in panic or errors")
}
func TestMemData_DeleteHard(t *testing.T) {
@@ -8,11 +8,11 @@ package accessor
import (
"context"
"fmt"
"sync"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store/image"
)
@@ -53,7 +53,7 @@ func (m *MemImage) ResetCleanupTimer(id string) error {
m.insertTime[id] = time.Now()
return nil
}
return errors.Errorf("image %s not found", id)
return fmt.Errorf("image %s not found", id)
}
// Load image by ID
@@ -65,7 +65,7 @@ func (m *MemImage) Load(id string) ([]byte, error) {
}
m.mu.RUnlock()
if !ok {
return nil, errors.Errorf("image %s not found", id)
return nil, fmt.Errorf("image %s not found", id)
}
return img, nil
}
@@ -76,7 +76,7 @@ func (m *MemImage) Commit(id string) error {
img, ok := m.imagesStaging[id]
m.mu.RUnlock()
if !ok {
return errors.Errorf("failed to commit %s, not found in staging", id)
return fmt.Errorf("failed to commit %s, not found in staging", id)
}
m.mu.Lock()
@@ -29,7 +29,6 @@ services:
environment:
- REMARK_URL=http://127.0.0.1:8080
- SECRET=123456
- BACKUP_PATH=/srv/var/backup
- DEBUG=true
- EMOJI=true
- AUTH_ANON=true
+36 -8
View File
@@ -1,14 +1,42 @@
module github.com/umputun/remark42/memory_store
go 1.16
go 1.20
require (
github.com/go-pkgz/jrpc v0.2.0
github.com/go-pkgz/lgr v0.10.4
github.com/pkg/errors v0.9.1
github.com/stretchr/testify v1.7.0
github.com/umputun/go-flags v1.5.1
github.com/umputun/remark42/backend v1.8.1
github.com/go-pkgz/jrpc v0.3.0
github.com/go-pkgz/lgr v0.11.0
github.com/jessevdk/go-flags v1.5.0
github.com/stretchr/testify v1.8.2
github.com/umputun/remark42/backend v1.1000.0
)
replace github.com/umputun/remark42/backend => ../../
require (
github.com/Depado/bfchroma/v2 v2.0.0 // indirect
github.com/PuerkitoBio/goquery v1.8.1 // indirect
github.com/ajg/form v1.5.1 // indirect
github.com/alecthomas/chroma/v2 v2.8.0 // indirect
github.com/andybalholm/cascadia v1.3.2 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/didip/tollbooth/v7 v7.0.1 // indirect
github.com/didip/tollbooth_chi v0.0.0-20220719025231-d662a7f6928f // indirect
github.com/dlclark/regexp2 v1.10.0 // indirect
github.com/go-chi/chi/v5 v5.0.10 // indirect
github.com/go-chi/render v1.0.3 // indirect
github.com/go-pkgz/expirable-cache v1.0.0 // indirect
github.com/go-pkgz/rest v1.17.0 // indirect
github.com/gorilla/css v1.0.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/microcosm-cc/bluemonday v1.0.25 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rs/xid v1.5.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
go.etcd.io/bbolt v1.3.7 // indirect
golang.org/x/image v0.11.0 // indirect
golang.org/x/net v0.14.0 // indirect
golang.org/x/sys v0.11.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
replace github.com/umputun/remark42/backend v1.1000.0 => ../../
+80 -739
View File
@@ -1,786 +1,127 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=
cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=
cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=
cloud.google.com/go v0.83.0/go.mod h1:Z7MJUsANfY0pYPdw0lbnivPx4/vhy/e2FEkSkF7vAVY=
cloud.google.com/go v0.84.0/go.mod h1:RazrYuxIK6Kb7YrzzhPoLmCVzl7Sup4NrbKPg8KHSUM=
cloud.google.com/go v0.87.0/go.mod h1:TpDYlFy7vuLzZMMZ+B6iRiELaY7z/gJPaqbMx6mlWcY=
cloud.google.com/go v0.90.0/go.mod h1:kRX0mNRHe0e2rC6oNakvwQqzyDmg57xJ+SZU1eT2aDQ=
cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI=
cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4=
cloud.google.com/go v0.97.0/go.mod h1:GF7l59pYBVlXQIBLx3a761cZ41F9bBH3JUlihCt2Udc=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Depado/bfchroma v1.3.0 h1:zz14vpvySU6S0CL6yGPr1vkFevQecIt8dJdCsMS2JpM=
github.com/Depado/bfchroma v1.3.0/go.mod h1:c0bFk0tFmT+clD3TIGurjWCfD/QV8/EebfM3JGr+98M=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PuerkitoBio/goquery v1.8.0 h1:PJTF7AmFCFKk1N6V6jmKfrNH9tV5pNE6lZMkG0gta/U=
github.com/PuerkitoBio/goquery v1.8.0/go.mod h1:ypIiRMtY7COPGk+I/YbZLbxsxn9g5ejnI2HSMtkjZvI=
github.com/Depado/bfchroma/v2 v2.0.0 h1:IRpN9BPkNwEpR6w1ectIcNWOuhDSLx+8f1pn83fzxx8=
github.com/Depado/bfchroma/v2 v2.0.0/go.mod h1:wFwW/Pw8Tnd0irzgO9Zxtxgzp3aPS8qBWlyadxujxmw=
github.com/PuerkitoBio/goquery v1.8.1 h1:uQxhNlArOIdbrH1tr0UXwdVFgDcZDrZVdcpygAcwmWM=
github.com/PuerkitoBio/goquery v1.8.1/go.mod h1:Q8ICL1kNUJ2sXGoAhPGUdYDJvgQgHzJsnnd3H7Ho5jQ=
github.com/ajg/form v1.5.1 h1:t9c7v8JUKu/XxOGBU0yjNpaMloxGEJhUkqFRq0ibGeU=
github.com/ajg/form v1.5.1/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY=
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38/go.mod h1:r7bzyVFMNntcxPZXK3/+KdruV1H5KSlyVY0gc+NgInI=
github.com/alecthomas/chroma v0.7.3/go.mod h1:sko8vR34/90zvl5QdcUdvzL3J8NKjAUx9va9jPuFNoM=
github.com/alecthomas/chroma v0.9.4 h1:YL7sOAE3p8HS96T9km7RgvmsZIctqbK1qJ0b7hzed44=
github.com/alecthomas/chroma v0.9.4/go.mod h1:jtJATyUxlIORhUOFNA9NZDWGAQ8wpxQQqNSB4rjA/1s=
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721/go.mod h1:QO9JBoKquHd+jz9nshCh40fOfO+JzsoXy8qTHF68zU0=
github.com/alecthomas/kong v0.2.4/go.mod h1:kQOmtJgV+Lb4aj+I2LEn40cbtawdWJ9Y8QLq+lElKxE=
github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
github.com/alecthomas/repr v0.0.0-20200325044227-4184120f674c/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
github.com/alicebob/gopher-json v0.0.0-20180125190556-5a6b3ba71ee6/go.mod h1:SGnFV6hVsYE877CKEZ6tDNTjaSXYUk6QqoIK6PrAtcc=
github.com/alicebob/miniredis/v2 v2.11.4/go.mod h1:VL3UDEfAH59bSa7MuHMuFToxkqyHh69s/WUbYlOAuyg=
github.com/andybalholm/cascadia v1.3.1 h1:nhxRkql1kdYCc8Snf7D5/D3spOX+dBgjA6u8x004T2c=
github.com/alecthomas/assert/v2 v2.2.1 h1:XivOgYcduV98QCahG8T5XTezV5bylXe+lBxLG2K2ink=
github.com/alecthomas/chroma/v2 v2.8.0 h1:w9WJUjFFmHHB2e8mRpL9jjy3alYDlU0QLDezj1xE264=
github.com/alecthomas/chroma/v2 v2.8.0/go.mod h1:yrkMI9807G1ROx13fhe1v6PN2DDeaR73L3d+1nmYQtw=
github.com/alecthomas/repr v0.2.0 h1:HAzS41CIzNW5syS8Mf9UwXhNH1J9aix/BvDRf1Ml2Yk=
github.com/andybalholm/cascadia v1.3.1/go.mod h1:R4bJ1UQfqADjvDa4P6HZHLh/3OxWWEqc0Sk8XGwHqvA=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/andybalholm/cascadia v1.3.2 h1:3Xi6Dw5lHF15JtdcmAHD3i1+T8plmv7BQ/nsViSLyss=
github.com/andybalholm/cascadia v1.3.2/go.mod h1:7gtRlve5FxPPgIgX36uWBX58OdBsSS6lUvCFb+h7KvU=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964/go.mod h1:Xd9hchkHSWYkEqJwUGisez3G1QY8Ryz0sdWrLPMGjLk=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dghubble/oauth1 v0.7.0/go.mod h1:8pFdfPkv/jr8mkChVbNVuJ0suiHe278BtWI4Tk1ujxk=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/didip/tollbooth/v6 v6.0.1/go.mod h1:j2pKs+JQ5PvU/K4jFnrnwntrmfUbYLJE5oSdxR37FD0=
github.com/didip/tollbooth/v6 v6.1.1 h1:Nt7PvWLa9Y94OrykXsFNBinVRQIu8xdy4avpl99Dc1M=
github.com/didip/tollbooth/v6 v6.1.1/go.mod h1:xjcse6CTHCLuOkzsWrEgdy9WPJFv+p/x6v+MyfP+O9s=
github.com/didip/tollbooth_chi v0.0.0-20200524181329-8b84cd7183d9/go.mod h1:YWyIfq3y4ArRfWZ9XksmuusP+7Mad+T0iFZ0kv0XG/M=
github.com/didip/tollbooth_chi v0.0.0-20200828173446-a7173453ea21 h1:x7YpwKSBIBcKe9I3aTNOqgSyJ6QKDdtOxnEkxBTsi9w=
github.com/didip/tollbooth_chi v0.0.0-20200828173446-a7173453ea21/go.mod h1:0ZVa6kSzS011nfTC1rELyxK4tjVf6vqBnOv7oY2KlsA=
github.com/dlclark/regexp2 v1.2.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/dlclark/regexp2 v1.4.0 h1:F1rxgk7p4uKjwIQxBs9oAXe5CqrXlCduYEJvrF4u93E=
github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fasthttp-contrib/websocket v0.0.0-20160511215533-1f3b11f56072/go.mod h1:duJ4Jxv5lDcvg4QuQr0oowTf7dz4/CR8NtyCooz9HL8=
github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/gavv/httpexpect v2.0.0+incompatible/go.mod h1:x+9tiU1YnrOvnB725RkpoLv1M62hOWzwo5OXotisrKc=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-chi/chi v4.1.1+incompatible h1:MmTgB0R8Bt/jccxp+t6S/1VGIKdJw5J74CK/c9tTfA4=
github.com/go-chi/chi v4.1.1+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ=
github.com/go-chi/chi/v5 v5.0.5/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-chi/cors v1.2.0/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/render v1.0.1 h1:4/5tis2cKaNdnv9zFLfXzcquC9HbeZgCnxGnKrltBS8=
github.com/didip/tollbooth/v7 v7.0.0/go.mod h1:VZhDSGl5bDSPj4wPsih3PFa4Uh9Ghv8hgacaTm5PRT4=
github.com/didip/tollbooth/v7 v7.0.1 h1:TkT4sBKoQoHQFPf7blQ54iHrZiTDnr8TceU+MulVAog=
github.com/didip/tollbooth/v7 v7.0.1/go.mod h1:VZhDSGl5bDSPj4wPsih3PFa4Uh9Ghv8hgacaTm5PRT4=
github.com/didip/tollbooth_chi v0.0.0-20220719025231-d662a7f6928f h1:jtKwihcLmUC9BAhoJ9adCUqdSSZcOdH2KL7mPTUm2aw=
github.com/didip/tollbooth_chi v0.0.0-20220719025231-d662a7f6928f/go.mod h1:q9C80dnsuVRP2dAskjnXRNWdUJqtGgwG9wNrzt0019s=
github.com/dlclark/regexp2 v1.10.0 h1:+/GIL799phkJqYW+3YbOd8LCcbHzT0Pbo8zl70MHsq0=
github.com/dlclark/regexp2 v1.10.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/go-chi/chi/v5 v5.0.7/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-chi/chi/v5 v5.0.10 h1:rLz5avzKpjqxrYwXNfmjkrYYXOyLJd37pz53UFHC6vk=
github.com/go-chi/chi/v5 v5.0.10/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-chi/render v1.0.1/go.mod h1:pq4Rr7HbnsdaeHagklXub+p6Wd16Af5l9koip1OvJns=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-pkgz/auth v1.18.1-0.20211207184750-78e5d290333d/go.mod h1:N1VM53c22Uw0UQSYgdTOBK1D2vtPjSc/RWGf/Fhc1y8=
github.com/go-pkgz/expirable-cache v0.0.3 h1:rTh6qNPp78z0bQE6HDhXBHUwqnV9i09Vm6dksJLXQDc=
github.com/go-pkgz/expirable-cache v0.0.3/go.mod h1:+IauqN00R2FqNRLCLA+X5YljQJrwB179PfiAoMPlTlQ=
github.com/go-pkgz/jrpc v0.2.0 h1:CLy/eZyekjraVrxZV18N2R1mYLMJ/nWrgdfyIOGPY/E=
github.com/go-pkgz/jrpc v0.2.0/go.mod h1:wd8vtQ4CgtCnuqua6x2b1SKIgv0VSOh5Dn0uUITbiUE=
github.com/go-pkgz/lcw v0.8.1/go.mod h1:Xw0/ZfApATgbjVPYRZO4XHdWyxAjErDWDWJ7TLlw1Vc=
github.com/go-pkgz/lgr v0.10.4 h1:l7qyFjqEZgwRgaQQSEp6tve4A3OU80VrfzpvtEX8ngw=
github.com/go-pkgz/lgr v0.10.4/go.mod h1:CD0s1z6EFpIUplV067gitF77tn25JItzwHNKAPqeCF0=
github.com/go-pkgz/repeater v1.1.3/go.mod h1:hVTavuO5x3Gxnu8zW7d6sQBfAneKV8X2FjU48kGfpKw=
github.com/go-pkgz/rest v1.5.0/go.mod h1:nQaM3RhSTUAmbBZWY4hfe4buyeC9VckvhoCktiQXJxI=
github.com/go-pkgz/rest v1.11.0 h1:Z//qgmM0NhBYfhXYEP/aJtDVLK5XlJGxqcb4sHFNN0E=
github.com/go-pkgz/rest v1.11.0/go.mod h1:wZ/dGipZUaF9to0vIQl7PwDHgWQDB0jsrFg1xnAKLDw=
github.com/go-pkgz/syncs v1.1.1/go.mod h1:bt9lxWRRJ9vOCMGc8Big8ttjYHLKP88ofj1y38UlaHE=
github.com/go-redis/redis/v7 v7.4.0/go.mod h1:JDNMw23GTyLNC4GZu9njt15ctBQVn7xjRfnwdHj/Dcg=
github.com/go-session/session v3.1.2+incompatible/go.mod h1:8B3iivBQjrz/JtC68Np2T1yBBLxTan3mn/3OM0CyRt0=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-stack/stack v1.8.1/go.mod h1:dcoOX6HbPZSZptuspn9bctJ+N/CnF5gGygcUP3XYfe4=
github.com/go-test/deep v1.0.4/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
github.com/gobuffalo/attrs v0.0.0-20190224210810-a9411de4debd/go.mod h1:4duuawTqi2wkkpB4ePgWMaai6/Kc6WEz83bhFwpHzj0=
github.com/gobuffalo/depgen v0.0.0-20190329151759-d478694a28d3/go.mod h1:3STtPUQYuzV0gBVOY3vy6CfMm/ljR4pABfrTeHNLHUY=
github.com/gobuffalo/depgen v0.1.0/go.mod h1:+ifsuy7fhi15RWncXQQKjWS9JPkdah5sZvtHc2RXGlg=
github.com/gobuffalo/envy v1.6.15/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI=
github.com/gobuffalo/envy v1.7.0/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI=
github.com/gobuffalo/flect v0.1.0/go.mod h1:d2ehjJqGOH/Kjqcoz+F7jHTBbmDb38yXA598Hb50EGs=
github.com/gobuffalo/flect v0.1.1/go.mod h1:8JCgGVbRjJhVgD6399mQr4fx5rRfGKVzFjbj6RE/9UI=
github.com/gobuffalo/flect v0.1.3/go.mod h1:8JCgGVbRjJhVgD6399mQr4fx5rRfGKVzFjbj6RE/9UI=
github.com/gobuffalo/genny v0.0.0-20190329151137-27723ad26ef9/go.mod h1:rWs4Z12d1Zbf19rlsn0nurr75KqhYp52EAGGxTbBhNk=
github.com/gobuffalo/genny v0.0.0-20190403191548-3ca520ef0d9e/go.mod h1:80lIj3kVJWwOrXWWMRzzdhW3DsrdjILVil/SFKBzF28=
github.com/gobuffalo/genny v0.1.0/go.mod h1:XidbUqzak3lHdS//TPu2OgiFB+51Ur5f7CSnXZ/JDvo=
github.com/gobuffalo/genny v0.1.1/go.mod h1:5TExbEyY48pfunL4QSXxlDOmdsD44RRq4mVZ0Ex28Xk=
github.com/gobuffalo/gitgen v0.0.0-20190315122116-cc086187d211/go.mod h1:vEHJk/E9DmhejeLeNt7UVvlSGv3ziL+djtTr3yyzcOw=
github.com/gobuffalo/gogen v0.0.0-20190315121717-8f38393713f5/go.mod h1:V9QVDIxsgKNZs6L2IYiGR8datgMhB577vzTDqypH360=
github.com/gobuffalo/gogen v0.1.0/go.mod h1:8NTelM5qd8RZ15VjQTFkAW6qOMx5wBbW4dSCS3BY8gg=
github.com/gobuffalo/gogen v0.1.1/go.mod h1:y8iBtmHmGc4qa3urIyo1shvOD8JftTtfcKi+71xfDNE=
github.com/gobuffalo/logger v0.0.0-20190315122211-86e12af44bc2/go.mod h1:QdxcLw541hSGtBnhUc4gaNIXRjiDppFGaDqzbrBd3v8=
github.com/gobuffalo/mapi v1.0.1/go.mod h1:4VAGh89y6rVOvm5A8fKFxYG+wIW6LO1FMTG9hnKStFc=
github.com/gobuffalo/mapi v1.0.2/go.mod h1:4VAGh89y6rVOvm5A8fKFxYG+wIW6LO1FMTG9hnKStFc=
github.com/gobuffalo/packd v0.0.0-20190315124812-a385830c7fc0/go.mod h1:M2Juc+hhDXf/PnmBANFCqx4DM3wRbgDvnVWeG2RIxq4=
github.com/gobuffalo/packd v0.1.0/go.mod h1:M2Juc+hhDXf/PnmBANFCqx4DM3wRbgDvnVWeG2RIxq4=
github.com/gobuffalo/packr/v2 v2.0.9/go.mod h1:emmyGweYTm6Kdper+iywB6YK5YzuKchGtJQZ0Odn4pQ=
github.com/gobuffalo/packr/v2 v2.2.0/go.mod h1:CaAwI0GPIAv+5wKLtv8Afwl+Cm78K/I/VCm/3ptBN+0=
github.com/gobuffalo/syncx v0.0.0-20190224160051-33c29581e754/go.mod h1:HhnNqWY95UYwwW3uSASeV7vtgYkT2t16hJgV3AEPUpw=
github.com/golang-jwt/jwt v3.2.2+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=
github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/gomodule/redigo v1.7.1-0.20190322064113-39e2c31b7ca3/go.mod h1:B4C85qUVwatsJoIUNIfCRsp7qO0iAmpGFZ4EELWSbC4=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.2.1/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/go-chi/render v1.0.3 h1:AsXqd2a1/INaIfUSKq3G5uA8weYx20FOsM7uSoCyyt4=
github.com/go-chi/render v1.0.3/go.mod h1:/gr3hVkmYR0YlEy3LxCuVRFzEu9Ruok+gFqbIofjao0=
github.com/go-pkgz/expirable-cache v0.1.0/go.mod h1:GTrEl0X+q0mPNqN6dtcQXksACnzCBQ5k/k1SwXJsZKs=
github.com/go-pkgz/expirable-cache v1.0.0 h1:ns5+1hjY8hntGv8bPaQd9Gr7Jyo+Uw5SLyII40aQdtA=
github.com/go-pkgz/expirable-cache v1.0.0/go.mod h1:GTrEl0X+q0mPNqN6dtcQXksACnzCBQ5k/k1SwXJsZKs=
github.com/go-pkgz/jrpc v0.3.0 h1:Fls38KqPsHzvp0FWfivr6cGnncC+iFBodHBqvUPY+0U=
github.com/go-pkgz/jrpc v0.3.0/go.mod h1:MFtKs75JESiSqVicsQkgN2iDFFuCd3gVT1/vKiwRi00=
github.com/go-pkgz/lgr v0.11.0 h1:9XH5o+vj09L0sRWEswIGK1lJ6g07xVB4/Z28RV9Z+qM=
github.com/go-pkgz/lgr v0.11.0/go.mod h1:4rdRmMSs4yGFjnUg0rSDbKx21LmFNZoH4y8OLl3qDnU=
github.com/go-pkgz/rest v1.15.6/go.mod h1:KUWAqbDteYGS/CiXftomQsKjtEOifXsJ36Ka0skYbmk=
github.com/go-pkgz/rest v1.17.0 h1:LoBI/lDBMuqwWhOOkc6thM9NnwJO+K9nWvCOjZ7BAgE=
github.com/go-pkgz/rest v1.17.0/go.mod h1:HHlLOt02NJc2sgffXBF6hYVMcRo4Gz3vjg43zTzN7VM=
github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
github.com/gorilla/feeds v1.1.1/go.mod h1:Nk0jZrvPFZX1OBe5NPiddPw7CfwF6Q9eqzaBbaightA=
github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imkira/go-interpol v1.1.0/go.mod h1:z0h2/2T3XF8kyEPpRgJ3kmNv+C43p+I/CoI+jC3w2iA=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88/go.mod h1:3w7q1U84EfirKl04SVQ/s7nPm1ZPhiXd34z40TNz36k=
github.com/karrick/godirwalk v1.8.0/go.mod h1:H5KPZjojv4lE+QYImBI8xVtrBRgYrIVsaRPx4tDPEn4=
github.com/karrick/godirwalk v1.10.3/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.8.2/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/cpuid v1.2.1/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/jessevdk/go-flags v1.5.0 h1:1jKYvbxEjfUl0fmqTCOfonvskHHXMjBySTLW4y9LFvc=
github.com/jessevdk/go-flags v1.5.0/go.mod h1:Fw0T6WPc1dYxT4mKEZRfG5kJhaTDP9pj1c2EWnYs/m4=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kyokomi/emoji/v2 v2.2.8/go.mod h1:JUcn42DTdsXJo1SWanHh4HKDEyPaR5CqkmoirZZP9qE=
github.com/markbates/oncer v0.0.0-20181203154359-bf2de49a0be2/go.mod h1:Ld9puTsIW75CHf65OeIOkyKbteujpZVXDpWK6YGZbxE=
github.com/markbates/safe v1.0.1/go.mod h1:nAqgmRi7cY2nqMc92/bSEeQA+R4OheNU2T1kNSCBdG0=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/microcosm-cc/bluemonday v1.0.16 h1:kHmAq2t7WPWLjiGvzKa5o3HzSfahUKiOq7fAPUiMNIc=
github.com/microcosm-cc/bluemonday v1.0.16/go.mod h1:Z0r70sCuXHig8YpBzCc5eGHAap2K7e/u082ZUpDRRqM=
github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc=
github.com/moul/http2curl v1.0.0/go.mod h1:8UbvGypXm98wA/IqH45anm5Y2Z6ep6O31QGOAZ3H0fQ=
github.com/nullrocks/identicon v0.0.0-20180626043057-7875f45b0022/go.mod h1:x4NsS+uc7ecH/Cbm9xKQ6XzmJM57rWTkjywjfB2yQ18=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.2/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/pelletier/go-toml v1.7.0/go.mod h1:vwGMzjaWMwyfHwgIBhI2YUM4fB6nL6lVAvS1LBMMhTE=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/microcosm-cc/bluemonday v1.0.25 h1:4NEwSfiJ+Wva0VxN5B8OwMicaJvD8r9tlJWm9rtloEg=
github.com/microcosm-cc/bluemonday v1.0.25/go.mod h1:ZIOjCQp1OrzBBPIJmfX4qDYFuhU02nx4bn030ixfHLE=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rakyll/statik v0.1.7/go.mod h1:AlZONWzMtEnMs7W4e/1LURLiI49pIMmp6V9Unghqrcc=
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.2.2/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rs/xid v1.3.0 h1:6NjYksEUlhurdVehpc7S7dk6DAmcKv8V9gG0FsVN2U4=
github.com/rs/xid v1.3.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc=
github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.4.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/slack-go/slack v0.9.5/go.mod h1:wWL//kk0ho+FcQXcBTmEafUI5dz4qz5f4mMk8oIkioQ=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tidwall/btree v0.0.0-20170113224114-9876f1454cf0/go.mod h1:huei1BkDWJ3/sLXmO+bsCNELL+Bp2Kks9OLyQFkzvA8=
github.com/tidwall/buntdb v1.1.0/go.mod h1:Y39xhcDW10WlyYXeLgGftXVbjtM0QP+/kpz8xl9cbzE=
github.com/tidwall/gjson v1.3.2/go.mod h1:P256ACg0Mn+j1RXIDXoss50DeIABTYK1PULOJHhxOls=
github.com/tidwall/gjson v1.11.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/grect v0.0.0-20161006141115-ba9a043346eb/go.mod h1:lKYYLFIr9OIgdgrtgkZ9zgRxRdvPYsExnYBsEAd8W5M=
github.com/tidwall/match v1.0.1/go.mod h1:LujAq0jyVjBy028G1WhWfIzbpQfMO8bBZ6Tyb0+pL9E=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/rtree v0.0.0-20180113144539-6cd427091e0e/go.mod h1:/h+UnNGt0IhNNJLkGikcdcJqm66zGD/uJGMRxK/9+Ao=
github.com/tidwall/tinyqueue v0.0.0-20180302190814-1e39f5511563/go.mod h1:mLqSmt7Dv/CNneF2wfcChfN1rvapyQr01LGKnKex0DQ=
github.com/umputun/go-flags v1.5.1 h1:vRauoXV3Ultt1HrxivSxowbintgZLJE+EcBy5ta3/mY=
github.com/umputun/go-flags v1.5.1/go.mod h1:nTbvsO/hKqe7Utri/NoyN18GR3+EWf+9RrmsdwdhrEc=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.6.0/go.mod h1:FstJa9V+Pj9vQ7OJie2qMHdwemEDaDiSdBnvPM1Su9w=
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
github.com/xdg-go/scram v1.0.2/go.mod h1:1WAq6h33pAW+iRreB34OORO2Nf7qel3VV3fjBj+hCSs=
github.com/xdg-go/stringprep v1.0.2/go.mod h1:8F9zXuvzgwmyT5DUm4GUfZGDdT3W+LCvS6+da4O5kxM=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0/go.mod h1:/LWChgwKmvncFJFHJ7Gvn9wZArjbV5/FppcK2fKk/tI=
github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA=
github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a/go.mod h1:ul22v+Nro/R083muKhosV54bj5niojjWZvU8xrevuH4=
github.com/yudai/gojsondiff v1.0.0/go.mod h1:AY32+k2cwILAkW1fbgxQ5mUmMiZFgLIV+FBNExI05xg=
github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82/go.mod h1:lgjkn3NuSvDfVJdfcVVdX+jpBxNmX4rDAzaS45IcYoM=
github.com/yudai/pp v2.0.1+incompatible/go.mod h1:PuxR/8QJ7cyCkFp/aUDS+JY727OFEZkTdatxwunjIkc=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/gopher-lua v0.0.0-20191220021717-ab39c6098bdb/go.mod h1:gqRgreBUhTSL0GeU64rtZ3Uq3wtjOa/TB2YfrtkCbVQ=
go.etcd.io/bbolt v1.3.6 h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU=
go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4=
go.mongodb.org/mongo-driver v1.7.4/go.mod h1:NqaYOwnXWr5Pm7AOpO5QFxKJ503nbMse/R79oO62zWg=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.etcd.io/bbolt v1.3.7 h1:j+zJOnnEjF/kyHlDDgGnVL/AIqIJPq8UoB2GSNfkUfQ=
go.etcd.io/bbolt v1.3.7/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190422162423-af44ce270edf/go.mod h1:WFFai1msRO1wXaEeE5yQxYXgSfI8pQAWXbQop6sCtWE=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/image v0.0.0-20211028202545-6944b10bf410 h1:hTftEOvwiOq2+O8k2D5/Q7COC7k5Qcrgc2TFURJYnvQ=
golang.org/x/image v0.0.0-20211028202545-6944b10bf410/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/image v0.11.0 h1:ds2RoQvBvYTiJkwpSFDwCcDFNX7DqjL2WsUgTNk0Ooo=
golang.org/x/image v0.11.0/go.mod h1:bglhjqbqVuEb9e9+eNR45Jfu7D+T4Qan+NhQk8Ck2P8=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210916014120-12bc252f5db8/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211105192438-b53810dc28af h1:SMeNJG/vclJ5wyBBd4xupMsSJIHTd1coW9g7q6KOjmY=
golang.org/x/net v0.0.0-20211105192438-b53810dc28af/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190412183630-56d357773e84/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
golang.org/x/net v0.14.0 h1:BONx9s002vGdD9umnlX1Po8vOZmrgH34qlHcD1MfK14=
golang.org/x/net v0.14.0/go.mod h1:PpSgVXXLK0OxS0F31C1/tv6XNguvCrnXIDrFMspZIUI=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190419153524-e8e3143a4f4a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190531175056-4c3a928424d2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200413165638-669c56c373c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211105183446-c75c47738b0c h1:+8miTPjMCTXwih7BQmvWwd0PjdBZq2MKp/qQaahSzEM=
golang.org/x/sys v0.0.0-20211105183446-c75c47738b0c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0 h1:eG7RXZHdqOJ1i+0lgLgCpSXAp6M3LYlAo6osgSi0xOM=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.7.0/go.mod h1:P32HKFT3hSsZrRxla30E9HqToFYAQPCMs/zFMBUFqPY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1 h1:NusfzzA6yGQ+ua51ck7E3omNUX/JuqbFSaRGqU8CcLI=
golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.12.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190329151228-23e29df326fe/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190416151739-9c9e1878f421/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190420181800-aa740d480789/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190531172133-b3315ee88b7d/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=
google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=
google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo=
google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtukyy4=
google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw=
google.golang.org/api v0.51.0/go.mod h1:t4HdrdoNgyN5cbEfm7Lum0lcLDLiise1F8qDKX00sOU=
google.golang.org/api v0.54.0/go.mod h1:7C4bFFOvVDGXjfDTAsgGwDgAxRDeQ4X8NvUedIt6z3k=
google.golang.org/api v0.55.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE=
google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdrMgI=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=
google.golang.org/genproto v0.0.0-20210513213006-bf773b8c8384/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A=
google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=
google.golang.org/genproto v0.0.0-20210604141403-392c879c8b08/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=
google.golang.org/genproto v0.0.0-20210608205507-b6d2f5bf0d7d/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=
google.golang.org/genproto v0.0.0-20210624195500-8bfb893ecb84/go.mod h1:SzzZ/N+nwJDaO1kznhnlzqS8ocJICar6hYhVyhi++24=
google.golang.org/genproto v0.0.0-20210713002101-d411969a0d9a/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k=
google.golang.org/genproto v0.0.0-20210716133855-ce7ef5c701ea/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k=
google.golang.org/genproto v0.0.0-20210728212813-7823e685a01f/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48=
google.golang.org/genproto v0.0.0-20210805201207-89edb61ffb67/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48=
google.golang.org/genproto v0.0.0-20210813162853-db860fec028c/go.mod h1:cFeNkxwySK631ADgubI+/XFU/xp8FD5KIVV4rj8UC5w=
google.golang.org/genproto v0.0.0-20210821163610-241b8fcbd6c8/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
google.golang.org/genproto v0.0.0-20210924002016-3dee208752a0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/grpc v1.37.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=
google.golang.org/grpc v1.37.1/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=
google.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=
google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE=
google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE=
google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34=
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/oauth2.v3 v3.12.0/go.mod h1:XEYgKqWX095YiPT+Aw5y3tCn+7/FMnlTFKrupgSiJ3I=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+8 -10
View File
@@ -12,7 +12,7 @@ import (
"github.com/go-pkgz/jrpc"
log "github.com/go-pkgz/lgr"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"github.com/umputun/remark42/memory_store/accessor"
"github.com/umputun/remark42/memory_store/server"
@@ -43,16 +43,14 @@ func main() {
adminStore := accessor.NewMemAdminStore(opts.Secret)
imgStore := accessor.NewMemImageStore()
rpcServer := jrpc.Server{
API: opts.API,
AuthUser: opts.AuthUser,
AuthPasswd: opts.AuthPasswd,
Version: revision,
AppName: "remark42-memory",
Logger: log.Default(),
}
rpcServer := jrpc.NewServer(
opts.API,
jrpc.Auth(opts.AuthUser, opts.AuthPasswd),
jrpc.WithSignature("remark42-memory", "umputun", revision),
jrpc.WithLogger(log.Default()),
)
srv := server.NewRPC(dataStore, adminStore, imgStore, &rpcServer)
srv := server.NewRPC(dataStore, adminStore, imgStore, rpcServer)
admRec := accessor.AdminRec{
SiteID: "remark",
@@ -198,25 +198,62 @@ func TestRPC_listFlagsHndl(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, "123456", id)
flagReq := engine.FlagRequest{
// verify user
verifyFlagReq := engine.FlagRequest{
Flag: engine.Verified,
UserID: "u1",
Locator: store.Locator{
SiteID: "test-site",
},
}
flags, err := re.ListFlags(flagReq)
flags, err := re.ListFlags(verifyFlagReq)
require.NoError(t, err)
assert.Equal(t, []interface{}{}, flags)
assert.Empty(t, flags)
flagReq.Update = engine.FlagTrue
status, err := re.Flag(flagReq)
verifyFlagReq.Update = engine.FlagTrue
status, err := re.Flag(verifyFlagReq)
require.NoError(t, err)
assert.Equal(t, true, status)
flags, err = re.ListFlags(flagReq)
flags, err = re.ListFlags(verifyFlagReq)
require.NoError(t, err)
assert.Equal(t, []interface{}{"u1"}, flags)
verifiedUsers := make([]string, 0, len(flags))
for _, v := range flags {
verifiedUsers = append(verifiedUsers, v.(string))
}
assert.Equal(t, []string{"u1"}, verifiedUsers)
// block user
blockFlagReq := engine.FlagRequest{
Flag: engine.Blocked,
UserID: "u1",
Locator: store.Locator{
SiteID: "test-site",
},
TTL: time.Hour,
}
flags, err = re.ListFlags(blockFlagReq)
require.NoError(t, err)
assert.Empty(t, flags)
blockFlagReq.Update = engine.FlagTrue
status, err = re.Flag(blockFlagReq)
require.NoError(t, err)
assert.Equal(t, true, status)
flags, err = re.ListFlags(blockFlagReq)
require.NoError(t, err)
assert.NotEmpty(t, flags)
blockedUsers := make([]store.BlockedUser, 0, len(flags))
for _, v := range flags {
blockedUsers = append(blockedUsers, v.(store.BlockedUser))
}
require.Equal(t, 1, len(blockedUsers))
blockedUserInfo := blockedUsers[0]
assert.Equal(t, "u1", blockedUserInfo.ID)
assert.True(t, blockedUserInfo.Until.After(time.Now().Add(time.Minute*59)), "blocked duration is more than 59m away")
assert.True(t, blockedUserInfo.Until.Before(time.Now().Add(time.Minute*61)), "blocked duration is less than 61m away")
}
func TestRPC_userDetailHndl(t *testing.T) {
@@ -301,6 +338,6 @@ func TestRPC_closeHndl(t *testing.T) {
api := fmt.Sprintf("http://localhost:%d/test", port)
re := engine.RPC{Client: jrpc.Client{API: api, Client: http.Client{Timeout: 1 * time.Second}}}
err := re.Close()
assert.NoError(t, err)
assert.NoError(t, re.Close())
assert.NoError(t, re.Close(), "second call should not result in panic or errors")
}
@@ -47,7 +47,7 @@ func prepTestStore(t *testing.T) (port int, teardown func()) {
mg := accessor.NewMemData()
adm := accessor.NewMemAdminStore("secret")
img := accessor.NewMemImageStore()
s := NewRPC(mg, adm, img, &jrpc.Server{API: "/test", Logger: jrpc.NoOpLogger})
s := NewRPC(mg, adm, img, jrpc.NewServer("/test"))
admRec := accessor.AdminRec{
SiteID: "test-site",
+6 -6
View File
@@ -1,10 +1,10 @@
package cmd
import (
"fmt"
"path"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
bolt "go.etcd.io/bbolt"
"github.com/go-pkgz/auth/avatar"
@@ -39,12 +39,12 @@ func (ac *AvatarCommand) Execute(_ []string) error {
src, err := ac.makeAvatarStore(ac.AvatarSrc)
if err != nil {
return errors.Wrapf(err, "can't make avatart store for %s", ac.AvatarSrc.Type)
return fmt.Errorf("can't make avatart store for %s: %w", ac.AvatarSrc.Type, err)
}
dst, err := ac.makeAvatarStore(ac.AvatarDst)
if err != nil {
return errors.Wrapf(err, "can't make avatart store for %s", ac.AvatarDst.Type)
return fmt.Errorf("can't make avatart store for %s: %w", ac.AvatarDst.Type, err)
}
if ac.migrator == nil {
@@ -72,14 +72,14 @@ func (ac *AvatarCommand) makeAvatarStore(gr AvatarGroup) (avatar.Store, error) {
switch gr.Type {
case "fs":
if err := makeDirs(gr.FS.Path); err != nil {
return nil, errors.Wrap(err, "failed to create avatar store")
return nil, fmt.Errorf("failed to create avatar store: %w", err)
}
return avatar.NewLocalFS(gr.FS.Path), nil
case "bolt":
if err := makeDirs(path.Dir(gr.Bolt.File)); err != nil {
return nil, errors.Wrap(err, "failed to create avatar store")
return nil, fmt.Errorf("failed to create avatar store: %w", err)
}
return avatar.NewBoltDB(gr.Bolt.File, bolt.Options{})
}
return nil, errors.Errorf("unsupported avatar store type %s", gr.Type)
return nil, fmt.Errorf("unsupported avatar store type %s", gr.Type)
}
+5 -4
View File
@@ -1,18 +1,17 @@
package cmd
import (
"errors"
"fmt"
"os"
"testing"
"github.com/go-pkgz/auth/avatar"
"github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/umputun/go-flags"
)
func TestAvatar_Execute(t *testing.T) {
defer os.RemoveAll("/tmp/ava-test")
// from fs to bolt
@@ -22,16 +21,18 @@ func TestAvatar_Execute(t *testing.T) {
_, err := p.ParseArgs([]string{"--src.type=fs", "--src.fs.path=/tmp/ava-test", "--dst.type=bolt",
"--dst.bolt.file=/tmp/ava-test.db"})
require.NoError(t, err)
defer os.Remove("/tmp/ava-test.db")
err = cmd.Execute(nil)
assert.NoError(t, err)
// failed
cmd = AvatarCommand{migrator: &avatarMigratorMock{retCount: 0, retError: errors.New("failed blah")}}
cmd = AvatarCommand{migrator: &avatarMigratorMock{retCount: 0, retError: fmt.Errorf("failed blah")}}
cmd.SetCommon(CommonOpts{RemarkURL: "", SharedSecret: "123456"})
p = flags.NewParser(&cmd, flags.Default)
_, err = p.ParseArgs([]string{"--src.type=fs", "--src.fs.path=/tmp/ava-test", "--dst.type=bolt",
"--dst.bolt.file=/tmp/ava-test2.db"})
require.NoError(t, err)
defer os.Remove("/tmp/ava-test2.db")
err = cmd.Execute(nil)
assert.Error(t, err, "failed blah")
}
+10 -11
View File
@@ -9,17 +9,15 @@ import (
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
// BackupCommand set of flags and command for export
// ExportPath used as a separate element to leverage BACKUP_PATH. If ExportFile has a path (i.e. with /) BACKUP_PATH ignored.
type BackupCommand struct {
ExportPath string `short:"p" long:"path" env:"BACKUP_PATH" default:"./var/backup" description:"export path"`
ExportFile string `short:"f" long:"file" default:"userbackup-{{.SITE}}-{{.TS}}.gz" description:"file name"`
Site string `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
Timeout time.Duration `long:"timeout" default:"15m" description:"export (backup) timeout"`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
ExportPath string `short:"p" long:"path" env:"BACKUP_PATH" default:"./var/backup" description:"export path"`
ExportFile string `short:"f" long:"file" default:"userbackup-{{.SITE}}-{{.TS}}.gz" description:"file name"`
SupportCmdOpts
CommonOpts
}
@@ -38,19 +36,20 @@ func (ec *BackupCommand) Execute(_ []string) error {
// prepare http client and request
client := http.Client{}
defer client.CloseIdleConnections()
ctx, cancel := context.WithTimeout(context.Background(), ec.Timeout)
defer cancel()
exportURL := fmt.Sprintf("%s/api/v1/admin/export?mode=file&site=%s", ec.RemarkURL, ec.Site)
req, err := http.NewRequest(http.MethodGet, exportURL, http.NoBody)
if err != nil {
return errors.Wrapf(err, "can't make export request for %s", exportURL)
return fmt.Errorf("can't make export request for %s: %w", exportURL, err)
}
req.SetBasicAuth("admin", ec.AdminPasswd)
// get with timeout
resp, err := client.Do(req.WithContext(ctx))
if err != nil {
return errors.Wrapf(err, "request failed for %s", exportURL)
return fmt.Errorf("request failed for %s: %w", exportURL, err)
}
defer func() {
if err = resp.Body.Close(); err != nil {
@@ -62,9 +61,9 @@ func (ec *BackupCommand) Execute(_ []string) error {
return responseError(resp)
}
fh, err := os.Create(fname)
fh, err := os.Create(fname) //nolint:gosec // harmless
if err != nil {
return errors.Wrapf(err, "can't create backup file %s", fname)
return fmt.Errorf("can't create backup file %s: %w", fname, err)
}
defer func() { //nolint:gosec // false positive on defer without error check when it's checked here
if err = fh.Close(); err != nil {
@@ -73,7 +72,7 @@ func (ec *BackupCommand) Execute(_ []string) error {
}()
if _, err = io.Copy(fh, resp.Body); err != nil {
return errors.Wrapf(err, "failed to write backup file %s", fname)
return fmt.Errorf("failed to write backup file %s: %w", fname, err)
}
log.Printf("[INFO] export completed, file %s", fname)
+1 -2
View File
@@ -7,8 +7,7 @@ import (
"os"
"testing"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
+31 -33
View File
@@ -9,21 +9,20 @@ import (
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
)
// CleanupCommand set of flags and command for cleanup
type CleanupCommand struct {
Site string `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
Dry bool `long:"dry" description:"dry mode, will not remove comments"`
From string `long:"from" description:"from yyyymmdd"`
To string `long:"to" description:"from yyyymmdd"`
BadWords []string `short:"w" long:"bword" description:"bad word(s)"`
BadUsers []string `short:"u" long:"buser" description:"bad user(s)"`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
SetTitle bool `long:"title" description:"title mode, will not remove comments, but reset titles to page's title'"`
Dry bool `long:"dry" description:"dry mode, will not remove comments"`
From string `long:"from" description:"from yyyymmdd"`
To string `long:"to" description:"from yyyymmdd"`
BadWords []string `short:"w" long:"bword" description:"bad word(s)"`
BadUsers []string `short:"u" long:"buser" description:"bad user(s)"`
SetTitle bool `long:"title" description:"title mode, will not remove comments, but reset titles to page's title'"`
SupportCmdOpts
CommonOpts
}
@@ -39,7 +38,7 @@ func (cc *CleanupCommand) Execute(_ []string) error {
posts, err := cc.postsInRange(cc.From, cc.To)
if err != nil {
return errors.Wrap(err, "can't get posts")
return fmt.Errorf("can't get posts: %w", err)
}
log.Printf("[DEBUG] got %d posts", len(posts))
@@ -55,7 +54,6 @@ func (cc *CleanupCommand) Execute(_ []string) error {
cc.procTitles(comments)
} else {
spamComments += cc.procSpam(comments)
}
}
@@ -100,7 +98,7 @@ func (cc *CleanupCommand) procTitles(comments []store.Comment) {
func (cc *CleanupCommand) postsInRange(fromS, toS string) ([]store.PostInfo, error) {
posts, err := cc.listPosts()
if err != nil {
return nil, errors.Wrapf(err, "can't list posts for %s", cc.Site)
return nil, fmt.Errorf("can't list posts for %s: %w", cc.Site, err)
}
from, to := defaultFrom, defaultTo
@@ -108,14 +106,14 @@ func (cc *CleanupCommand) postsInRange(fromS, toS string) ([]store.PostInfo, err
if fromS != "" {
from, err = time.ParseInLocation("20060102", fromS, time.Local)
if err != nil {
return nil, errors.Wrap(err, "can't parse --from")
return nil, fmt.Errorf("can't parse --from: %w", err)
}
}
if toS != "" {
to, err = time.ParseInLocation("20060102", toS, time.Local)
if err != nil {
return nil, errors.Wrap(err, "can't parse --to")
return nil, fmt.Errorf("can't parse --to: %w", err)
}
}
@@ -132,37 +130,38 @@ func (cc *CleanupCommand) postsInRange(fromS, toS string) ([]store.PostInfo, err
func (cc *CleanupCommand) listPosts() ([]store.PostInfo, error) {
listURL := fmt.Sprintf("%s/api/v1/list?site=%s&limit=10000", cc.RemarkURL, cc.Site)
client := http.Client{Timeout: 30 * time.Second}
defer client.CloseIdleConnections()
r, err := client.Get(listURL)
if err != nil {
return nil, errors.Wrapf(err, "get request failed for list of posts, site %s", cc.Site)
return nil, fmt.Errorf("get request failed for list of posts, site %s: %w", cc.Site, err)
}
defer func() { _ = r.Body.Close() }()
if r.StatusCode != 200 {
return nil, errors.Errorf("request %s failed with status %d", listURL, r.StatusCode)
return nil, fmt.Errorf("request %s failed with status %d", listURL, r.StatusCode)
}
list := []store.PostInfo{}
if err = json.NewDecoder(r.Body).Decode(&list); err != nil {
return nil, errors.Wrapf(err, "can't decode list of posts for site %s", cc.Site)
return nil, fmt.Errorf("can't decode list of posts for site %s: %w", cc.Site, err)
}
return list, nil
}
// get all comments for post url via /find?site=siteID&url=post-url&format=[tree|plain]
func (cc *CleanupCommand) listComments(postURL string) ([]store.Comment, error) {
commentsURL := fmt.Sprintf("%s/api/v1/find?site=%s&url=%s&format=plain", cc.RemarkURL, cc.Site, postURL)
var r *http.Response
var err error
// handle 429 error from limiter
client := http.Client{Timeout: 30 * time.Second}
defer client.CloseIdleConnections()
for {
client := http.Client{Timeout: 30 * time.Second}
r, err = client.Get(commentsURL)
if err != nil {
return nil, errors.Wrapf(err, "get request failed for comments, %s", postURL)
return nil, fmt.Errorf("get request failed for comments, %s: %w", postURL, err)
}
if r.StatusCode == http.StatusTooManyRequests {
_ = r.Body.Close()
@@ -175,7 +174,7 @@ func (cc *CleanupCommand) listComments(postURL string) ([]store.Comment, error)
defer func() { _ = r.Body.Close() }()
if r.StatusCode != http.StatusOK {
return nil, errors.Errorf("request %s failed with status %d", commentsURL, r.StatusCode)
return nil, fmt.Errorf("request %s failed with status %d", commentsURL, r.StatusCode)
}
commentsWithInfo := struct {
@@ -184,58 +183,57 @@ func (cc *CleanupCommand) listComments(postURL string) ([]store.Comment, error)
}{}
if err = json.NewDecoder(r.Body).Decode(&commentsWithInfo); err != nil {
return nil, errors.Wrapf(err, "can't decode list of comments for %s", postURL)
return nil, fmt.Errorf("can't decode list of comments for %s: %w", postURL, err)
}
return commentsWithInfo.Comments, nil
}
// deleteComment with DELETE /admin/comment/{id}?site=siteID&url=post-url
func (cc *CleanupCommand) deleteComment(c store.Comment) error {
func (cc *CleanupCommand) deleteComment(c store.Comment) error { //nolint:dupl // not worth combining
deleteURL := fmt.Sprintf("%s/api/v1/admin/comment/%s?site=%s&url=%s&format=plain", cc.RemarkURL, c.ID, cc.Site, c.Locator.URL)
req, err := http.NewRequest("DELETE", deleteURL, http.NoBody)
if err != nil {
return errors.Wrapf(err, "failed to make delete request for comment %s, %s", c.ID, c.Locator.URL)
return fmt.Errorf("failed to make delete request for comment %s, %s: %w", c.ID, c.Locator.URL, err)
}
req.SetBasicAuth("admin", cc.AdminPasswd)
client := http.Client{}
defer client.CloseIdleConnections()
r, err := client.Do(req)
if err != nil {
return errors.Wrapf(err, "delete request failed for comment %s, %s", c.ID, c.Locator.URL)
return fmt.Errorf("delete request failed for comment %s, %s: %w", c.ID, c.Locator.URL, err)
}
defer func() { _ = r.Body.Close() }()
if r.StatusCode != http.StatusOK {
return errors.Errorf("delete request failed with status %s", r.Status)
return fmt.Errorf("delete request failed with status %s", r.Status)
}
return nil
}
// setTitle with PUT /admin/title/{id}?site=siteID&url=post-url
func (cc *CleanupCommand) setTitle(c store.Comment) error {
func (cc *CleanupCommand) setTitle(c store.Comment) error { //nolint:dupl // not worth combining
titleURL := fmt.Sprintf("%s/api/v1/admin/title/%s?site=%s&url=%s&format=plain", cc.RemarkURL, c.ID, cc.Site, c.Locator.URL)
req, err := http.NewRequest("PUT", titleURL, http.NoBody)
if err != nil {
return errors.Wrapf(err, "failed to make title request for comment %s, %s", c.ID, c.Locator.URL)
return fmt.Errorf("failed to make title request for comment %s, %s: %w", c.ID, c.Locator.URL, err)
}
req.SetBasicAuth("admin", cc.AdminPasswd)
client := http.Client{}
defer client.CloseIdleConnections()
r, err := client.Do(req)
if err != nil {
return errors.Wrapf(err, "title request failed for comment %s, %s", c.ID, c.Locator.URL)
return fmt.Errorf("title request failed for comment %s, %s: %w", c.ID, c.Locator.URL, err)
}
defer func() { _ = r.Body.Close() }()
if r.StatusCode != http.StatusOK {
return errors.Errorf("title request failed with status %s", r.Status)
return fmt.Errorf("title request failed with status %s", r.Status)
}
return nil
}
// isSpam calculates spam's probability as a score
func (cc *CleanupCommand) isSpam(comment store.Comment) (isSpam bool, spamScore float64) {
badWord := func(txt string) float64 {
res := 0.0
for _, w := range cc.BadWords {
+1 -3
View File
@@ -10,9 +10,9 @@ import (
"time"
"github.com/go-chi/chi/v5"
"github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/umputun/go-flags"
"github.com/umputun/remark42/backend/app/store"
)
@@ -59,7 +59,6 @@ func TestCleanup_IsSpam(t *testing.T) {
}
func TestCleanup_postsInRange(t *testing.T) {
r := chi.NewRouter()
cleanupRoutes(t, r, nil)
ts := httptest.NewServer(r)
@@ -211,5 +210,4 @@ func cleanupRoutes(t *testing.T, r *chi.Mux, c *cleanedComments) {
c.ids = append(c.ids, r.URL.Path)
c.lock.Unlock()
})
}
+16 -8
View File
@@ -4,6 +4,7 @@ package cmd
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
@@ -13,7 +14,6 @@ import (
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
// CommonOptionsCommander extends flags.Commander with SetCommon
@@ -31,11 +31,20 @@ type CommonOpts struct {
Revision string
}
// SupportCmdOpts is set of commands shared among similar commands like backup/restore and such.
// Order of fields defines the help command output order.
type SupportCmdOpts struct {
Site string `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" default:"" description:"admin basic auth password"`
Timeout time.Duration `long:"timeout" default:"60m" description:"timeout for the command run"`
}
// DeprecatedFlag contains information about deprecated option
type DeprecatedFlag struct {
Old string
New string
Version string
Old string
New string
Version string
Collision bool
}
// SetCommon satisfies CommonOptionsCommander interface and sets common option fields
@@ -58,7 +67,6 @@ type fileParser struct {
// parse apply template and also concat path and file. In case if file contains path separator path will be ignored
func (p *fileParser) parse(now time.Time) (string, error) {
// file/location parameters my have template masks
fileTemplate := struct {
YYYYMMDD string
@@ -87,7 +95,7 @@ func (p *fileParser) parse(now time.Time) (string, error) {
}
if err := template.Must(template.New("bb").Parse(fname)).Execute(&bb, fileTemplate); err != nil {
return "", errors.Wrapf(err, "failed to parse %q", fname)
return "", fmt.Errorf("failed to parse %q: %w", fname, err)
}
return bb.String(), nil
}
@@ -107,14 +115,14 @@ func responseError(resp *http.Response) error {
if e != nil {
body = []byte("")
}
return errors.Errorf("error response %q, %s", resp.Status, body)
return fmt.Errorf("error response %q, %s", resp.Status, body)
}
// mkdir -p for all dirs
func makeDirs(dirs ...string) error {
for _, dir := range dirs {
if err := os.MkdirAll(dir, 0o700); err != nil { // If path is already a directory, MkdirAll does nothing
return errors.Wrapf(err, "can't make directory %s", dir)
return fmt.Errorf("can't make directory %s: %w", dir, err)
}
}
return nil
+11 -13
View File
@@ -8,19 +8,16 @@ import (
"net/http"
"os"
"strings"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
// ImportCommand set of flags and command for import
type ImportCommand struct {
InputFile string `short:"f" long:"file" description:"input file name" required:"true"`
Provider string `short:"p" long:"provider" default:"disqus" choice:"disqus" choice:"wordpress" choice:"commento" description:"import format"` //nolint
Site string `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
Timeout time.Duration `long:"timeout" default:"15m" description:"import timeout"`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
InputFile string `short:"f" long:"file" description:"input file name" required:"true"`
Provider string `short:"p" long:"provider" default:"disqus" choice:"disqus" choice:"wordpress" choice:"commento" description:"import format"` //nolint
SupportCmdOpts
CommonOpts
}
@@ -31,22 +28,23 @@ func (ic *ImportCommand) Execute(_ []string) error {
reader, err := ic.reader(ic.InputFile)
if err != nil {
return errors.Wrapf(err, "can't open import file %s", ic.InputFile)
return fmt.Errorf("can't open import file %s: %w", ic.InputFile, err)
}
client := http.Client{}
defer client.CloseIdleConnections()
ctx, cancel := context.WithTimeout(context.Background(), ic.Timeout)
defer cancel()
importURL := fmt.Sprintf("%s/api/v1/admin/import?site=%s&provider=%s", ic.RemarkURL, ic.Site, ic.Provider)
req, err := http.NewRequest(http.MethodPost, importURL, reader)
if err != nil {
return errors.Wrapf(err, "can't make import request for %s", importURL)
return fmt.Errorf("can't make import request for %s: %w", importURL, err)
}
req.SetBasicAuth("admin", ic.AdminPasswd)
resp, err := client.Do(req.WithContext(ctx)) // closes request's reader
if err != nil {
return errors.Wrapf(err, "request failed for %s", importURL)
return fmt.Errorf("request failed for %s: %w", importURL, err)
}
defer func() {
if err = resp.Body.Close(); err != nil {
@@ -59,7 +57,7 @@ func (ic *ImportCommand) Execute(_ []string) error {
body, err := io.ReadAll(resp.Body)
if err != nil {
return errors.Wrap(err, "can't get response from importer")
return fmt.Errorf("can't get response from importer: %w", err)
}
log.Printf("[INFO] completed, status=%d, %s", resp.StatusCode, string(body))
@@ -70,13 +68,13 @@ func (ic *ImportCommand) Execute(_ []string) error {
func (ic *ImportCommand) reader(inp string) (reader io.Reader, err error) {
inpFile, err := os.Open(inp) // nolint
if err != nil {
return nil, errors.Wrapf(err, "import failed, can't open %s", inp)
return nil, fmt.Errorf("import failed, can't open %s: %w", inp, err)
}
reader = inpFile
if strings.HasSuffix(ic.InputFile, ".gz") {
if reader, err = gzip.NewReader(inpFile); err != nil {
return nil, errors.Wrap(err, "can't make gz reader")
return nil, fmt.Errorf("can't make gz reader: %w", err)
}
}
return reader, nil
+1 -5
View File
@@ -9,14 +9,12 @@ import (
"time"
log "github.com/go-pkgz/lgr"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestImport_Execute(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
assert.Equal(t, "POST", r.Method)
@@ -49,7 +47,6 @@ func TestImport_Execute(t *testing.T) {
}
func TestImport_ExecuteFailed(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
assert.Equal(t, "POST", r.Method)
@@ -103,7 +100,6 @@ func TestImport_ExecuteTimeout(t *testing.T) {
time.Sleep(500 * time.Millisecond)
fmt.Fprintln(w, "some response")
fmt.Fprintln(w, string(body))
}))
defer ts.Close()
+8 -10
View File
@@ -6,19 +6,16 @@ import (
"io"
"net/http"
"os"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
// RemapCommand set of flags and command for change linkage between comments to
// different urls based on given rules (input file)
type RemapCommand struct {
Site string `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
InputFile string `short:"f" long:"file" description:"input file name" required:"true"`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
Timeout time.Duration `long:"timeout" default:"15m" description:"remap timeout"`
InputFile string `short:"f" long:"file" description:"input file name" required:"true"`
SupportCmdOpts
CommonOpts
}
@@ -29,22 +26,23 @@ func (rc *RemapCommand) Execute(_ []string) error {
rulesReader, err := os.Open(rc.InputFile)
if err != nil {
return errors.Wrapf(err, "cant open file %s", rc.InputFile)
return fmt.Errorf("cant open file %s: %w", rc.InputFile, err)
}
client := http.Client{}
defer client.CloseIdleConnections()
ctx, cancel := context.WithTimeout(context.Background(), rc.Timeout)
defer cancel()
remapURL := fmt.Sprintf("%s/api/v1/admin/remap?site=%s", rc.RemarkURL, rc.Site)
req, err := http.NewRequest(http.MethodPost, remapURL, rulesReader)
if err != nil {
return errors.Wrapf(err, "can't make remap request for %s", remapURL)
return fmt.Errorf("can't make remap request for %s: %w", remapURL, err)
}
req.SetBasicAuth("admin", rc.AdminPasswd)
resp, err := client.Do(req.WithContext(ctx))
if err != nil {
return errors.Wrapf(err, "request failed for %s", remapURL)
return fmt.Errorf("request failed for %s: %w", remapURL, err)
}
defer func() {
if err = resp.Body.Close(); err != nil {
@@ -57,7 +55,7 @@ func (rc *RemapCommand) Execute(_ []string) error {
body, err := io.ReadAll(resp.Body)
if err != nil {
return errors.Wrap(err, "can't get response")
return fmt.Errorf("can't get response: %w", err)
}
log.Printf("[INFO] completed, status=%d, %s", resp.StatusCode, string(body))
+1 -3
View File
@@ -6,14 +6,12 @@ import (
"net/http/httptest"
"testing"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRemap_Execute(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, r.URL.Path, "/api/v1/admin/remap")
assert.Equal(t, "POST", r.Method)
+5 -9
View File
@@ -11,9 +11,7 @@ type RestoreCommand struct {
ImportPath string `short:"p" long:"path" env:"BACKUP_PATH" default:"./var/backup" description:"export path"`
ImportFile string `short:"f" long:"file" default:"userbackup-{{.SITE}}-{{.YYYYMMDD}}.gz" description:"file name" required:"true"`
Site string `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
Timeout time.Duration `long:"timeout" default:"15m" description:"import timeout"`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
SupportCmdOpts
CommonOpts
}
@@ -29,12 +27,10 @@ func (rc *RestoreCommand) Execute(args []string) error {
return err
}
importer := ImportCommand{
InputFile: fname,
Site: rc.Site,
Provider: "native",
Timeout: rc.Timeout,
AdminPasswd: rc.AdminPasswd,
CommonOpts: rc.CommonOpts,
InputFile: fname,
Provider: "native",
SupportCmdOpts: rc.SupportCmdOpts,
CommonOpts: rc.CommonOpts,
}
return importer.Execute(args)
}
+1 -3
View File
@@ -7,14 +7,12 @@ import (
"net/http/httptest"
"testing"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRestore_Execute(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
assert.Equal(t, "POST", r.Method)
+223 -144
View File
@@ -2,6 +2,7 @@ package cmd
import (
"context"
"embed"
"fmt"
"net/http"
"net/url"
@@ -16,9 +17,9 @@ import (
"github.com/go-pkgz/jrpc"
"github.com/go-pkgz/lcw/eventbus"
log "github.com/go-pkgz/lgr"
ntf "github.com/go-pkgz/notify"
"github.com/golang-jwt/jwt"
"github.com/kyokomi/emoji/v2"
"github.com/pkg/errors"
bolt "go.etcd.io/bbolt"
"github.com/go-pkgz/auth"
@@ -41,6 +42,9 @@ import (
"github.com/umputun/remark42/backend/app/templates"
)
//go:embed web
var webFS embed.FS
// ServerCommand with command line flags and env
type ServerCommand struct {
Store StoreGroup `group:"store" namespace:"store" env-namespace:"STORE"`
@@ -77,10 +81,11 @@ type ServerCommand struct {
RestrictedWords []string `long:"restricted-words" env:"RESTRICTED_WORDS" description:"words prohibited to use in comments" env-delim:","`
RestrictedNames []string `long:"restricted-names" env:"RESTRICTED_NAMES" description:"names prohibited to use by user" env-delim:","`
EnableEmoji bool `long:"emoji" env:"EMOJI" description:"enable emoji"`
SimpleView bool `long:"simpler-view" env:"SIMPLE_VIEW" description:"minimal comment editor mode"`
SimpleView bool `long:"simple-view" env:"SIMPLE_VIEW" description:"minimal comment editor mode"`
ProxyCORS bool `long:"proxy-cors" env:"PROXY_CORS" description:"disable internal CORS and delegate it to proxy"`
AllowedHosts []string `long:"allowed-hosts" env:"ALLOWED_HOSTS" description:"limit hosts/sources allowed to embed comments"`
AllowedHosts []string `long:"allowed-hosts" env:"ALLOWED_HOSTS" description:"limit hosts/sources allowed to embed comments" env-delim:","`
SubscribersOnly bool `long:"subscribers-only" env:"SUBSCRIBERS_ONLY" description:"enable commenting only for Patreon subscribers"`
DisableSignature bool `long:"disable-signature" env:"DISABLE_SIGNATURE" description:"disable server signature in headers"`
Auth struct {
TTL struct {
@@ -91,16 +96,17 @@ type ServerCommand struct {
SendJWTHeader bool `long:"send-jwt-header" env:"SEND_JWT_HEADER" description:"send JWT as a header instead of cookie"`
SameSite string `long:"same-site" env:"SAME_SITE" description:"set same site policy for cookies" choice:"default" choice:"none" choice:"lax" choice:"strict" default:"default"` // nolint
Google AuthGroup `group:"google" namespace:"google" env-namespace:"GOOGLE" description:"Google OAuth"`
Github AuthGroup `group:"github" namespace:"github" env-namespace:"GITHUB" description:"Github OAuth"`
Facebook AuthGroup `group:"facebook" namespace:"facebook" env-namespace:"FACEBOOK" description:"Facebook OAuth"`
Microsoft AuthGroup `group:"microsoft" namespace:"microsoft" env-namespace:"MICROSOFT" description:"Microsoft OAuth"`
Yandex AuthGroup `group:"yandex" namespace:"yandex" env-namespace:"YANDEX" description:"Yandex OAuth"`
Twitter AuthGroup `group:"twitter" namespace:"twitter" env-namespace:"TWITTER" description:"Twitter OAuth"`
Patreon AuthGroup `group:"patreon" namespace:"patreon" env-namespace:"PATREON" description:"Patreon OAuth"`
Telegram bool `long:"telegram" env:"TELEGRAM" description:"Enable Telegram auth (using token from telegram.token)"`
Dev bool `long:"dev" env:"DEV" description:"enable dev (local) oauth2"`
Anonymous bool `long:"anon" env:"ANON" description:"enable anonymous login"`
Apple AppleGroup `group:"apple" namespace:"apple" env-namespace:"APPLE" description:"Apple OAuth"`
Google AuthGroup `group:"google" namespace:"google" env-namespace:"GOOGLE" description:"Google OAuth"`
Github AuthGroup `group:"github" namespace:"github" env-namespace:"GITHUB" description:"Github OAuth"`
Facebook AuthGroup `group:"facebook" namespace:"facebook" env-namespace:"FACEBOOK" description:"Facebook OAuth"`
Microsoft AuthGroup `group:"microsoft" namespace:"microsoft" env-namespace:"MICROSOFT" description:"Microsoft OAuth"`
Yandex AuthGroup `group:"yandex" namespace:"yandex" env-namespace:"YANDEX" description:"Yandex OAuth"`
Twitter AuthGroup `group:"twitter" namespace:"twitter" env-namespace:"TWITTER" description:"Twitter OAuth"`
Patreon AuthGroup `group:"patreon" namespace:"patreon" env-namespace:"PATREON" description:"Patreon OAuth"`
Telegram bool `long:"telegram" env:"TELEGRAM" description:"Enable Telegram auth (using token from telegram.token)"`
Dev bool `long:"dev" env:"DEV" description:"enable dev (local) oauth2"`
Anonymous bool `long:"anon" env:"ANON" description:"enable anonymous login"`
Email struct {
Enable bool `long:"enable" env:"ENABLE" description:"enable auth via email"`
From string `long:"from" env:"FROM" description:"from email address"`
@@ -128,6 +134,14 @@ type ImageProxyGroup struct {
CacheExternal bool `long:"cache-external" env:"CACHE_EXTERNAL" description:"enable caching for external images"`
}
// AppleGroup defines options for Apple auth params
type AppleGroup struct {
CID string `long:"cid" env:"CID" description:"Apple client ID"`
TID string `long:"tid" env:"TID" description:"Apple service ID"`
KID string `long:"kid" env:"KID" description:"Private key ID"`
PrivateKeyFilePath string `long:"private-key-filepath" env:"PRIVATE_KEY_FILEPATH" description:"Private key file location" default:"/srv/var/apple.p8"`
}
// AuthGroup defines options group for auth params
type AuthGroup struct {
CID string `long:"cid" env:"CID" description:"OAuth client ID"`
@@ -154,7 +168,7 @@ type ImageGroup struct {
} `group:"fs" namespace:"fs" env-namespace:"FS"`
Bolt struct {
File string `long:"file" env:"FILE" default:"./var/pictures.db" description:"images bolt file location"`
} `group:"bolt" namespace:"bolt" env-namespace:"bolt"`
} `group:"bolt" namespace:"bolt" env-namespace:"BOLT"`
MaxSize int `long:"max-size" env:"MAX_SIZE" default:"5000000" description:"max size of image file"`
ResizeWidth int `long:"resize-width" env:"RESIZE_WIDTH" default:"2400" description:"width of a resized image"`
ResizeHeight int `long:"resize-height" env:"RESIZE_HEIGHT" default:"900" description:"height of a resized image"`
@@ -169,7 +183,7 @@ type AvatarGroup struct {
} `group:"fs" namespace:"fs" env-namespace:"FS"`
Bolt struct {
File string `long:"file" env:"FILE" default:"./var/avatars.db" description:"avatars bolt file location"`
} `group:"bolt" namespace:"bolt" env-namespace:"bolt"`
} `group:"bolt" namespace:"bolt" env-namespace:"BOLT"`
URI string `long:"uri" env:"URI" default:"./var/avatars" description:"avatars store URI"`
RszLmt int `long:"rsz-lmt" env:"RESIZE" default:"0" description:"max image size for resizing avatars on save"`
}
@@ -192,7 +206,7 @@ type AdminGroup struct {
Admins []string `long:"id" env:"ID" description:"admin(s) ids" env-delim:","`
Email []string `long:"email" env:"EMAIL" description:"admin emails" env-delim:","`
} `group:"shared" namespace:"shared" env-namespace:"SHARED"`
RPC RPCGroup `group:"rpc" namespace:"rpc" env-namespace:"RPC"`
RPC AdminRPCGroup `group:"rpc" namespace:"rpc" env-namespace:"RPC"`
}
// TelegramGroup defines token for Telegram used in notify and auth modules
@@ -203,12 +217,14 @@ type TelegramGroup struct {
// SMTPGroup defines options for SMTP server connection, used in auth and notify modules
type SMTPGroup struct {
Host string `long:"host" env:"HOST" description:"SMTP host"`
Port int `long:"port" env:"PORT" description:"SMTP port"`
Username string `long:"username" env:"USERNAME" description:"SMTP user name"`
Password string `long:"password" env:"PASSWORD" description:"SMTP password"`
TLS bool `long:"tls" env:"TLS" description:"enable TLS"`
TimeOut time.Duration `long:"timeout" env:"TIMEOUT" default:"10s" description:"SMTP TCP connection timeout"`
Host string `long:"host" env:"HOST" description:"SMTP host"`
Port int `long:"port" env:"PORT" description:"SMTP port"`
Username string `long:"username" env:"USERNAME" description:"SMTP user name"`
Password string `long:"password" env:"PASSWORD" description:"SMTP password"`
TLS bool `long:"tls" env:"TLS" description:"enable TLS"`
LoginAuth bool `long:"login_auth" env:"LOGIN_AUTH" description:"enable LOGIN auth instead of PLAIN"`
StartTLS bool `long:"starttls" env:"STARTTLS" description:"enable StartTLS"`
TimeOut time.Duration `long:"timeout" env:"TIMEOUT" default:"10s" description:"SMTP TCP connection timeout"`
}
// NotifyGroup defines options for notification
@@ -233,10 +249,10 @@ type NotifyGroup struct {
Channel string `long:"chan" env:"CHAN" description:"slack channel for admin notifications"`
} `group:"slack" namespace:"slack" env-namespace:"SLACK"`
Webhook struct {
WebhookURL string `long:"url" env:"URL" description:"webhook URL for admin notifications"`
Template string `long:"template" env:"TEMPLATE" description:"webhook authentication template" default:"{\"text\": \"{{.Text}}\"}"`
Headers []string `long:"headers" description:"webhook authentication headers in format --notify.webhook.headers=Header1:Value1,Value2,..."` // env NOTIFY_WEBHOOK_HEADERS split in code bellow to allow , inside ""
Timeout time.Duration `long:"timeout" env:"TIMEOUT" description:"webhook timeout" default:"5s"`
URL string `long:"url" env:"URL" description:"webhook URL for admin notifications"`
Template string `long:"template" env:"TEMPLATE" description:"webhook authentication template" default:"{\"text\": \"{{.Text}}\"}"`
Headers []string `long:"headers" description:"webhook authentication headers in format --notify.webhook.headers=Header1:Value1,Value2,... [$NOTIFY_WEBHOOK_HEADERS]"` // env NOTIFY_WEBHOOK_HEADERS split in code bellow to allow , inside ""
Timeout time.Duration `long:"timeout" env:"TIMEOUT" description:"webhook timeout" default:"5s"`
} `group:"webhook" namespace:"webhook" env-namespace:"WEBHOOK"`
}
@@ -258,6 +274,12 @@ type RPCGroup struct {
AuthPassword string `long:"auth_passwd" env:"AUTH_PASSWD" description:"basic auth user password"`
}
// AdminRPCGroup defines options for remote admin store
type AdminRPCGroup struct {
RPCGroup
SecretPerSite bool `long:"secret_per_site" env:"SECRET_PER_SITE" description:"enable JWT secret retrieval per aud, which is site_id in this case"`
}
// LoadingCache defines interface for caching
type LoadingCache interface {
Get(key cache.Key, fn func() ([]byte, error)) (data []byte, err error) // load from cache if found or put to cache and return
@@ -321,7 +343,10 @@ func (s *ServerCommand) Execute(_ []string) error {
return nil
}
// HandleDeprecatedFlags sets new flags from deprecated returns their list
// HandleDeprecatedFlags sets new flags from deprecated returns their list.
// Returned list has DeprecatedFlag.Old and DeprecatedFlag.Version set, and DeprecatedFlag.New is optional
// (as some entries are removed without substitute).
// Also it returns flags found by findDeprecatedFlagsCollisions, with DeprecatedFlag.Collision flag set.
func (s *ServerCommand) HandleDeprecatedFlags() (result []DeprecatedFlag) {
if s.Auth.Email.Host != "" && s.SMTP.Host == "" {
s.SMTP.Host = s.Auth.Email.Host
@@ -343,7 +368,8 @@ func (s *ServerCommand) HandleDeprecatedFlags() (result []DeprecatedFlag) {
s.SMTP.Password = s.Auth.Email.SMTPPassword
result = append(result, DeprecatedFlag{Old: "auth.email.passwd", New: "smtp.password", Version: "1.5"})
}
if s.Auth.Email.TimeOut != 10*time.Second && s.SMTP.TimeOut == 10*time.Second {
const emailDefaultTimout = 10 * time.Second
if s.Auth.Email.TimeOut != emailDefaultTimout && s.SMTP.TimeOut == emailDefaultTimout {
s.SMTP.TimeOut = s.Auth.Email.TimeOut
result = append(result, DeprecatedFlag{Old: "auth.email.timeout", New: "smtp.timeout", Version: "1.5"})
}
@@ -354,9 +380,9 @@ func (s *ServerCommand) HandleDeprecatedFlags() (result []DeprecatedFlag) {
s.ImageProxy.HTTP2HTTPS = s.LegacyImageProxy
result = append(result, DeprecatedFlag{Old: "img-proxy", New: "image-proxy.http2https", Version: "1.5"})
}
if len(s.Notify.Type) != 0 && // explicitly empty, would likely never happen due to "none" default
!(len(s.Notify.Type) == 1 && contains("none", s.Notify.Type)) && // ignore default, "none" notify type
(len(s.Notify.Users) != 0 || len(s.Notify.Admins) != 0) { // new notify param(s) are used, safe to ignore the old one
if !contains("none", s.Notify.Type) &&
contains("none", s.Notify.Users) &&
contains("none", s.Notify.Admins) { // if new notify param(s) are used, safe to ignore the old one
s.handleDeprecatedNotifications()
result = append(result, DeprecatedFlag{Old: "notify.type", New: "notify.(users|admins)", Version: "1.9"})
}
@@ -368,14 +394,49 @@ func (s *ServerCommand) HandleDeprecatedFlags() (result []DeprecatedFlag) {
s.Telegram.Token = s.Notify.Telegram.Token
result = append(result, DeprecatedFlag{Old: "notify.telegram.token", New: "telegram.token", Version: "1.9"})
}
const telegramDefaultDuration = time.Second * 5
if s.Notify.Telegram.Timeout != telegramDefaultDuration && s.Telegram.Timeout == telegramDefaultDuration {
const telegramDefaultTimeout = time.Second * 5
if s.Notify.Telegram.Timeout != telegramDefaultTimeout && s.Telegram.Timeout == telegramDefaultTimeout {
s.Telegram.Timeout = s.Notify.Telegram.Timeout
result = append(result, DeprecatedFlag{Old: "notify.telegram.timeout", New: "telegram.timeout", Version: "1.9"})
}
if s.Notify.Telegram.API != "https://api.telegram.org/bot" {
result = append(result, DeprecatedFlag{Old: "notify.telegram.api", Version: "1.9"})
}
return append(result, s.findDeprecatedFlagsCollisions()...)
}
// findDeprecatedFlagsCollisions returns flags which are set both old (deprecated) and new way,
// which means new ones are used and old ones are ignored by deprecated flag handler.
// It returns DeprecatedFlag list which always has only DeprecatedFlag.Old and DeprecatedFlag.New set,
// and DeprecatedFlag.Collision set to true.
func (s *ServerCommand) findDeprecatedFlagsCollisions() (result []DeprecatedFlag) {
if stringsSetAndDifferent(s.Auth.Email.Host, s.SMTP.Host) {
result = append(result, DeprecatedFlag{Old: "auth.email.host", New: "smtp.host", Collision: true})
}
if s.Auth.Email.Port != 0 && s.SMTP.Port != 0 && s.Auth.Email.Port != s.SMTP.Port {
result = append(result, DeprecatedFlag{Old: "auth.email.port", New: "smtp.port", Collision: true})
}
if stringsSetAndDifferent(s.Auth.Email.SMTPUserName, s.SMTP.Username) {
result = append(result, DeprecatedFlag{Old: "auth.email.user", New: "smtp.username", Collision: true})
}
if stringsSetAndDifferent(s.Auth.Email.SMTPPassword, s.SMTP.Password) {
result = append(result, DeprecatedFlag{Old: "auth.email.passwd", New: "smtp.password", Collision: true})
}
const emailDefaultTimout = 10 * time.Second
if s.Auth.Email.TimeOut != emailDefaultTimout && s.SMTP.TimeOut != emailDefaultTimout && s.Auth.Email.TimeOut != s.SMTP.TimeOut {
result = append(result, DeprecatedFlag{Old: "auth.email.timeout", New: "smtp.timeout", Collision: true})
}
if !contains("none", s.Notify.Type) &&
(!contains("none", s.Notify.Users) || !contains("none", s.Notify.Admins)) {
result = append(result, DeprecatedFlag{Old: "notify.type", New: "notify.(users|admins)", Collision: true})
}
if stringsSetAndDifferent(s.Notify.Telegram.Token, s.Telegram.Token) {
result = append(result, DeprecatedFlag{Old: "notify.telegram.token", New: "telegram.token", Collision: true})
}
const telegramDefaultTimeout = time.Second * 5
if s.Notify.Telegram.Timeout != telegramDefaultTimeout && s.Telegram.Timeout != telegramDefaultTimeout && s.Notify.Telegram.Timeout != s.Telegram.Timeout {
result = append(result, DeprecatedFlag{Old: "notify.telegram.timeout", New: "telegram.timeout", Collision: true})
}
return result
}
@@ -390,6 +451,13 @@ func (s *ServerCommand) handleDeprecatedNotifications() {
}
}
func stringsSetAndDifferent(s1, s2 string) bool {
if s1 != "" && s2 != "" && s1 != s2 {
return true
}
return false
}
func contains(s string, a []string) bool {
for _, t := range a {
if t == s {
@@ -402,29 +470,28 @@ func contains(s string, a []string) bool {
// newServerApp prepares application and return it with all active parts
// doesn't start anything
func (s *ServerCommand) newServerApp(ctx context.Context) (*serverApp, error) {
if err := makeDirs(s.BackupLocation); err != nil {
return nil, errors.Wrap(err, "failed to create backup store")
return nil, fmt.Errorf("failed to create backup store: %w", err)
}
if !strings.HasPrefix(s.RemarkURL, "http://") && !strings.HasPrefix(s.RemarkURL, "https://") {
return nil, errors.Errorf("invalid remark42 url %s", s.RemarkURL)
return nil, fmt.Errorf("invalid remark42 url %s", s.RemarkURL)
}
log.Printf("[INFO] root url=%s", s.RemarkURL)
storeEngine, err := s.makeDataStore()
if err != nil {
return nil, errors.Wrap(err, "failed to make data store engine")
return nil, fmt.Errorf("failed to make data store engine: %w", err)
}
adminStore, err := s.makeAdminStore()
if err != nil {
return nil, errors.Wrap(err, "failed to make admin store")
return nil, fmt.Errorf("failed to make admin store: %w", err)
}
imageService, err := s.makePicturesStore()
if err != nil {
return nil, errors.Wrap(err, "failed to make pictures store")
return nil, fmt.Errorf("failed to make pictures store: %w", err)
}
log.Printf("[DEBUG] image service for url=%s, EditDuration=%v", imageService.ImageAPI, imageService.EditDuration)
@@ -446,24 +513,25 @@ func (s *ServerCommand) newServerApp(ctx context.Context) (*serverApp, error) {
loadingCache, err := s.makeCache()
if err != nil {
_ = dataService.Close()
return nil, errors.Wrap(err, "failed to make cache")
return nil, fmt.Errorf("failed to make cache: %w", err)
}
avatarStore, err := s.makeAvatarStore()
if err != nil {
_ = dataService.Close()
return nil, errors.Wrap(err, "failed to make avatar store")
return nil, fmt.Errorf("failed to make avatar store: %w", err)
}
authRefreshCache := newAuthRefreshCache()
authenticator := s.getAuthenticator(dataService, avatarStore, adminStore, authRefreshCache)
telegramAuth := s.makeTelegramAuth(authenticator) // telegram auth requires TelegramAPI listener which is constructed below
telegramService, telegramBotUsername := s.startTelegramAuthAndNotify(ctx, telegramAuth)
telegramService := s.startTelegramAuthAndNotify(ctx, telegramAuth)
err = s.addAuthProviders(authenticator)
if err != nil {
_ = dataService.Close()
return nil, errors.Wrap(err, "failed to make authenticator")
_ = authRefreshCache.Close()
return nil, fmt.Errorf("failed to make authenticator: %w", err)
}
exporter := &migrator.Native{DataStore: dataService}
@@ -502,35 +570,38 @@ func (s *ServerCommand) newServerApp(ctx context.Context) (*serverApp, error) {
sslConfig, err := s.makeSSLConfig()
if err != nil {
_ = dataService.Close()
return nil, errors.Wrap(err, "failed to make config of ssl server params")
_ = authRefreshCache.Close()
return nil, fmt.Errorf("failed to make config of ssl server params: %w", err)
}
srv := &api.Rest{
Version: s.Revision,
DataService: dataService,
WebRoot: s.WebRoot,
RemarkURL: s.RemarkURL,
ImageProxy: imgProxy,
CommentFormatter: commentFormatter,
Migrator: migr,
ReadOnlyAge: s.ReadOnlyAge,
SharedSecret: s.SharedSecret,
Authenticator: authenticator,
Cache: loadingCache,
NotifyService: notifyService,
TelegramService: telegramService,
SSLConfig: sslConfig,
UpdateLimiter: s.UpdateLimit,
ImageService: imageService,
EmailNotifications: contains("email", s.Notify.Users),
TelegramBotUsername: telegramBotUsername,
EmojiEnabled: s.EnableEmoji,
AnonVote: s.AnonymousVote && s.RestrictVoteIP,
SimpleView: s.SimpleView,
ProxyCORS: s.ProxyCORS,
AllowedAncestors: s.AllowedHosts,
SendJWTHeader: s.Auth.SendJWTHeader,
SubscribersOnly: s.SubscribersOnly,
Version: s.Revision,
DataService: dataService,
WebRoot: s.WebRoot,
WebFS: webFS,
RemarkURL: s.RemarkURL,
ImageProxy: imgProxy,
CommentFormatter: commentFormatter,
Migrator: migr,
ReadOnlyAge: s.ReadOnlyAge,
SharedSecret: s.SharedSecret,
Authenticator: authenticator,
Cache: loadingCache,
NotifyService: notifyService,
TelegramService: telegramService,
SSLConfig: sslConfig,
UpdateLimiter: s.UpdateLimit,
ImageService: imageService,
EmailNotifications: contains("email", s.Notify.Users),
TelegramNotifications: contains("telegram", s.Notify.Users) && telegramService != nil,
EmojiEnabled: s.EnableEmoji,
AnonVote: s.AnonymousVote && s.RestrictVoteIP,
SimpleView: s.SimpleView,
ProxyCORS: s.ProxyCORS,
AllowedAncestors: s.AllowedHosts,
SendJWTHeader: s.Auth.SendJWTHeader,
SubscribersOnly: s.SubscribersOnly,
DisableSignature: s.DisableSignature,
}
srv.ScoreThresholds.Low, srv.ScoreThresholds.Critical = s.LowScore, s.CriticalScore
@@ -540,7 +611,8 @@ func (s *ServerCommand) newServerApp(ctx context.Context) (*serverApp, error) {
da, errDevAuth := authenticator.DevAuth()
if errDevAuth != nil {
_ = dataService.Close()
return nil, errors.Wrap(errDevAuth, "can't make dev oauth2 server")
_ = authRefreshCache.Close()
return nil, fmt.Errorf("can't make dev oauth2 server: %w", errDevAuth)
}
devAuth = da
}
@@ -640,7 +712,7 @@ func (s *ServerCommand) makeDataStore() (result engine.Interface, err error) {
switch s.Store.Type {
case "bolt":
if err = makeDirs(s.Store.Bolt.Path); err != nil {
return nil, errors.Wrap(err, "failed to create bolt store")
return nil, fmt.Errorf("failed to create bolt store: %w", err)
}
sites := []engine.BoltSite{}
for _, site := range s.Sites {
@@ -656,9 +728,12 @@ func (s *ServerCommand) makeDataStore() (result engine.Interface, err error) {
}}
return r, nil
default:
return nil, errors.Errorf("unsupported store type %s", s.Store.Type)
return nil, fmt.Errorf("unsupported store type %s", s.Store.Type)
}
return result, errors.Wrap(err, "can't initialize data store")
if err != nil {
return nil, fmt.Errorf("can't initialize data store: %w", err)
}
return result, nil
}
func (s *ServerCommand) makeAvatarStore() (avatar.Store, error) {
@@ -667,18 +742,18 @@ func (s *ServerCommand) makeAvatarStore() (avatar.Store, error) {
switch s.Avatar.Type {
case "fs":
if err := makeDirs(s.Avatar.FS.Path); err != nil {
return nil, errors.Wrap(err, "failed to create avatar store")
return nil, fmt.Errorf("failed to create avatar store: %w", err)
}
return avatar.NewLocalFS(s.Avatar.FS.Path), nil
case "bolt":
if err := makeDirs(path.Dir(s.Avatar.Bolt.File)); err != nil {
return nil, errors.Wrap(err, "failed to create avatar store")
return nil, fmt.Errorf("failed to create avatar store: %w", err)
}
return avatar.NewBoltDB(s.Avatar.Bolt.File, bolt.Options{})
case "uri":
return avatar.NewStore(s.Avatar.URI)
}
return nil, errors.Errorf("unsupported avatar store type %s", s.Avatar.Type)
return nil, fmt.Errorf("unsupported avatar store type %s", s.Avatar.Type)
}
func (s *ServerCommand) makePicturesStore() (*image.Service, error) {
@@ -699,7 +774,7 @@ func (s *ServerCommand) makePicturesStore() (*image.Service, error) {
return image.NewService(boltImageStore, imageServiceParams), nil
case "fs":
if err := makeDirs(s.Image.FS.Path); err != nil {
return nil, errors.Wrap(err, "failed to create pictures store")
return nil, fmt.Errorf("failed to create pictures store: %w", err)
}
return image.NewService(&image.FileSystem{
Location: s.Image.FS.Path,
@@ -715,7 +790,7 @@ func (s *ServerCommand) makePicturesStore() (*image.Service, error) {
AuthPasswd: s.Image.RPC.AuthPassword,
}}, imageServiceParams), nil
}
return nil, errors.Errorf("unsupported pictures store type %s", s.Image.Type)
return nil, fmt.Errorf("unsupported pictures store type %s", s.Image.Type)
}
func (s *ServerCommand) makeAdminStore() (admin.Store, error) {
@@ -741,7 +816,7 @@ func (s *ServerCommand) makeAdminStore() (admin.Store, error) {
}}
return r, nil
default:
return nil, errors.Errorf("unsupported admin store type %s", s.Admin.Type)
return nil, fmt.Errorf("unsupported admin store type %s", s.Admin.Type)
}
}
@@ -751,33 +826,49 @@ func (s *ServerCommand) makeCache() (LoadingCache, error) {
case "redis_pub_sub":
redisPubSub, err := eventbus.NewRedisPubSub(s.Cache.RedisAddr, "remark42-cache")
if err != nil {
return nil, errors.Wrap(err, "cache backend initialization, redis PubSub initialisation")
return nil, fmt.Errorf("cache backend initialization, redis PubSub initialisation: %w", err)
}
backend, err := cache.NewLruCache(cache.MaxCacheSize(s.Cache.Max.Size), cache.MaxValSize(s.Cache.Max.Value),
cache.MaxKeys(s.Cache.Max.Items), cache.EventBus(redisPubSub))
if err != nil {
return nil, errors.Wrap(err, "cache backend initialization")
return nil, fmt.Errorf("cache backend initialization: %w", err)
}
return cache.NewScache(backend), nil
case "mem":
backend, err := cache.NewLruCache(cache.MaxCacheSize(s.Cache.Max.Size), cache.MaxValSize(s.Cache.Max.Value),
cache.MaxKeys(s.Cache.Max.Items))
if err != nil {
return nil, errors.Wrap(err, "cache backend initialization")
return nil, fmt.Errorf("cache backend initialization: %w", err)
}
return cache.NewScache(backend), nil
case "none":
return cache.NewScache(&cache.Nop{}), nil
}
return nil, errors.Errorf("unsupported cache type %s", s.Cache.Type)
return nil, fmt.Errorf("unsupported cache type %s", s.Cache.Type)
}
//nolint:gocyclo // simple code but many if checks
func (s *ServerCommand) addAuthProviders(authenticator *auth.Service) error {
providersCount := 0
if s.Auth.Telegram {
providersCount++
}
if s.Auth.Apple.CID != "" && s.Auth.Apple.TID != "" && s.Auth.Apple.KID != "" {
err := authenticator.AddAppleProvider(
provider.AppleConfig{
ClientID: s.Auth.Apple.CID,
TeamID: s.Auth.Apple.TID,
KeyID: s.Auth.Apple.KID,
ResponseMode: "query", // default is form_post which wouldn't work here
},
provider.LoadApplePrivateKeyFromFile(s.Auth.Apple.PrivateKeyFilePath),
)
if err != nil {
return err
}
providersCount++
}
if s.Auth.Google.CID != "" && s.Auth.Google.CSEC != "" {
authenticator.AddProvider("google", s.Auth.Google.CID, s.Auth.Google.CSEC)
providersCount++
@@ -809,7 +900,11 @@ func (s *ServerCommand) addAuthProviders(authenticator *auth.Service) error {
if s.Auth.Dev {
log.Print("[INFO] dev access enabled")
authenticator.AddProvider("dev", "", "")
u, errURL := url.Parse(s.RemarkURL)
if errURL != nil {
return fmt.Errorf("can't parse Remark42 URL: %w", errURL)
}
authenticator.AddDevProvider(u.Hostname(), 8084)
providersCount++
}
@@ -820,23 +915,26 @@ func (s *ServerCommand) addAuthProviders(authenticator *auth.Service) error {
SMTPUserName: s.SMTP.Username,
SMTPPassword: s.SMTP.Password,
TimeOut: s.SMTP.TimeOut,
StartTLS: s.SMTP.StartTLS,
LoginAuth: s.SMTP.LoginAuth,
TLS: s.SMTP.TLS,
Charset: "UTF-8",
From: s.Auth.Email.From,
Subject: s.Auth.Email.Subject,
ContentType: s.Auth.Email.ContentType,
}
sndr := sender.NewEmailClient(params, log.Default())
tmpl, err := s.loadEmailTemplate()
tmpl, err := templates.Read(s.Auth.Email.MsgTemplate)
if err != nil {
return err
}
authenticator.AddVerifProvider("email", tmpl, sndr)
authenticator.AddVerifProvider("email", string(tmpl), sndr)
}
if s.Auth.Anonymous {
log.Print("[INFO] anonymous access enabled")
var isValidAnonName = regexp.MustCompile(`^[\p{L}\d_ ]+$`).MatchString
authenticator.AddDirectProvider("anonymous", provider.CredCheckerFunc(func(user, _ string) (ok bool, err error) {
authenticator.AddDirectProviderWithUserIDFunc("anonymous", provider.CredCheckerFunc(func(user, _ string) (ok bool, err error) {
// don't allow anon with space prefix or suffix
if strings.HasPrefix(user, " ") || strings.HasSuffix(user, " ") {
@@ -859,7 +957,12 @@ func (s *ServerCommand) addAuthProviders(authenticator *auth.Service) error {
return false, nil
}
return true, nil
}))
}),
// Custom user ID generator, used to distinguish anonymous users with the same login
// coming from different IPs
func(user string, r *http.Request) string {
return user + r.RemoteAddr
})
}
if providersCount == 0 {
@@ -869,27 +972,6 @@ func (s *ServerCommand) addAuthProviders(authenticator *auth.Service) error {
return nil
}
// loadEmailTemplate trying to get template from statik
func (s *ServerCommand) loadEmailTemplate() (string, error) {
var file []byte
var err error
if s.Auth.Email.MsgTemplate == "email_confirmation_login.html.tmpl" {
fs := templates.NewFS()
file, err = fs.ReadFile(s.Auth.Email.MsgTemplate)
} else {
// deprecated loading from an external file, should be removed before v1.9.0
file, err = os.ReadFile(s.Auth.Email.MsgTemplate)
log.Printf("[INFO] template %s will be read from disk", s.Auth.Email.MsgTemplate)
}
if err != nil {
return "", errors.Wrapf(err, "failed to read file %s", s.Auth.Email.MsgTemplate)
}
return string(file), nil
}
// creates and registers telegram auth, which we need separately from other auth providers
func (s *ServerCommand) makeTelegramAuth(authenticator *auth.Service) providers.TGUpdatesReceiver {
if s.Auth.Telegram {
@@ -911,7 +993,8 @@ func (s *ServerCommand) makeNotifyService(dataStore *service.DataStore, destinat
if destinations == nil {
destinations = []notify.Destination{}
}
if telegram != nil {
// it's possible that telegram notification service was created for auth but should not be used for notifications
if telegram != nil && (contains("telegram", s.Notify.Users) || contains("telegram", s.Notify.Admins)) {
destinations = append(destinations, telegram)
}
@@ -927,29 +1010,26 @@ func (s *ServerCommand) makeNotifyDestinations(authenticator *auth.Service) ([]n
destinations := make([]notify.Destination, 0)
if contains("webhook", s.Notify.Admins) {
client := &http.Client{Timeout: 5 * time.Second}
webhookHeaders := s.Notify.Webhook.Headers
if len(webhookHeaders) == 0 {
webhookHeaders = splitAtCommas(os.Getenv("NOTIFY_WEBHOOK_HEADERS")) // env value may have comma inside "", parsed separately
}
whParams := notify.WebhookParams{
WebhookURL: s.Notify.Webhook.WebhookURL,
Template: s.Notify.Webhook.Template,
Headers: webhookHeaders,
URL: s.Notify.Webhook.URL,
Template: s.Notify.Webhook.Template,
Headers: webhookHeaders,
Timeout: time.Second * 5,
}
webhook, err := notify.NewWebhook(client, whParams)
webhook, err := notify.NewWebhook(whParams)
if err != nil {
return destinations, errors.Wrap(err, "failed to create webhook notification destination")
return destinations, fmt.Errorf("failed to create webhook notification destination: %w", err)
}
destinations = append(destinations, webhook)
}
if contains("slack", s.Notify.Admins) {
slack, err := notify.NewSlack(s.Notify.Slack.Token, s.Notify.Slack.Channel)
if err != nil {
return destinations, errors.Wrap(err, "failed to create slack notification destination")
}
slack := notify.NewSlack(s.Notify.Slack.Token, s.Notify.Slack.Channel)
destinations = append(destinations, slack)
}
@@ -976,7 +1056,7 @@ func (s *ServerCommand) makeNotifyDestinations(authenticator *auth.Service) ([]n
}
tkn, err := authenticator.TokenService().Token(claims)
if err != nil {
return "", errors.Wrapf(err, "failed to make unsubscription token")
return "", fmt.Errorf("failed to make unsubscription token: %w", err)
}
return tkn, nil
},
@@ -984,17 +1064,21 @@ func (s *ServerCommand) makeNotifyDestinations(authenticator *auth.Service) ([]n
if contains("email", s.Notify.Admins) {
emailParams.AdminEmails = s.Admin.Shared.Email
}
smtpParams := notify.SMTPParams{
Host: s.SMTP.Host,
Port: s.SMTP.Port,
TLS: s.SMTP.TLS,
Username: s.SMTP.Username,
Password: s.SMTP.Password,
TimeOut: s.SMTP.TimeOut,
smtpParams := ntf.SMTPParams{
Host: s.SMTP.Host,
Port: s.SMTP.Port,
TLS: s.SMTP.TLS,
StartTLS: s.SMTP.StartTLS,
LoginAuth: s.SMTP.LoginAuth,
Username: s.SMTP.Username,
Password: s.SMTP.Password,
TimeOut: s.SMTP.TimeOut,
ContentType: "text/html",
Charset: "UTF-8",
}
emailService, err := notify.NewEmail(emailParams, smtpParams)
if err != nil {
return destinations, errors.Wrap(err, "failed to create email notification destination")
return destinations, fmt.Errorf("failed to create email notification destination: %w", err)
}
destinations = append(destinations, emailService)
}
@@ -1005,7 +1089,7 @@ func (s *ServerCommand) makeNotifyDestinations(authenticator *auth.Service) ([]n
// constructs Telegram notify service
func (s *ServerCommand) makeTelegramNotify() (*notify.Telegram, error) {
if contains("telegram", s.Notify.Admins) && s.Notify.Telegram.Channel == "" {
return nil, errors.New("--notify.telegram.channel must be set for admin notifications to work")
return nil, fmt.Errorf("--notify.telegram.channel must be set for admin notifications to work")
}
telegramParams := notify.TelegramParams{
AdminChannelID: s.Notify.Telegram.Channel,
@@ -1016,7 +1100,7 @@ func (s *ServerCommand) makeTelegramNotify() (*notify.Telegram, error) {
}
tg, err := notify.NewTelegram(telegramParams)
if err != nil {
return nil, errors.Wrap(err, "failed to create telegram notification destination")
return nil, fmt.Errorf("failed to create telegram notification destination: %w", err)
}
return tg, nil
}
@@ -1027,10 +1111,10 @@ func (s *ServerCommand) makeSSLConfig() (config api.SSLConfig, err error) {
config.SSLMode = api.None
case "static":
if s.SSL.Cert == "" {
return config, errors.New("path to cert.pem is required")
return config, fmt.Errorf("path to cert.pem is required")
}
if s.SSL.Key == "" {
return config, errors.New("path to key.pem is required")
return config, fmt.Errorf("path to key.pem is required")
}
config.SSLMode = api.Static
config.Port = s.SSL.Port
@@ -1062,7 +1146,7 @@ func (s *ServerCommand) getAuthenticator(ds *service.DataStore, avas avatar.Stor
SameSiteCookie: s.parseSameSite(s.Auth.SameSite),
SecureCookies: strings.HasPrefix(s.RemarkURL, "https://"),
SecretReader: token.SecretFunc(func(aud string) (string, error) { // get secret per site
return admns.Key("")
return admns.Key(aud)
}),
ClaimsUpd: token.ClaimsUpdFunc(func(c token.Claims) token.Claims { // set attributes, on new token or refresh
if c.User == nil {
@@ -1107,6 +1191,7 @@ func (s *ServerCommand) getAuthenticator(ds *service.DataStore, avas avatar.Stor
Logger: log.Default(),
RefreshCache: authRefreshCache,
UseGravatar: true,
AudSecrets: s.Admin.RPC.SecretPerSite,
})
}
@@ -1127,20 +1212,15 @@ func (s *ServerCommand) parseSameSite(ss string) http.SameSite {
// startTelegramAuthAndNotify initializes telegram notify and auth Telegram Bot listen loop.
// Does nothing if telegram auth and notifications are disabled.
// Doesn't return telegram bot username if user notifications are disabled, as that is the way frontend knows they are enabled.
func (s *ServerCommand) startTelegramAuthAndNotify(ctx context.Context, telegramAuth providers.TGUpdatesReceiver) (tg *notify.Telegram, telegramBotUsername string) {
func (s *ServerCommand) startTelegramAuthAndNotify(ctx context.Context, telegramAuth providers.TGUpdatesReceiver) (tg *notify.Telegram) {
if !contains("telegram", s.Notify.Users) && !contains("telegram", s.Notify.Admins) && !s.Auth.Telegram {
return nil, ""
return nil
}
var err error
if tg, err = s.makeTelegramNotify(); err != nil {
log.Printf("[WARN] failed to make telegram notify service, %s", err)
return nil, ""
}
if contains("telegram", s.Notify.Users) {
telegramBotUsername = tg.GetBotUsername()
return nil
}
telegramReceivers := []providers.TGUpdatesReceiver{tg}
@@ -1150,14 +1230,13 @@ func (s *ServerCommand) startTelegramAuthAndNotify(ctx context.Context, telegram
// start bot messages receiver for both notify and auth services
go providers.DispatchTelegramUpdates(ctx, tg, telegramReceivers, time.Second*5)
return tg, telegramBotUsername
return tg
}
// splitAtCommas split s at commas, ignoring commas in strings.
// Eliminate leading and trailing dbl quotes in each element only if both presented
// based on https://stackoverflow.com/a/59318708
func splitAtCommas(s string) []string {
cleanup := func(s string) string {
if s == "" {
return s
+138 -46
View File
@@ -17,7 +17,7 @@ import (
"github.com/go-pkgz/auth/token"
"github.com/golang-jwt/jwt"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"go.uber.org/goleak"
"github.com/stretchr/testify/assert"
@@ -36,9 +36,10 @@ func TestServerApp(t *testing.T) {
// send ping
resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
defer http.DefaultClient.CloseIdleConnections()
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, "pong", string(body))
@@ -78,12 +79,13 @@ func TestServerApp_DevMode(t *testing.T) {
waitForHTTPServerStart(port)
providers := app.restSrv.Authenticator.Providers()
require.Equal(t, 9+1, len(providers), "extra auth provider")
require.Equal(t, 10+1, len(providers), "extra auth provider")
assert.Equal(t, "dev", providers[len(providers)-2].Name(), "dev auth provider")
// send ping
resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
defer http.DefaultClient.CloseIdleConnections()
require.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
@@ -105,11 +107,14 @@ func TestServerApp_AnonMode(t *testing.T) {
waitForHTTPServerStart(port)
providers := app.restSrv.Authenticator.Providers()
require.Equal(t, 9+1, len(providers), "extra auth provider for anon")
require.Equal(t, 10+1, len(providers), "extra auth provider for anon")
assert.Equal(t, "anonymous", providers[len(providers)-1].Name(), "anon auth provider")
client := http.Client{Timeout: 10 * time.Second}
defer client.CloseIdleConnections()
// send ping
resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
resp, err := client.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
@@ -118,14 +123,12 @@ func TestServerApp_AnonMode(t *testing.T) {
assert.Equal(t, "pong", string(body))
// try to login with good name
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=blah123&aud=remark", port))
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=blah123&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
// try to add a comment as good anonymous
client := http.Client{Timeout: 10 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/blah1", "site": "remark"}}`))
require.NoError(t, err)
@@ -140,50 +143,57 @@ func TestServerApp_AnonMode(t *testing.T) {
assert.Equal(t, http.StatusCreated, resp.StatusCode)
// try to login with non-latin name
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=Раз_Два%20%20Три_34567&aud=remark", port))
time.Sleep(time.Second)
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=Раз_Два%20%20Три_34567&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
// try to login with bad name
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=**blah123&aud=remark", port))
time.Sleep(time.Second)
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=**blah123&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
// try to login with short name
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=bl%%20%%20&aud=remark", port))
time.Sleep(time.Second)
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=bl%%20%%20&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
// try to login with name what have space in prefix
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%%20somebody&aud=remark", port))
time.Sleep(time.Second)
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%%20somebody&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
// try to login with name what have space in suffix
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=somebody%%20&aud=remark", port))
time.Sleep(time.Second)
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=somebody%%20&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
// try to login with long name
time.Sleep(time.Second)
ln := strings.Repeat("x", 65)
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%s&aud=remark", port, ln))
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%s&aud=remark", port, ln))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
// try to login with admin name
resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=umpUtun&aud=remark", port))
time.Sleep(time.Second)
resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=umpUtun&aud=remark", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
// try to add a comment as anonymous with admin name
client = http.Client{Timeout: 10 * time.Second}
time.Sleep(time.Second)
req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/blah1", "site": "remark"}}`))
require.NoError(t, err)
@@ -226,6 +236,9 @@ func TestServerApp_WithSSL(t *testing.T) {
"--ssl.type=static", "--ssl.cert=testdata/cert.pem", "--ssl.key=testdata/key.pem",
"--ssl.port=" + strconv.Itoa(sslPort), "--image.fs.path=/tmp"})
require.NoError(t, err)
defer os.Remove("/tmp/xyz")
defer os.Remove("/tmp/xyz/remark.db")
defer os.Remove("/tmp/ava-test.db")
// create app
app, err := opts.newServerApp(context.Background())
@@ -246,19 +259,20 @@ func TestServerApp_WithSSL(t *testing.T) {
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
defer client.CloseIdleConnections()
// check http to https redirect response
resp, err := client.Get(fmt.Sprintf("http://localhost:%d/blah?param=1", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 307, resp.StatusCode)
assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
assert.Equal(t, fmt.Sprintf("https://localhost:%d/blah?param=1", sslPort), resp.Header.Get("Location"))
// check https server
resp, err = client.Get(fmt.Sprintf("https://localhost:%d/ping", sslPort))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, "pong", string(body))
@@ -268,7 +282,6 @@ func TestServerApp_WithSSL(t *testing.T) {
}
func TestServerApp_WithRemote(t *testing.T) {
opts := ServerCommand{}
opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
@@ -277,7 +290,8 @@ func TestServerApp_WithRemote(t *testing.T) {
port := chooseRandomUnusedPort()
_, err := p.ParseArgs([]string{"--admin-passwd=password", "--cache.type=none",
"--store.type=rpc", "--store.rpc.api=http://127.0.0.1",
"--port=" + strconv.Itoa(port), "--admin.type=rpc", "--admin.rpc.api=http://127.0.0.1", "--avatar.fs.path=/tmp"})
"--port=" + strconv.Itoa(port), "--avatar.fs.path=/tmp",
"--admin.type=rpc", "--admin.rpc.secret_per_site", "--admin.rpc.api=http://127.0.0.1"})
require.NoError(t, err)
opts.Auth.Github.CSEC, opts.Auth.Github.CID = "csec", "cid"
opts.BackupLocation, opts.Image.FS.Path = "/tmp", "/tmp"
@@ -292,9 +306,10 @@ func TestServerApp_WithRemote(t *testing.T) {
// send ping
resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
defer http.DefaultClient.CloseIdleConnections()
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, "pong", string(body))
@@ -322,6 +337,7 @@ func TestServerApp_Failed(t *testing.T) {
_, err = p.ParseArgs([]string{"--store.bolt.path=/tmp", "--backup=/dev/null/not-writable"})
assert.NoError(t, err)
defer os.Remove("/tmp/remark.db")
_, err = opts.newServerApp(context.Background())
assert.EqualError(t, err, "failed to create backup store: can't make directory /dev/null/not-writable: mkdir /dev/null: not a directory")
t.Log(err)
@@ -336,6 +352,7 @@ func TestServerApp_Failed(t *testing.T) {
assert.EqualError(t, err, "invalid remark42 url demo.remark42.com")
t.Log(err)
// wrong store type
opts = ServerCommand{}
opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
@@ -359,6 +376,19 @@ func TestServerApp_Failed(t *testing.T) {
"problem subscribing to channel remark42-cache on address wrong_address: "+
"dial tcp: address wrong_address: missing port in address")
t.Log(err)
// wrong apple private key type
opts = ServerCommand{}
opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
p = flags.NewParser(&opts, flags.Default)
_, err = p.ParseArgs([]string{"--auth.apple.cid=123", "--auth.apple.tid=123",
"--auth.apple.kid=123", "--auth.apple.private-key-filepath=testdata/apple-bad.p8"})
assert.NoError(t, err)
_, err = opts.newServerApp(context.Background())
assert.EqualError(t, err,
"failed to make authenticator: an AppleProvider creating failed: "+
"provided private key is not ECDSA")
t.Log(err)
}
func TestServerApp_Shutdown(t *testing.T) {
@@ -377,7 +407,6 @@ func TestServerApp_Shutdown(t *testing.T) {
}
func TestServerApp_MainSignal(t *testing.T) {
done := make(chan struct{})
go func() {
<-done
@@ -393,6 +422,8 @@ func TestServerApp_MainSignal(t *testing.T) {
port := chooseRandomUnusedPort()
args := []string{"test", "--store.bolt.path=/tmp/xyz", "--backup=/tmp", "--avatar.type=bolt",
"--avatar.bolt.file=/tmp/ava-test.db", "--port=" + strconv.Itoa(port), "--image.fs.path=/tmp"}
defer os.Remove("/tmp/xyz")
defer os.Remove("/tmp/xyz/remark.db")
defer os.Remove("/tmp/ava-test.db")
_, err := p.ParseArgs(args)
require.NoError(t, err)
@@ -410,7 +441,12 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
p := flags.NewParser(&s, flags.Default)
args := []string{
"test",
"--notify.type=email",
"--notify.type=telegram",
"--notify.users=none",
"--notify.admins=none",
"--img-proxy",
"--notify.email.notify_admin",
"--auth.email.host=smtp.example.org",
"--auth.email.port=666",
"--auth.email.tls",
@@ -420,6 +456,7 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
"--auth.email.template=file.tmpl",
"--notify.telegram.token=abcd",
"--notify.telegram.timeout=3m",
"--notify.telegram.api=http://example.org",
}
assert.Empty(t, s.SMTP.Host)
assert.Empty(t, s.SMTP.Port)
@@ -439,9 +476,12 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
{Old: "auth.email.passwd", New: "smtp.password", Version: "1.5"},
{Old: "auth.email.timeout", New: "smtp.timeout", Version: "1.5"},
{Old: "auth.email.template", Version: "1.5"},
{Old: "img-proxy", New: "image-proxy.http2https", Version: "1.5"},
{Old: "notify.email.notify_admin", New: "notify.admins=email", Version: "1.9"},
{Old: "notify.type", New: "notify.(users|admins)", Version: "1.9"},
{Old: "notify.telegram.token", New: "telegram.token", Version: "1.9"},
{Old: "notify.telegram.timeout", New: "telegram.timeout", Version: "1.9"},
{Old: "notify.telegram.api", Version: "1.9"},
},
deprecatedFlags)
assert.Equal(t, "smtp.example.org", s.SMTP.Host)
@@ -452,6 +492,62 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
assert.Equal(t, 15*time.Second, s.SMTP.TimeOut)
}
func TestServerApp_DeprecatedArgsCollisions(t *testing.T) {
s := ServerCommand{}
s.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
p := flags.NewParser(&s, flags.Default)
args := []string{
"test",
"--auth.email.host=smtp-old.example.org",
"--smtp.host=smtp-new.example.org",
"--auth.email.port=666",
"--smtp.port=999",
"--auth.email.user=test_user",
"--smtp.username=new_test_user",
"--auth.email.passwd=test_password",
"--smtp.password=new_test_password",
"--auth.email.timeout=15s",
"--smtp.timeout=20s",
"--notify.type=telegram",
"--notify.users=telegram",
"--notify.admins=none",
"--notify.telegram.token=abcd",
"--telegram.token=dcba",
"--notify.telegram.timeout=3m",
"--telegram.timeout=5m",
}
_, err := p.ParseArgs(args)
require.NoError(t, err)
deprecatedFlagsCollisions := s.findDeprecatedFlagsCollisions()
assert.ElementsMatch(t,
[]DeprecatedFlag{
{Old: "notify.type", New: "notify.(users|admins)", Collision: true},
{Old: "auth.email.host", New: "smtp.host", Collision: true},
{Old: "auth.email.port", New: "smtp.port", Collision: true},
{Old: "auth.email.user", New: "smtp.username", Collision: true},
{Old: "auth.email.passwd", New: "smtp.password", Collision: true},
{Old: "auth.email.timeout", New: "smtp.timeout", Collision: true},
{Old: "notify.telegram.token", New: "telegram.token", Collision: true},
{Old: "notify.telegram.timeout", New: "telegram.timeout", Collision: true},
},
deprecatedFlagsCollisions)
// case which should return nothing
s = ServerCommand{}
s.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
p = flags.NewParser(&s, flags.Default)
args = []string{
"test",
"--auth.email.host=smtp-old.example.org",
"--smtp.host=''",
}
_, err = p.ParseArgs(args)
require.NoError(t, err)
deprecatedFlagsCollisions = s.findDeprecatedFlagsCollisions()
assert.Empty(t, []DeprecatedFlag{}, deprecatedFlagsCollisions)
}
func Test_ACMEEmail(t *testing.T) {
cmd := ServerCommand{}
cmd.SetCommon(CommonOpts{RemarkURL: "https://remark.com:443", SharedSecret: "123456"})
@@ -524,8 +620,10 @@ func TestServerAuthHooks(t *testing.T) {
require.NoError(t, err)
t.Log(tk)
// add comment
client := http.Client{Timeout: 10 * time.Second}
defer client.CloseIdleConnections()
// add comment
req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/p/2018/12/29/podcast-630/", "site": "remark"}}`))
require.NoError(t, err)
@@ -584,21 +682,7 @@ func TestServerAuthHooks(t *testing.T) {
client.CloseIdleConnections()
}
func TestServer_loadEmailTemplate(t *testing.T) {
cmd := ServerCommand{}
cmd.Auth.Email.MsgTemplate = "testdata/email.tmpl"
r, err := cmd.loadEmailTemplate()
assert.NoError(t, err)
assert.Equal(t, "The token is {{.Token}}", r)
cmd.Auth.Email.MsgTemplate = "badpath.tmpl"
r, err = cmd.loadEmailTemplate()
assert.EqualError(t, err, "failed to read file badpath.tmpl: open badpath.tmpl: no such file or directory")
assert.Equal(t, r, "")
}
func TestServerCommand_parseSameSite(t *testing.T) {
tbl := []struct {
inp string
res http.SameSite
@@ -621,7 +705,6 @@ func TestServerCommand_parseSameSite(t *testing.T) {
}
func Test_splitAtCommas(t *testing.T) {
tbl := []struct {
inp string
res []string
@@ -657,6 +740,7 @@ func chooseRandomUnusedPort() (port int) {
func waitForHTTPServerStart(port int) {
// wait for up to 3 seconds for server to start before returning it
client := http.Client{Timeout: time.Second}
defer client.CloseIdleConnections()
for i := 0; i < 300; i++ {
time.Sleep(time.Millisecond * 10)
if resp, err := client.Get(fmt.Sprintf("http://localhost:%d", port)); err == nil {
@@ -686,9 +770,11 @@ func prepServerApp(t *testing.T, fn func(o ServerCommand) ServerCommand) (*serve
p := flags.NewParser(&cmd, flags.Default)
_, err := p.ParseArgs([]string{"--admin-passwd=password", "--site=remark"})
require.NoError(t, err)
cmd.Avatar.FS.Path, cmd.Avatar.Type, cmd.BackupLocation, cmd.Image.FS.Path = "/tmp", "fs", "/tmp", "/tmp"
cmd.Avatar.FS.Path, cmd.Avatar.Type, cmd.BackupLocation, cmd.Image.FS.Path = "/tmp/remark42_test", "fs", "/tmp/remark42_test", "/tmp/remark42_test"
cmd.Store.Bolt.Path = fmt.Sprintf("/tmp/%d", cmd.Port)
cmd.Store.Bolt.Timeout = 10 * time.Second
cmd.Auth.Apple.CID, cmd.Auth.Apple.KID, cmd.Auth.Apple.TID = "cid", "kid", "tid"
cmd.Auth.Apple.PrivateKeyFilePath = "testdata/apple.p8"
cmd.Auth.Github.CSEC, cmd.Auth.Github.CID = "csec", "cid"
cmd.Auth.Google.CSEC, cmd.Auth.Google.CID = "csec", "cid"
cmd.Auth.Facebook.CSEC, cmd.Auth.Facebook.CID = "csec", "cid"
@@ -718,17 +804,23 @@ func prepServerApp(t *testing.T, fn func(o ServerCommand) ServerCommand) (*serve
cmd.emailVerificationTemplatePath = "../../templates/email_confirmation_subscription.html.tmpl"
cmd = fn(cmd)
os.Remove(cmd.Store.Bolt.Path + "/remark.db")
app, ctx, cancel := createAppFromCmd(t, cmd)
return createAppFromCmd(t, cmd)
// cleanup the remark.db file after context is canceled
go func() {
<-ctx.Done()
os.RemoveAll(cmd.Store.Bolt.Path)
os.RemoveAll(cmd.Avatar.FS.Path)
}()
return app, ctx, cancel
}
func createAppFromCmd(t *testing.T, cmd ServerCommand) (*serverApp, context.Context, context.CancelFunc) {
app, err := cmd.newServerApp(context.Background())
require.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
rand.Seed(time.Now().UnixNano())
app, err := cmd.newServerApp(ctx)
require.NoError(t, err)
return app, ctx, cancel
}
+16
View File
@@ -0,0 +1,16 @@
-----BEGIN PRIVATE KEY-----
MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKNwapOQ6rQJHetP
HRlJBIh1OsOsUBiXb3rXXE3xpWAxAha0MH+UPRblOko+5T2JqIb+xKf9Vi3oTM3t
KvffaOPtzKXZauscjq6NGzA3LgeiMy6q19pvkUUOlGYK6+Xfl+B7Xw6+hBMkQuGE
nUS8nkpR5mK4ne7djIyfHFfMu4ptAgMBAAECgYA+s0PPtMq1osG9oi4xoxeAGikf
JB3eMUptP+2DYW7mRibc+ueYKhB9lhcUoKhlQUhL8bUUFVZYakP8xD21thmQqnC4
f63asad0ycteJMLb3r+z26LHuCyOdPg1pyLk3oQ32lVQHBCYathRMcVznxOG16VK
I8BFfstJTaJu0lK/wQJBANYFGusBiZsJQ3utrQMVPpKmloO2++4q1v6ZR4puDQHx
TjLjAIgrkYfwTJBLBRZxec0E7TmuVQ9uJ+wMu/+7zaUCQQDDf2xMnQqYknJoKGq+
oAnyC66UqWC5xAnQS32mlnJ632JXA0pf9pb1SXAYExB1p9Dfqd3VAwQDwBsDDgP6
HD8pAkEA0lscNQZC2TaGtKZk2hXkdcH1SKru/g3vWTkRHxfCAznJUaza1fx0wzdG
GcES1Bdez0tbW4llI5By/skZc2eE3QJAFl6fOskBbGHde3Oce0F+wdZ6XIJhEgCP
iukIcKZoZQzoiMJUoVRrA5gqnmaYDI5uRRl/y57zt6YksR3KcLUIuQJAd242M/WF
6YAZat3q/wEeETeQq1wrooew+8lHl05/Nt0cCpV48RGEhJ83pzBm3mnwHf8lTBJH
x6XroMXsmbnsEw==
-----END PRIVATE KEY-----
+6
View File
@@ -0,0 +1,6 @@
-----BEGIN PRIVATE KEY-----
MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQgGH2MylyZjjRdauTk
xxXW6p8VSHqIeVRRKSJPg1xn6+KgCgYIKoZIzj0DAQehRANCAAS/mNzQ7aBbIBr3
DiHiJGIDEzi6+q3mmyhH6ZWQWFdFei2qgdyM1V6qtRPVq+yHBNSBebbR4noE/IYO
hMdWYrKn
-----END PRIVATE KEY-----
+1
View File
@@ -0,0 +1 @@
This stub page would be replaced by the frontend statically built HTML during the Docker image build.
+21 -11
View File
@@ -8,7 +8,7 @@ import (
"syscall"
log "github.com/go-pkgz/lgr"
"github.com/umputun/go-flags"
"github.com/jessevdk/go-flags"
"github.com/umputun/remark42/backend/app/cmd"
)
@@ -23,7 +23,8 @@ type Opts struct {
CleanupCmd cmd.CleanupCommand `command:"cleanup"`
RemapCmd cmd.RemapCommand `command:"remap"`
RemarkURL string `long:"url" env:"REMARK_URL" required:"true" description:"url to remark"`
RemarkURL string `long:"url" env:"REMARK_URL" required:"true" description:"url to remark"`
// SharedSecret is only used in server command, but defined for all commands for historical reasons
SharedSecret string `long:"secret" env:"SECRET" required:"true" description:"the shared secret key used to sign JWT, should be a random, long, hard-to-guess string"`
Dbg bool `long:"dbg" env:"DEBUG" description:"debug mode"`
@@ -45,13 +46,7 @@ func main() {
SharedSecret: opts.SharedSecret,
Revision: revision,
})
for _, entry := range c.HandleDeprecatedFlags() {
deprecationNote := fmt.Sprintf("[WARN] --%s is deprecated since v%s and will be removed in the future", entry.Old, entry.Version)
if entry.New != "" {
deprecationNote += fmt.Sprintf(", please use --%s instead", entry.New)
}
log.Print(deprecationNote)
}
logDeprecatedParams(c.HandleDeprecatedFlags())
err := c.Execute(args)
if err != nil {
log.Printf("[ERROR] failed with %+v", err)
@@ -62,9 +57,8 @@ func main() {
if _, err := p.Parse(); err != nil {
if flagsErr, ok := err.(*flags.Error); ok && flagsErr.Type == flags.ErrHelp {
os.Exit(0)
} else {
os.Exit(1)
}
os.Exit(1)
}
}
@@ -76,6 +70,22 @@ func setupLog(dbg bool) {
log.Setup(log.Msec, log.LevelBraces)
}
// logs usual and "collision" deprecated parameters
func logDeprecatedParams(params []cmd.DeprecatedFlag) {
for _, entry := range params {
var deprecationNote string
if entry.Collision {
deprecationNote = fmt.Sprintf("[ERROR] deprecated --%s and new --%s options are set to different values, old one is ignored: please remove it", entry.Old, entry.New)
} else {
deprecationNote = fmt.Sprintf("[WARN] --%s is deprecated since v%s and will be removed in the future", entry.Old, entry.Version)
if entry.New != "" {
deprecationNote += fmt.Sprintf(", please use --%s instead", entry.New)
}
}
log.Print(deprecationNote)
}
}
// getDump reads runtime stack and returns as a string
func getDump() string {
maxSize := 5 * 1024 * 1024
+5 -6
View File
@@ -3,7 +3,6 @@ package main
import (
"fmt"
"io"
"io/ioutil"
"math/rand"
"net"
"net/http"
@@ -22,8 +21,7 @@ import (
)
func Test_Main(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "remark42")
dir, err := os.MkdirTemp(os.TempDir(), "remark42")
require.NoError(t, err)
defer os.RemoveAll(dir)
@@ -54,14 +52,14 @@ func Test_Main(t *testing.T) {
resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, "pong", string(body))
}
func TestMain_WithWebhook(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "remark42")
dir, err := os.MkdirTemp(os.TempDir(), "remark42")
require.NoError(t, err)
defer os.RemoveAll(dir)
@@ -118,7 +116,7 @@ func TestMain_WithWebhook(t *testing.T) {
strings.NewReader(`{"text": "env test", "locator":{"url": "https://radio-t.com", "site": "remark"}}`))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 201, resp.StatusCode)
assert.Equal(t, http.StatusCreated, resp.StatusCode)
}
func TestGetDump(t *testing.T) {
@@ -143,6 +141,7 @@ func chooseRandomUnusedPort() (port int) {
func waitForHTTPServerStart(port int) {
// wait for up to 10 seconds for server to start before returning it
client := http.Client{Timeout: time.Second}
defer client.CloseIdleConnections()
for i := 0; i < 100; i++ {
time.Sleep(time.Millisecond * 100)
if resp, err := client.Get(fmt.Sprintf("http://localhost:%d", port)); err == nil {
+6 -7
View File
@@ -10,7 +10,6 @@ import (
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
// AutoBackup struct handles daily backups params for siteID
@@ -48,20 +47,20 @@ func (ab AutoBackup) Do(ctx context.Context) {
func (ab AutoBackup) makeBackup() (string, error) {
log.Printf("[DEBUG] make backup for %s", ab.SiteID)
backupFile := fmt.Sprintf("%s/backup-%s-%s.gz", ab.BackupLocation, ab.SiteID, time.Now().Format("20060102"))
fh, err := os.Create(backupFile)
fh, err := os.Create(backupFile) //nolint:gosec // harmless
if err != nil {
return "", errors.Wrapf(err, "can't create backup file %s", backupFile)
return "", fmt.Errorf("can't create backup file %s: %w", backupFile, err)
}
gz := gzip.NewWriter(fh)
if _, err = ab.Exporter.Export(gz, ab.SiteID); err != nil {
return "", errors.Wrapf(err, "export failed for %s", ab.SiteID)
return "", fmt.Errorf("export failed for %s: %w", ab.SiteID, err)
}
if err = gz.Close(); err != nil {
return "", errors.Wrapf(err, "can't close gz for %s", backupFile)
return "", fmt.Errorf("can't close gz for %s: %w", backupFile, err)
}
if err = fh.Close(); err != nil {
return "", errors.Wrapf(err, "can't close file handler for %s", backupFile)
return "", fmt.Errorf("can't close file handler for %s: %w", backupFile, err)
}
log.Printf("[DEBUG] created backup file %s", backupFile)
return backupFile, nil
@@ -84,7 +83,7 @@ func (ab AutoBackup) removeOldBackupFiles() {
backFiles = append(backFiles, info)
}
}
sort.Slice(backFiles, func(i int, j int) bool { return backFiles[i].Name() < backFiles[j].Name() })
sort.Slice(backFiles, func(i, j int) bool { return backFiles[i].Name() < backFiles[j].Name() })
if len(backFiles) > ab.KeepMax {
for i := 0; i < len(backFiles)-ab.KeepMax; i++ {
+3 -4
View File
@@ -2,10 +2,10 @@ package migrator
import (
"encoding/json"
"fmt"
"io"
"time"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
log "github.com/go-pkgz/lgr"
@@ -68,9 +68,9 @@ func (d *Commento) Import(r io.Reader, siteID string) (size int, err error) {
}
if failed > 0 {
err = errors.Errorf("failed to save %d comments", failed)
err = fmt.Errorf("failed to save %d comments", failed)
if passed == 0 {
err = errors.New("import failed")
err = fmt.Errorf("import failed")
}
}
@@ -85,7 +85,6 @@ func (d *Commento) convert(r io.Reader, siteID string) (ch chan store.Comment) {
decoder := json.NewDecoder(r)
go func() {
var exportedData commentoExport
err := decoder.Decode(&exportedData)
if err != nil {
+2 -1
View File
@@ -7,11 +7,12 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
bolt "go.etcd.io/bbolt"
"github.com/umputun/remark42/backend/app/store"
"github.com/umputun/remark42/backend/app/store/admin"
"github.com/umputun/remark42/backend/app/store/engine"
"github.com/umputun/remark42/backend/app/store/service"
bolt "go.etcd.io/bbolt"
)
func TestCommento_Import(t *testing.T) {
+3 -4
View File
@@ -2,12 +2,12 @@ package migrator
import (
"encoding/xml"
"fmt"
"io"
"strings"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
)
@@ -68,9 +68,9 @@ func (d *Disqus) Import(r io.Reader, siteID string) (size int, err error) {
}
if failed > 0 {
err = errors.Errorf("failed to save %d comments", failed)
err = fmt.Errorf("failed to save %d comments", failed)
if passed == 0 {
err = errors.New("import failed")
err = fmt.Errorf("import failed")
}
}
@@ -82,7 +82,6 @@ func (d *Disqus) Import(r io.Reader, siteID string) (size int, err error) {
// convert disqus stream (xml) from reader and fill channel of comments.
// runs async and closes channel on completion.
func (d *Disqus) convert(r io.Reader, siteID string) (ch chan store.Comment) {
postsMap := map[string]string{} // tid:url
decoder := xml.NewDecoder(r)
commentsCh := make(chan store.Comment)
+2 -2
View File
@@ -1,7 +1,7 @@
package migrator
import (
"errors"
"fmt"
"io"
"strings"
)
@@ -42,7 +42,7 @@ func (u *URLMapper) loadRules(reader io.Reader) error {
row = strings.TrimSpace(row)
urls := strings.Split(row, " ")
if len(urls) != 2 {
return errors.New("bad row " + row)
return fmt.Errorf("bad row %s", row)
}
from, to := strings.TrimSpace(urls[0]), strings.TrimSpace(urls[1])
+4 -4
View File
@@ -4,11 +4,11 @@
package migrator
import (
"fmt"
"io"
"os"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
"github.com/umputun/remark42/backend/app/store/service"
@@ -38,7 +38,7 @@ type MapperMaker func(reader io.Reader) (Mapper, error)
type Store interface {
Create(comment store.Comment) (commentID string, err error)
Find(locator store.Locator, sort string, user store.User) ([]store.Comment, error)
List(siteID string, limit int, skip int) ([]store.PostInfo, error)
List(siteID string, limit, skip int) ([]store.PostInfo, error)
DeleteAll(siteID string) error
Metas(siteID string) (umetas []service.UserMetaData, pmetas []service.PostMetaData, err error)
SetMetas(siteID string, umetas []service.UserMetaData, pmetas []service.PostMetaData) error
@@ -69,12 +69,12 @@ func ImportComments(p ImportParams) (int, error) {
case "native":
importer = &Native{DataStore: p.DataStore}
default:
return 0, errors.Errorf("unsupported import provider %s", p.Provider)
return 0, fmt.Errorf("unsupported import provider %s", p.Provider)
}
fh, err := os.Open(p.InputFile)
if err != nil {
return 0, errors.Wrapf(err, "can't open import file %s", p.InputFile)
return 0, fmt.Errorf("can't open import file %s: %w", p.InputFile, err)
}
defer func() { //nolint:gosec // false positive on defer without error check when it's checked here
+9 -12
View File
@@ -4,12 +4,12 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"sync/atomic"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/syncs"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
"github.com/umputun/remark42/backend/app/store/service"
@@ -35,9 +35,8 @@ type meta struct {
// Export all comments to writer as json strings. Each comment is one string, separated by "\n"
// The final file is a valid json
func (n *Native) Export(w io.Writer, siteID string) (size int, err error) {
if err = n.exportMeta(siteID, w); err != nil {
return 0, errors.Wrapf(err, "failed to export meta for site %s", siteID)
return 0, fmt.Errorf("failed to export meta for site %s: %w", siteID, err)
}
topics, err := n.DataStore.List(siteID, 0, 0)
@@ -55,16 +54,15 @@ func (n *Native) Export(w io.Writer, siteID string) (size int, err error) {
}
for _, comment := range comments {
buf := &bytes.Buffer{}
enc := json.NewEncoder(buf)
enc.SetEscapeHTML(false)
if err = enc.Encode(comment); err != nil {
return commentsCount, errors.Wrapf(err, "can't marshal %v", comments)
return commentsCount, fmt.Errorf("can't marshal %v: %w", comments, err)
}
if _, err = w.Write(buf.Bytes()); err != nil {
return commentsCount, errors.Wrap(err, "can't write comment data")
return commentsCount, fmt.Errorf("can't write comment data: %w", err)
}
commentsCount++
}
@@ -78,11 +76,11 @@ func (n *Native) exportMeta(siteID string, w io.Writer) (err error) {
m := meta{Version: nativeVersion}
m.Users, m.Posts, err = n.DataStore.Metas(siteID)
if err != nil {
return errors.Wrap(err, "can't get meta")
return fmt.Errorf("can't get meta: %w", err)
}
if err = json.NewEncoder(w).Encode(m); err != nil {
return errors.Wrap(err, "can't encode meta")
return fmt.Errorf("can't encode meta: %w", err)
}
return nil
}
@@ -133,11 +131,11 @@ func (n *Native) Import(reader io.Reader, siteID string) (size int, err error) {
m := meta{}
dec := json.NewDecoder(reader)
if err = dec.Decode(&m); err != nil {
return 0, errors.Wrapf(err, "failed to import meta for site %s", siteID)
return 0, fmt.Errorf("failed to import meta for site %s: %w", siteID, err)
}
if m.Version != nativeVersion && m.Version != 0 { // this version allows back compatibility with 0 version
return 0, errors.Errorf("unexpected import file version %d", m.Version)
return 0, fmt.Errorf("unexpected import file version %d", m.Version)
}
if e := n.DataStore.DeleteAll(siteID); e != nil {
@@ -180,13 +178,12 @@ func (n *Native) Import(reader io.Reader, siteID string) (size int, err error) {
log.Printf("[DEBUG] imported %d comments", num)
}
})
}
grp.Wait()
if failed > 0 {
return int(comments), errors.Errorf("failed to save %d comments", failed)
return int(comments), fmt.Errorf("failed to save %d comments", failed)
}
log.Printf("[INFO] imported %d comments from %d records", comments, total)
+1 -3
View File
@@ -153,7 +153,6 @@ func TestNative_ImportWrongVersion(t *testing.T) {
size, err := r.Import(strings.NewReader(inp), "radio-t")
assert.EqualError(t, err, "unexpected import file version 2")
assert.Equal(t, 0, size)
}
func TestNative_ImportManyWithError(t *testing.T) {
b, teardown := prep(t) // write 2 comments
@@ -164,7 +163,7 @@ func TestNative_ImportManyWithError(t *testing.T) {
buf := &bytes.Buffer{}
buf.WriteString(`{"version":1, "users":[], "posts":[]}` + "\n")
for i := 0; i < 100; i++ {
buf.WriteString(fmt.Sprintf(goodRec, i))
fmt.Fprintf(buf, goodRec, i)
}
buf.WriteString("{}\n")
buf.WriteString("{}\n")
@@ -181,7 +180,6 @@ func TestNative_ImportManyWithError(t *testing.T) {
// makes new boltdb, put two records
func prep(t *testing.T) (ds *service.DataStore, teardown func()) {
testDB := fmt.Sprintf("/tmp/migrator-%d.db", rand.Intn(999999999))
boltStore, err := engine.NewBoltDB(bolt.Options{}, engine.BoltSite{SiteID: "radio-t", FileName: testDB})
+3 -5
View File
@@ -2,12 +2,12 @@ package migrator
import (
"encoding/xml"
"fmt"
"html"
"io"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
)
@@ -60,7 +60,6 @@ func (w *WordPress) Convert(text string) string {
// Import comments from WP and save to store
func (w *WordPress) Import(r io.Reader, siteID string) (size int, err error) {
if e := w.DataStore.DeleteAll(siteID); e != nil {
return 0, e
}
@@ -76,9 +75,9 @@ func (w *WordPress) Import(r io.Reader, siteID string) (size int, err error) {
}
if failed > 0 {
err = errors.Errorf("failed to save %d comments", failed)
err = fmt.Errorf("failed to save %d comments", failed)
if passed == 0 {
err = errors.New("import failed")
err = fmt.Errorf("import failed")
}
}
@@ -88,7 +87,6 @@ func (w *WordPress) Import(r io.Reader, siteID string) (size int, err error) {
}
func (w *WordPress) convert(r io.Reader, siteID string) chan store.Comment {
decoder := xml.NewDecoder(r)
commentsCh := make(chan store.Comment)
+59 -205
View File
@@ -3,20 +3,15 @@ package notify
import (
"bytes"
"context"
"crypto/tls"
"fmt"
"io"
"mime"
"mime/quotedprintable"
"net"
"net/smtp"
"net/url"
"text/template"
"time"
log "github.com/go-pkgz/lgr"
ntf "github.com/go-pkgz/notify"
"github.com/go-pkgz/repeater"
"github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/templates"
)
@@ -34,50 +29,15 @@ type EmailParams struct {
TokenGenFn func(userID, email, site string) (string, error) // Unsubscribe token generation function
}
// SMTPParams contain settings for smtp server connection
type SMTPParams struct {
Host string // SMTP host
Port int // SMTP port
TLS bool // TLS auth
Username string // user name
Password string // password
TimeOut time.Duration // TCP connection timeout
}
// Email implements notify.Destination for email
type Email struct {
EmailParams
SMTPParams
*ntf.Email
smtp smtpClientCreator
EmailParams
msgTmpl *template.Template // parsed request message template
verifyTmpl *template.Template // parsed verification message template
}
// default email client implementation
type emailClient struct{ smtpClientCreator }
// smtpClient interface defines subset of net/smtp used by email client
type smtpClient interface {
Mail(string) error
Auth(smtp.Auth) error
Rcpt(string) error
Data() (io.WriteCloser, error)
Quit() error
Close() error
}
// smtpClientCreator interface defines function for creating new smtpClients
type smtpClientCreator interface {
Create(SMTPParams) (smtpClient, error)
}
type emailMessage struct {
from string
to string
message string
}
// msgTmplData store data for message from request template execution
type msgTmplData struct {
UserName string
@@ -113,15 +73,14 @@ const (
)
// NewEmail makes new Email object, returns error in case of e.MsgTemplate or e.VerificationTemplate parsing error
func NewEmail(emailParams EmailParams, smtpParams SMTPParams) (*Email, error) {
func NewEmail(emailParams EmailParams, smtpParams ntf.SMTPParams) (*Email, error) {
// set up Email emailParams
res := Email{EmailParams: emailParams}
res.smtp = &emailClient{}
res.SMTPParams = smtpParams
if res.TimeOut <= 0 {
res.TimeOut = defaultEmailTimeout
if smtpParams.TimeOut <= 0 {
smtpParams.TimeOut = defaultEmailTimeout
}
res := Email{Email: ntf.NewEmail(smtpParams), EmailParams: emailParams}
if res.VerificationSubject == "" {
res.VerificationSubject = defaultVerificationSubject
}
@@ -129,7 +88,7 @@ func NewEmail(emailParams EmailParams, smtpParams SMTPParams) (*Email, error) {
// initialize templates
err := res.setTemplates()
if err != nil {
return nil, errors.Wrap(err, "can't set templates")
return nil, fmt.Errorf("can't set templates: %w", err)
}
log.Printf("[DEBUG] Create new email notifier for server %s with user %s, timeout=%s",
@@ -141,7 +100,6 @@ func NewEmail(emailParams EmailParams, smtpParams SMTPParams) (*Email, error) {
func (e *Email) setTemplates() error {
var err error
var msgTmplFile, verifyTmplFile []byte
fs := templates.NewFS()
if e.VerificationTemplatePath == "" {
e.VerificationTemplatePath = defaultEmailVerificationTemplatePath
@@ -151,17 +109,17 @@ func (e *Email) setTemplates() error {
e.MsgTemplatePath = defaultEmailTemplatePath
}
if msgTmplFile, err = fs.ReadFile(e.MsgTemplatePath); err != nil {
return errors.Wrapf(err, "can't read message template")
if msgTmplFile, err = templates.Read(e.MsgTemplatePath); err != nil {
return fmt.Errorf("can't read message template: %w", err)
}
if verifyTmplFile, err = fs.ReadFile(e.VerificationTemplatePath); err != nil {
return errors.Wrapf(err, "can't read verification template")
if verifyTmplFile, err = templates.Read(e.VerificationTemplatePath); err != nil {
return fmt.Errorf("can't read verification template: %w", err)
}
if e.msgTmpl, err = template.New("msgTmpl").Parse(string(msgTmplFile)); err != nil {
return errors.Wrapf(err, "can't parse message template")
return fmt.Errorf("can't parse message template: %w", err)
}
if e.verifyTmpl, err = template.New("verifyTmpl").Parse(string(verifyTmplFile)); err != nil {
return errors.Wrapf(err, "can't parse verification template")
return fmt.Errorf("can't parse verification template: %w", err)
}
return nil
@@ -173,7 +131,7 @@ func (e *Email) setTemplates() error {
func (e *Email) Send(ctx context.Context, req Request) error {
select {
case <-ctx.Done():
return errors.Errorf("sending email messages about comment %q aborted due to canceled context", req.Comment.ID)
return fmt.Errorf("sending email messages about comment %q aborted due to canceled context", req.Comment.ID)
default:
}
@@ -181,12 +139,16 @@ func (e *Email) Send(ctx context.Context, req Request) error {
for _, email := range req.Emails {
err := e.buildAndSendMessage(ctx, req, email, false)
result = multierror.Append(errors.Wrapf(err, "problem sending user email notification to %q", email))
if err != nil {
result = multierror.Append(fmt.Errorf("problem sending user email notification to %q: %w", email, err))
}
}
for _, email := range e.AdminEmails {
err := e.buildAndSendMessage(ctx, req, email, true)
result = multierror.Append(errors.Wrapf(err, "problem sending admin email notification to %q", email))
if err != nil {
result = multierror.Append(fmt.Errorf("problem sending admin email notification to %q: %w", email, err))
}
}
return result.ErrorOrNil()
@@ -202,7 +164,16 @@ func (e *Email) buildAndSendMessage(ctx context.Context, req Request, email stri
return repeater.NewDefault(5, time.Millisecond*250).Do(
ctx,
func() error {
return e.sendMessage(emailMessage{from: e.From, to: email, message: msg})
return e.Email.Send(
ctx,
fmt.Sprintf("mailto:%s?from=%s&unsubscribeLink=%s&subject=%s",
email,
e.From,
url.QueryEscape(msg.unsubscribeLink),
url.QueryEscape(msg.subject),
),
msg.body,
)
})
}
@@ -215,7 +186,7 @@ func (e *Email) SendVerification(ctx context.Context, req VerificationRequest) e
}
select {
case <-ctx.Done():
return errors.Errorf("sending message to %q aborted due to canceled context", req.User)
return fmt.Errorf("sending message to %q aborted due to canceled context", req.User)
default:
}
@@ -228,13 +199,20 @@ func (e *Email) SendVerification(ctx context.Context, req VerificationRequest) e
return repeater.NewDefault(5, time.Millisecond*250).Do(
ctx,
func() error {
return e.sendMessage(emailMessage{from: e.From, to: req.Email, message: msg})
return e.Email.Send(
ctx,
fmt.Sprintf("mailto:%s?from=%s&subject=%s",
req.Email,
e.From,
url.QueryEscape(e.VerificationSubject),
),
msg,
)
})
}
// buildVerificationMessage generates verification email message based on given input
func (e *Email) buildVerificationMessage(user, email, token, site string) (string, error) {
subject := e.VerificationSubject
msg := bytes.Buffer{}
err := e.verifyTmpl.Execute(&msg, verifyTmplData{
User: user,
@@ -244,13 +222,19 @@ func (e *Email) buildVerificationMessage(user, email, token, site string) (strin
SubscribeURL: e.SubscribeURL,
})
if err != nil {
return "", errors.Wrapf(err, "error executing template to build verification message")
return "", fmt.Errorf("error executing template to build verification message: %w", err)
}
return e.buildMessage(subject, msg.String(), email, "text/html", "")
return msg.String(), nil
}
type commentMessage struct {
subject string
body string
unsubscribeLink string
}
// buildMessageFromRequest generates email message based on Request using e.MsgTemplate
func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool) (string, error) {
func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool) (commentMessage, error) {
subject := "New reply to your comment"
if forAdmin {
subject = "New comment to your site"
@@ -261,7 +245,7 @@ func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool
token, err := e.TokenGenFn(req.parent.User.ID, email, req.Comment.Locator.SiteID)
if err != nil {
return "", errors.Wrapf(err, "error creating token for unsubscribe link")
return commentMessage{}, fmt.Errorf("error creating token for unsubscribe link: %w", err)
}
unsubscribeLink := e.UnsubscribeURL + "?site=" + req.Comment.Locator.SiteID + "&tkn=" + token
if forAdmin {
@@ -291,141 +275,11 @@ func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool
}
err = e.msgTmpl.Execute(&msg, tmplData)
if err != nil {
return "", errors.Wrapf(err, "error executing template to build comment reply message")
return commentMessage{}, fmt.Errorf("error executing template to build comment reply message: %w", err)
}
return e.buildMessage(subject, msg.String(), email, "text/html", unsubscribeLink)
}
// buildMessage generates email message to send using net/smtp.Data()
func (e *Email) buildMessage(subject, body, to, contentType, unsubscribeLink string) (message string, err error) {
addHeader := func(msg, h, v string) string {
msg += fmt.Sprintf("%s: %s\n", h, v)
return msg
}
message = addHeader(message, "From", e.From)
message = addHeader(message, "To", to)
message = addHeader(message, "Subject", mime.BEncoding.Encode("utf-8", subject))
message = addHeader(message, "Content-Transfer-Encoding", "quoted-printable")
if contentType != "" {
message = addHeader(message, "MIME-version", "1.0")
message = addHeader(message, "Content-Type", contentType+`; charset="UTF-8"`)
}
if unsubscribeLink != "" {
// https://support.google.com/mail/answer/81126 -> "Include option to unsubscribe"
message = addHeader(message, "List-Unsubscribe-Post", "List-Unsubscribe=One-Click")
message = addHeader(message, "List-Unsubscribe", "<"+unsubscribeLink+">")
}
message = addHeader(message, "Date", time.Now().Format(time.RFC1123Z))
buff := &bytes.Buffer{}
qp := quotedprintable.NewWriter(buff)
if _, err := qp.Write([]byte(body)); err != nil {
return "", err
}
// flush now, must NOT use defer, for small body, defer may cause buff.String() got empty body
if err := qp.Close(); err != nil {
return "", fmt.Errorf("quotedprintable Write failed: %w", err)
}
m := buff.String()
message += "\n" + m
return message, nil
}
// sendMessage sends messages to server in a new connection, closing the connection after finishing.
// Thread safe.
func (e *Email) sendMessage(m emailMessage) error {
if e.smtp == nil {
return errors.New("sendMessage called without client set")
}
client, err := e.smtp.Create(e.SMTPParams)
if err != nil {
return errors.Wrap(err, "failed to make smtp Create")
}
defer func() {
if err = client.Quit(); err != nil {
log.Printf("[WARN] failed to send quit command to %s:%d, %v", e.Host, e.Port, err)
if err = client.Close(); err != nil {
log.Printf("[WARN] can't close smtp connection, %v", err)
}
}
}()
if err = client.Mail(m.from); err != nil {
return errors.Wrapf(err, "bad from address %q", m.from)
}
if err = client.Rcpt(m.to); err != nil {
return errors.Wrapf(err, "bad to address %q", m.to)
}
writer, err := client.Data()
if err != nil {
return errors.Wrap(err, "can't make email writer")
}
defer func() {
if err = writer.Close(); err != nil {
log.Printf("[WARN] can't close smtp body writer, %v", err)
}
}()
buf := bytes.NewBufferString(m.message)
if _, err = buf.WriteTo(writer); err != nil {
return errors.Wrapf(err, "failed to send email body to %q", m.to)
}
return nil
}
// String representation of Email object
func (e *Email) String() string {
return fmt.Sprintf("email: from %q with username '%s' at server %s:%d", e.From, e.Username, e.Host, e.Port)
}
// Create establish SMTP connection with server using credentials in smtpClientWithCreator.SMTPParams
// and returns pointer to it. Thread safe.
func (s *emailClient) Create(params SMTPParams) (smtpClient, error) {
authenticate := func(c *smtp.Client) error {
if params.Username == "" || params.Password == "" {
return nil
}
auth := smtp.PlainAuth("", params.Username, params.Password, params.Host)
if err := c.Auth(auth); err != nil {
return errors.Wrapf(err, "failed to auth to smtp %s:%d", params.Host, params.Port)
}
return nil
}
var c *smtp.Client
srvAddress := fmt.Sprintf("%s:%d", params.Host, params.Port)
if params.TLS {
tlsConf := &tls.Config{
InsecureSkipVerify: false,
ServerName: params.Host,
MinVersion: tls.VersionTLS12,
}
conn, err := tls.Dial("tcp", srvAddress, tlsConf)
if err != nil {
return nil, errors.Wrapf(err, "failed to dial smtp tls to %s", srvAddress)
}
if c, err = smtp.NewClient(conn, params.Host); err != nil {
return nil, errors.Wrapf(err, "failed to make smtp client for %s", srvAddress)
}
return c, authenticate(c)
}
conn, err := net.DialTimeout("tcp", srvAddress, params.TimeOut)
if err != nil {
return nil, errors.Wrapf(err, "timeout connecting to %s", srvAddress)
}
c, err = smtp.NewClient(conn, params.Host)
if err != nil {
return nil, errors.Wrap(err, "failed to dial")
}
return c, authenticate(c)
return commentMessage{
subject: subject,
body: msg.String(),
unsubscribeLink: unsubscribeLink,
}, err
}
+53 -254
View File
@@ -1,16 +1,12 @@
package notify
import (
"bytes"
"context"
"errors"
"io"
"net/smtp"
"sync"
"fmt"
"testing"
"text/template"
"time"
ntf "github.com/go-pkgz/notify"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -23,13 +19,13 @@ func TestEmailNew(t *testing.T) {
VerificationTemplatePath: "testdata/verification.html.tmpl",
MsgTemplatePath: "testdata/msg.html.tmpl",
}
smtpParams := SMTPParams{
smtpParams := ntf.SMTPParams{
Host: "test@host",
Port: 1000,
TLS: true,
StartTLS: true,
Username: "test@username",
Password: "test@password",
TimeOut: time.Second,
}
email, err := NewEmail(emailParams, smtpParams)
@@ -49,6 +45,8 @@ func TestEmailNew(t *testing.T) {
assert.Equal(t, smtpParams.Password, email.Password, "emailParams.Password unchanged after creation")
assert.Equal(t, smtpParams.Port, email.Port, "emailParams.Port unchanged after creation")
assert.Equal(t, smtpParams.TLS, email.TLS, "emailParams.TLS unchanged after creation")
assert.Equal(t, smtpParams.StartTLS, email.StartTLS, "emailParams.TLS unchanged after creation")
assert.Equal(t, "email: with username 'test@username' at server test@host:1000 with TLS with StartTLS", email.String())
}
func Test_initTemplatesErr(t *testing.T) {
@@ -59,18 +57,16 @@ func Test_initTemplatesErr(t *testing.T) {
}{
{
name: "with wrong path to verification template",
errText: "can't read verification template: open notfount.tmpl: no such file or directory",
errText: "notfound.tmpl: file does not exist",
emailParams: EmailParams{
VerificationTemplatePath: "notfount.tmpl",
MsgTemplatePath: "testdata/msg.html.tmpl",
VerificationTemplatePath: "notfound.tmpl",
},
},
{
name: "with wrong path to message template",
errText: "can't read message template: open notfount.tmpl: no such file or directory",
errText: "notfound.tmpl: file does not exist",
emailParams: EmailParams{
VerificationTemplatePath: "testdata/verification.html.tmpl",
MsgTemplatePath: "notfount.tmpl",
MsgTemplatePath: "notfound.tmpl",
},
},
{
@@ -94,9 +90,9 @@ func Test_initTemplatesErr(t *testing.T) {
for _, d := range testSet {
d := d
t.Run(d.name, func(t *testing.T) {
e := Email{EmailParams: d.emailParams}
err := e.setTemplates()
e, err := NewEmail(d.emailParams, ntf.SMTPParams{})
require.Error(t, err)
require.Nil(t, e)
assert.Contains(t, err.Error(), d.errText)
})
}
@@ -125,7 +121,6 @@ func TestEmailSendErrors(t *testing.T) {
assert.EqualError(t, e.Send(ctx, Request{Comment: store.Comment{ID: "999"}, parent: store.Comment{User: store.User{ID: "test"}}, Emails: []string{"bad@example.org"}}),
"sending email messages about comment \"999\" aborted due to canceled context")
e.smtp = &fakeTestSMTP{}
assert.EqualError(t, e.Send(context.Background(), Request{Comment: store.Comment{ID: "999"}, parent: store.Comment{User: store.User{ID: "error"}}, Emails: []string{"bad@example.org"}}),
"1 error occurred:\n\t* problem sending user email notification to \"bad@example.org\":"+
" error creating token for unsubscribe link: token generation error\n\n")
@@ -135,7 +130,7 @@ func TestEmailSend_ExitConditions(t *testing.T) {
email, err := NewEmail(EmailParams{
VerificationTemplatePath: "testdata/verification.html.tmpl",
MsgTemplatePath: "testdata/msg.html.tmpl",
}, SMTPParams{})
}, ntf.SMTPParams{})
assert.NoError(t, err)
assert.NotNil(t, email, "expecting email returned")
// prevent triggering e.autoFlush creation
@@ -144,67 +139,14 @@ func TestEmailSend_ExitConditions(t *testing.T) {
"Message without Emails and AdminEmails is not sent and returns nil")
}
func TestEmailSendClientError(t *testing.T) {
var testSet = []struct {
name string
smtp *fakeTestSMTP
err string
}{
{name: "failed to verify receiver", smtp: &fakeTestSMTP{fail: map[string]bool{"mail": true}},
err: "bad from address \"\": failed to verify sender"},
{name: "failed to verify sender", smtp: &fakeTestSMTP{fail: map[string]bool{"rcpt": true}},
err: "bad to address \"\": failed to verify receiver"},
{name: "failed to close connection", smtp: &fakeTestSMTP{fail: map[string]bool{"quit": true, "close": true}}},
{name: "failed to make email writer", smtp: &fakeTestSMTP{fail: map[string]bool{"data": true}},
err: "can't make email writer: failed to send"},
}
for _, d := range testSet {
d := d
t.Run(d.name, func(t *testing.T) {
e := Email{smtp: d.smtp}
if d.err != "" {
assert.EqualError(t, e.sendMessage(emailMessage{}), d.err,
"expected error for e.sendMessage")
} else {
assert.NoError(t, e.sendMessage(emailMessage{}),
"expected no error for e.sendMessage")
}
})
}
e := Email{}
e.smtp = nil
assert.Error(t, e.sendMessage(emailMessage{}),
"nil e.smtp should return error")
e.smtp = &fakeTestSMTP{}
assert.NoError(t, e.sendMessage(emailMessage{}), "",
"no error expected for e.sendMessage in normal flow")
e.smtp = &fakeTestSMTP{fail: map[string]bool{"quit": true}}
assert.NoError(t, e.sendMessage(emailMessage{}), "",
"no error expected for e.sendMessage with failed smtpClient.Quit but successful smtpClient.Close")
e.smtp = &fakeTestSMTP{fail: map[string]bool{"create": true}}
assert.EqualError(t, e.sendMessage(emailMessage{}), "failed to make smtp Create: failed to create client",
"e.send called without smtpClient set returns error")
}
func TestEmail_DefaultTemplates(t *testing.T) {
email, err := NewEmail(EmailParams{}, SMTPParams{})
assert.Error(t, err)
assert.Nil(t, email)
email, err = NewEmail(EmailParams{VerificationTemplatePath: "testdata/verification.html.tmpl"}, SMTPParams{})
assert.Error(t, err)
assert.Nil(t, email)
}
func TestEmail_Send(t *testing.T) {
email, err := NewEmail(EmailParams{
From: "from@example.org",
VerificationTemplatePath: "testdata/verification.html.tmpl",
MsgTemplatePath: "testdata/msg.html.tmpl",
}, SMTPParams{})
}, ntf.SMTPParams{})
assert.NoError(t, err)
assert.NotNil(t, email)
fakeSMTP := fakeTestSMTP{}
email.smtp = &fakeSMTP
email.TokenGenFn = TokenGenFn
email.UnsubscribeURL = "https://remark42.com/api/v1/email/unsubscribe"
req := Request{
@@ -212,22 +154,21 @@ func TestEmail_Send(t *testing.T) {
parent: store.Comment{ID: "1", User: store.User{ID: "999", Name: "parent_user"}},
Emails: []string{"test@example.org"},
}
assert.NoError(t, email.Send(context.TODO(), req))
assert.Equal(t, "from@example.org", fakeSMTP.readMail())
assert.Equal(t, 1, fakeSMTP.readQuitCount())
assert.Equal(t, "test@example.org", fakeSMTP.readRcpt())
assert.Contains(t, email.Send(context.Background(), req).Error(), "problem sending user email notification to \"test@example.org\"")
// test buildMessageFromRequest separately for message text
res, err := email.buildMessageFromRequest(req, req.Emails[0], false)
msg, err := email.buildMessageFromRequest(req, req.Emails[0], false)
assert.NoError(t, err)
assert.Contains(t, res, `From: from@example.org
To: test@example.org
Subject: New reply to your comment for "test_title"
Content-Transfer-Encoding: quoted-printable
MIME-version: 1.0
Content-Type: text/html; charset="UTF-8"
List-Unsubscribe-Post: List-Unsubscribe=One-Click
List-Unsubscribe: <https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token>
Date: `)
assert.Equal(t, `
New reply from test_user on your comment to «test_title»
User: test_user
01.01.0001 at 00:00
Comment:
test@example.org for parent_user
Unsubscribe link: https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token
`, msg.body)
assert.Equal(t, "https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token", msg.unsubscribeLink)
assert.Equal(t, `New reply to your comment for "test_title"`, msg.subject)
// send email to both user and admin, without parent set
email.AdminEmails = []string{"admin@example.org"}
@@ -235,51 +176,19 @@ Date: `)
Comment: store.Comment{ID: "999", User: store.User{ID: "1", Name: "test_user"}, PostTitle: "test_title"},
Emails: []string{"test@example.org"},
}
assert.NoError(t, email.Send(context.TODO(), req))
assert.Equal(t, "from@example.org", fakeSMTP.readMail())
assert.Equal(t, 3, fakeSMTP.readQuitCount(), "plus two emails: one for user and one for admin")
assert.Equal(t, "admin@example.org", fakeSMTP.readRcpt())
res, err = email.buildMessageFromRequest(req, email.AdminEmails[0], true)
assert.Error(t, email.Send(context.Background(), req))
msg, err = email.buildMessageFromRequest(req, email.AdminEmails[0], true)
assert.NoError(t, err)
assert.Contains(t, res, `From: from@example.org
To: admin@example.org
Subject: New comment to your site for "test_title"
Content-Transfer-Encoding: quoted-printable
MIME-version: 1.0
Content-Type: text/html; charset="UTF-8"
Date: `)
}
assert.Equal(t, `
New comment from test_user on your site to «test_title»
func TestEmail_SendWithUnicodeInSubject(t *testing.T) {
email, err := NewEmail(EmailParams{
From: "from@example.org",
VerificationTemplatePath: "testdata/verification.html.tmpl",
MsgTemplatePath: "testdata/msg.html.tmpl",
}, SMTPParams{})
assert.NoError(t, err)
assert.NotNil(t, email)
fakeSMTP := fakeTestSMTP{}
email.smtp = &fakeSMTP
email.TokenGenFn = TokenGenFn
email.UnsubscribeURL = "https://remark42.com/api/v1/email/unsubscribe"
req := Request{
Comment: store.Comment{ID: "999", User: store.User{ID: "1", Name: "test_user"}, ParentID: "1", PostTitle: "Привет"},
parent: store.Comment{ID: "1", User: store.User{ID: "999", Name: "parent_user"}},
Emails: []string{"test@example.org"},
}
// test buildMessageFromRequest separately for message text
res, err := email.buildMessageFromRequest(req, req.Emails[0], false)
assert.NoError(t, err)
// `=?utf-8?b?TmV3IHJlcGx5IHRvIHlvdXIgY29tbWVudCBmb3IgItCf0YDQuNCy0LXRgiI=?=` -> `New reply to your comment for "Привет"` in base64 + required prefix and suffix
assert.Contains(t, res, `From: from@example.org
To: test@example.org
Subject: =?utf-8?b?TmV3IHJlcGx5IHRvIHlvdXIgY29tbWVudCBmb3IgItCf0YDQuNCy0LXRgiI=?=
Content-Transfer-Encoding: quoted-printable
MIME-version: 1.0
Content-Type: text/html; charset="UTF-8"
List-Unsubscribe-Post: List-Unsubscribe=One-Click
List-Unsubscribe: <https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token>
Date: `)
User: test_user
01.01.0001 at 00:00
Comment:
admin@example.org
`, msg.body)
assert.Equal(t, `New comment to your site for "test_title"`, msg.subject)
assert.Empty(t, msg.unsubscribeLink)
}
func TestEmail_SendVerification(t *testing.T) {
@@ -287,11 +196,9 @@ func TestEmail_SendVerification(t *testing.T) {
From: "from@example.org",
VerificationTemplatePath: "testdata/verification.html.tmpl",
MsgTemplatePath: "testdata/msg.html.tmpl",
}, SMTPParams{})
}, ntf.SMTPParams{})
assert.NoError(t, err)
assert.NotNil(t, email)
fakeSMTP := fakeTestSMTP{}
email.smtp = &fakeSMTP
email.TokenGenFn = TokenGenFn
// proper VerificationRequest without email
req := VerificationRequest{
@@ -299,149 +206,41 @@ func TestEmail_SendVerification(t *testing.T) {
User: "test_username",
Token: "secret_",
}
assert.NoError(t, email.SendVerification(context.TODO(), req))
assert.Equal(t, "", fakeSMTP.readMail())
assert.Equal(t, 0, fakeSMTP.readQuitCount())
assert.Equal(t, "", fakeSMTP.readRcpt())
assert.NoError(t, email.SendVerification(context.Background(), req))
// proper VerificationRequest with email
req.Email = "test@example.org"
assert.NoError(t, email.SendVerification(context.TODO(), req))
assert.Equal(t, "from@example.org", fakeSMTP.readMail())
assert.Equal(t, 1, fakeSMTP.readQuitCount())
assert.Equal(t, "test@example.org", fakeSMTP.readRcpt())
assert.Error(t, email.SendVerification(context.Background(), req), "failed to make smtp client")
// VerificationRequest with canceled context
ctx, cancel := context.WithCancel(context.TODO())
ctx, cancel := context.WithCancel(context.Background())
cancel()
assert.EqualError(t, email.SendVerification(ctx, req), "sending message to \"test_username\" aborted due to canceled context")
// test buildVerificationMessage separately for message text
res, err := email.buildVerificationMessage(req.User, req.Email, req.Token, req.SiteID)
assert.NoError(t, err)
assert.Contains(t, res, `From: from@example.org
To: test@example.org
Subject: Email verification
Content-Transfer-Encoding: quoted-printable
MIME-version: 1.0
Content-Type: text/html; charset="UTF-8"
Date: `)
assert.Equal(t, res, `Confirmation for test_username on site remark
Token:secret_
Sent to test@example.org
`)
assert.Contains(t, res, `secret_`)
assert.NotContains(t, res, `https://example.org/`)
email.SubscribeURL = "https://example.org/subscribe.html?token="
res, err = email.buildVerificationMessage(req.User, req.Email, req.Token, req.SiteID)
assert.NoError(t, err)
assert.Contains(t, res, `From: from@example.org
To: test@example.org
Subject: Email verification
Content-Transfer-Encoding: quoted-printable
MIME-version: 1.0
Content-Type: text/html; charset="UTF-8"
Date: `)
assert.Contains(t, res, `https://example.org/subscribe.html?token=3Dsecret_`)
}
assert.Equal(t, res, `Confirmation for test_username on site remark
Subscribe url: https://example.org/subscribe.html?token=secret_
Token:secret_
Sent to test@example.org
func Test_emailClient_Create(t *testing.T) {
creator := emailClient{}
client, err := creator.Create(SMTPParams{})
assert.Error(t, err, "absence of address to connect results in error")
assert.Nil(t, client, "no client returned in case of error")
}
type fakeTestSMTP struct {
fail map[string]bool
buff bytes.Buffer
mail, rcpt string
auth bool
close bool
quitCount int
lock sync.RWMutex
}
func (f *fakeTestSMTP) Create(SMTPParams) (smtpClient, error) {
if f.fail["create"] {
return nil, errors.New("failed to create client")
}
return f, nil
}
func (f *fakeTestSMTP) Auth(smtp.Auth) error { f.auth = true; return nil }
func (f *fakeTestSMTP) Mail(m string) error {
f.lock.Lock()
f.mail = m
f.lock.Unlock()
if f.fail["mail"] {
return errors.New("failed to verify sender")
}
return nil
}
func (f *fakeTestSMTP) Rcpt(r string) error {
f.lock.Lock()
f.rcpt = r
f.lock.Unlock()
if f.fail["rcpt"] {
return errors.New("failed to verify receiver")
}
return nil
}
func (f *fakeTestSMTP) Quit() error {
f.lock.Lock()
f.quitCount++
f.lock.Unlock()
if f.fail["quit"] {
return errors.New("failed to quit")
}
return nil
}
func (f *fakeTestSMTP) Close() error {
f.close = true
if f.fail["close"] {
return errors.New("failed to close")
}
return nil
}
func (f *fakeTestSMTP) Data() (io.WriteCloser, error) {
if f.fail["data"] {
return nil, errors.New("failed to send")
}
return nopCloser{&f.buff}, nil
}
func (f *fakeTestSMTP) readRcpt() string {
f.lock.RLock()
defer f.lock.RUnlock()
return f.rcpt
}
func (f *fakeTestSMTP) readMail() string {
f.lock.RLock()
defer f.lock.RUnlock()
return f.mail
}
func (f *fakeTestSMTP) readQuitCount() int {
f.lock.RLock()
defer f.lock.RUnlock()
return f.quitCount
`)
}
func TokenGenFn(user, _, _ string) (string, error) {
if user == "error" {
return "", errors.New("token generation error")
return "", fmt.Errorf("token generation error")
}
return "token", nil
}
type nopCloser struct {
io.Writer
}
func (nopCloser) Close() error {
return nil
}
+8 -2
View File
@@ -34,8 +34,8 @@ type Destination interface {
// Store defines the minimal interface accessing stored comments used by notifier
type Store interface {
Get(locator store.Locator, id string, user store.User) (store.Comment, error)
GetUserEmail(siteID string, userID string) (string, error)
GetUserTelegram(siteID string, userID string) (string, error)
GetUserEmail(siteID, userID string) (string, error)
GetUserTelegram(siteID, userID string) (string, error)
}
// used for email and telegram retrieval from user details
@@ -143,6 +143,12 @@ func (s *Service) SubmitVerification(req VerificationRequest) {
// Close queue channel and wait for completion
func (s *Service) Close() {
if s.queue != nil {
// don't panic in case service is already closed
select {
case <-s.ctx.Done():
return
default:
}
log.Print("[DEBUG] close notifier")
close(s.queue)
close(s.verificationQueue)
+4 -3
View File
@@ -1,7 +1,6 @@
package notify
import (
"errors"
"fmt"
"math/rand"
"sync/atomic"
@@ -22,6 +21,8 @@ func TestService_NoDestinations(t *testing.T) {
s.Submit(Request{Comment: store.Comment{ID: "123"}})
s.Submit(Request{Comment: store.Comment{ID: "123"}})
s.Close()
// second call should not result in panic
s.Close()
}
func TestService_WithDestinations(t *testing.T) {
@@ -284,7 +285,7 @@ type mockStore struct {
func (m mockStore) getUserDetail(userID string) (string, error) {
detail, ok := m.userDetails[userID]
if !ok {
return "", errors.New("no such user")
return "", fmt.Errorf("no such user")
}
return detail, nil
}
@@ -292,7 +293,7 @@ func (m mockStore) getUserDetail(userID string) (string, error) {
func (m mockStore) Get(_ store.Locator, id string, _ store.User) (store.Comment, error) {
res, ok := m.data[id]
if !ok {
return store.Comment{}, errors.New("no such id")
return store.Comment{}, fmt.Errorf("no such id")
}
return res, nil
}
+19 -59
View File
@@ -2,43 +2,32 @@ package notify
import (
"context"
"fmt"
"net/url"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/slack-go/slack"
ntf "github.com/go-pkgz/notify"
)
// Slack implements notify.Destination for Slack
type Slack struct {
channelID string
*ntf.Slack
channelName string
client *slack.Client
}
// NewSlack makes Slack bot for notifications
func NewSlack(token, channelName string, opts ...slack.Option) (*Slack, error) {
func NewSlack(token, channelName string) *Slack {
log.Printf("[DEBUG] create new slack notifier for chan %s", channelName)
if channelName == "" {
channelName = "general"
}
client := slack.New(token, opts...)
res := &Slack{client: client, channelName: channelName}
channelID, err := res.findChannelIDByName(channelName)
if err != nil {
return nil, errors.Wrap(err, "can not find slack channel '"+channelName+"'")
}
res.channelID = channelID
log.Printf("[DEBUG] create new slack notifier for chan %s", channelID)
return res, nil
return &Slack{Slack: ntf.NewSlack(token), channelName: channelName}
}
// Send to Slack channel
func (t *Slack) Send(ctx context.Context, req Request) error {
func (s *Slack) Send(ctx context.Context, req Request) error {
log.Printf("[DEBUG] send slack notification, comment id %s", req.Comment.ID)
user := req.Comment.User.Name
@@ -51,51 +40,22 @@ func (t *Slack) Send(ctx context.Context, req Request) error {
title = "↦ " + req.Comment.PostTitle
}
_, _, err := t.client.PostMessageContext(ctx, t.channelID,
slack.MsgOptionText("New comment from "+user, false),
slack.MsgOptionAttachments(
slack.Attachment{
TitleLink: req.Comment.Locator.URL + uiNav + req.Comment.ID,
Title: title,
Text: req.Comment.Orig,
},
),
destination := fmt.Sprintf(
"slack:%s?title=%s&attachmentText=%s&titleLink=%s",
s.channelName,
url.QueryEscape(title),
url.QueryEscape(req.Comment.Orig),
url.QueryEscape(req.Comment.Locator.URL+uiNav+req.Comment.ID),
)
return err
return s.Slack.Send(ctx, destination, "New comment from "+user)
}
// SendVerification is not implemented for Slack
func (t *Slack) SendVerification(_ context.Context, _ VerificationRequest) error {
func (s *Slack) SendVerification(_ context.Context, _ VerificationRequest) error {
return nil
}
func (t *Slack) String() string {
return "slack: " + t.channelName + " (" + t.channelID + ")"
}
func (t *Slack) findChannelIDByName(name string) (string, error) {
params := slack.GetConversationsParameters{}
for {
chans, next, err := t.client.GetConversations(&params)
if err != nil {
return "", err
}
for _, channel := range chans {
if channel.Name == name {
return channel.ID, nil
}
}
if next == "" {
break
}
params.Cursor = next
}
return "", errors.New("no such channel")
func (s *Slack) String() string {
return s.Slack.String() + " for channel " + s.channelName + ""
}
+13 -143
View File
@@ -2,169 +2,39 @@ package notify
import (
"context"
"log"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"github.com/slack-go/slack"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/umputun/remark42/backend/app/store"
)
func TestSlack_New(t *testing.T) {
ts := newMockSlackServer()
defer ts.Close()
tb, err := ts.newClient("general")
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, "C12345678", tb.channelID)
_, err = ts.newClient("unknown-channel")
require.Error(t, err)
assert.Contains(t, err.Error(), "no such channel")
ts := NewSlack("", "")
assert.NotNil(t, ts)
assert.Equal(t, "general", ts.channelName)
}
func TestSlack_Send(t *testing.T) {
ts := NewSlack("", "")
ts := newMockSlackServer()
defer ts.Close()
tb, err := ts.newClient("general")
assert.NoError(t, err)
assert.NotNil(t, tb)
c := store.Comment{Text: "some text", ParentID: "1", ID: "999"}
c := store.Comment{PostTitle: "test title", Text: "some text", ParentID: "1", ID: "999"}
c.User.Name = "from"
cp := store.Comment{Text: "some parent text"}
cp.User.Name = "to"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
c.PostTitle = "test title"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
c.PostTitle = "[test title]"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
tb, err = ts.newClient("general")
assert.NoError(t, err)
ts.isServerDown = true
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
require.Error(t, err)
assert.Contains(t, err.Error(), "slack server error", "send on broken client")
ctx, cancel := context.WithCancel(context.Background())
cancel()
err := ts.Send(ctx, Request{Comment: c, parent: cp})
assert.Error(t, err)
}
func TestSlack_Name(t *testing.T) {
ts := newMockSlackServer()
defer ts.Close()
tb, err := ts.newClient("general")
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, "slack: general (C12345678)", tb.String())
tb := NewSlack("", "test-channel")
assert.Equal(t, "slack notifications destination for channel test-channel", tb.String())
}
func TestSlack_SendVerification(t *testing.T) {
ts := newMockSlackServer()
defer ts.Close()
tb, err := ts.newClient("general")
assert.NoError(t, err)
assert.NotNil(t, tb)
err = tb.SendVerification(context.TODO(), VerificationRequest{})
assert.NoError(t, err)
}
type mockSlackServer struct {
*httptest.Server
isServerDown bool
}
func (ts *mockSlackServer) newClient(channelName string) (*Slack, error) {
return NewSlack("any-token", channelName, slack.OptionAPIURL(ts.URL+"/"))
}
func newMockSlackServer() *mockSlackServer {
mockServer := mockSlackServer{}
router := chi.NewRouter()
router.Post("/conversations.list", func(w http.ResponseWriter, r *http.Request) {
s := `{
"ok": true,
"channels": [
{
"id": "C12345678",
"name": "general",
"is_channel": true,
"is_group": false,
"is_im": false,
"created": 1503888888,
"is_archived": false,
"is_general": false,
"unlinked": 0,
"name_normalized": "random",
"is_shared": false,
"parent_conversation": null,
"creator": "U12345678",
"is_ext_shared": false,
"is_org_shared": false,
"pending_shared": [],
"pending_connected_team_ids": [],
"is_pending_ext_shared": false,
"is_member": false,
"is_private": false,
"is_mpim": false,
"previous_names": [],
"num_members": 1
}
],
"response_metadata": {
"next_cursor": ""
}
}`
_, _ = w.Write([]byte(s))
})
router.Post("/chat.postMessage", func(w http.ResponseWriter, r *http.Request) {
if mockServer.isServerDown {
w.WriteHeader(500)
} else {
s := `{
"ok": true,
"channel": "C12345678",
"ts": "1617008342.000100",
"message": {
"type": "message",
"subtype": "bot_message",
"text": "wowo",
"ts": "1617008342.000100",
"username": "slackbot",
"bot_id": "B12345678"
}
}`
_, _ = w.Write([]byte(s))
}
})
router.NotFound(func(w http.ResponseWriter, r *http.Request) {
log.Printf("..... 404 for %s .....\n", r.URL)
})
mockServer.Server = httptest.NewServer(router)
return &mockServer
ts := NewSlack("", "")
assert.NoError(t, ts.SendVerification(context.Background(), VerificationRequest{}))
}
+38 -421
View File
@@ -1,26 +1,13 @@
package notify
import (
"bytes"
"context"
"encoding/json"
"fmt"
"golang.org/x/net/html"
"io"
"net/http"
neturl "net/url"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/hashicorp/go-multierror"
"github.com/microcosm-cc/bluemonday"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/repeater"
"github.com/pkg/errors"
ntf "github.com/go-pkgz/notify"
"github.com/hashicorp/go-multierror"
)
// TelegramParams contain settings for telegram notifications
@@ -30,78 +17,29 @@ type TelegramParams struct {
Timeout time.Duration // http client timeout
UserNotifications bool // flag which enables user notifications
ErrorMsg, SuccessMsg string // messages for successful and unsuccessful subscription requests to bot
apiPrefix string // changed only in tests
}
// Telegram implements notify.Destination for telegram
type Telegram struct {
TelegramParams
*ntf.Telegram
// Identifier of the first update to be requested.
// Should be equal to LastSeenUpdateID + 1
// See https://core.telegram.org/bots/api#getupdates
updateOffset int
apiPollInterval time.Duration // interval to check updates from Telegram API and answer to users
expiredCleanupInterval time.Duration // interval to check and clean up expired notification requests
username string // bot username
run int32 // non-zero if Run goroutine has started
requests struct {
sync.RWMutex
data map[string]tgAuthRequest
}
AdminChannelID string // unique identifier for the target chat or username of the target channel (in the format @channelusername)
UserNotifications bool // flag which enables user notifications
}
// telegramMsg is used to send message trough Telegram bot API
type telegramMsg struct {
Text string `json:"text"`
ParseMode string `json:"parse_mode,omitempty"`
}
type tgAuthRequest struct {
confirmed bool // whether login request has been confirmed and user info set
expires time.Time
telegramID string
user string
site string
}
// TelegramBotInfo structure contains information about telegram bot, which is used from whole telegram API response
type TelegramBotInfo struct {
Username string `json:"username"`
}
const telegramTimeOut = 5000 * time.Millisecond
const telegramAPIPrefix = "https://api.telegram.org/bot"
const tgPollInterval = time.Second * 5
const tgCleanupInterval = time.Minute * 5
// NewTelegram makes telegram bot for notifications
func NewTelegram(params TelegramParams) (*Telegram, error) {
res := Telegram{TelegramParams: params}
if res.apiPrefix == "" {
res.apiPrefix = telegramAPIPrefix
}
if res.Timeout == 0 {
res.Timeout = telegramTimeOut
}
res.apiPollInterval = tgPollInterval
res.expiredCleanupInterval = tgCleanupInterval
log.Printf("[DEBUG] create new telegram notifier for api=%s, timeout=%s", res.apiPrefix, res.Timeout)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
botInfo, err := res.botInfo(ctx)
client, err := ntf.NewTelegram(ntf.TelegramParams{
Token: params.Token,
Timeout: params.Timeout,
ErrorMsg: params.ErrorMsg,
SuccessMsg: params.SuccessMsg,
})
if err != nil {
return nil, errors.Wrapf(err, "can't retrieve bot info from Telegram API")
return nil, err
}
res.username = botInfo.Username
res.requests.data = make(map[string]tgAuthRequest)
return &res, nil
return &Telegram{Telegram: client, AdminChannelID: params.AdminChannelID, UserNotifications: params.UserNotifications}, nil
}
// Send to telegram recipients
@@ -109,121 +47,55 @@ func (t *Telegram) Send(ctx context.Context, req Request) error {
log.Printf("[DEBUG] send telegram notification for comment ID %s", req.Comment.ID)
result := new(multierror.Error)
msg, err := buildMessage(req)
if err != nil {
return errors.Wrapf(err, "failed to make telegram message body for comment ID %s", req.Comment.ID)
}
msg := t.buildMessage(req)
if t.AdminChannelID != "" {
err := t.sendMessage(ctx, msg, t.AdminChannelID)
result = multierror.Append(errors.Wrapf(err,
"problem sending admin telegram notification about comment ID %s to %s", req.Comment.ID, t.AdminChannelID),
)
err := t.Telegram.Send(ctx, fmt.Sprintf("telegram:%s?parseMode=HTML", t.AdminChannelID), msg)
if err != nil {
result = multierror.Append(result,
fmt.Errorf("problem sending admin telegram notification about comment ID %s to %s: %w",
req.Comment.ID, t.AdminChannelID, err,
),
)
}
}
if t.UserNotifications {
for _, user := range req.Telegrams {
err := t.sendMessage(ctx, msg, user)
result = multierror.Append(errors.Wrapf(err,
"problem sending user telegram notification about comment ID %s to %q", req.Comment.ID, user),
)
err := t.Telegram.Send(ctx, fmt.Sprintf("telegram:%s?parseMode=HTML", user), msg)
if err != nil {
result = multierror.Append(result,
fmt.Errorf("problem sending user telegram notification about comment ID %s to %q: %w",
req.Comment.ID, user, err,
),
)
}
}
}
return result.ErrorOrNil()
}
func (t *Telegram) sendMessage(ctx context.Context, b []byte, chatID string) error {
if _, err := strconv.ParseInt(chatID, 10, 64); err != nil {
chatID = "@" + chatID // if chatID not a number enforce @ prefix
}
url := fmt.Sprintf("sendMessage?chat_id=%s&disable_web_page_preview=true", chatID)
return t.Request(ctx, url, b, &struct{}{})
}
// buildMessage generates message for generic notification about new comment
func buildMessage(req Request) ([]byte, error) {
func (t *Telegram) buildMessage(req Request) string {
commentURLPrefix := req.Comment.Locator.URL + uiNav
msg := fmt.Sprintf(`<a href=%q>%s</a>`, commentURLPrefix+req.Comment.ID, escapeTelegramText(req.Comment.User.Name))
msg := fmt.Sprintf(`<a href=%q>%s</a>`, commentURLPrefix+req.Comment.ID, ntf.EscapeTelegramText(req.Comment.User.Name))
if req.Comment.ParentID != "" {
msg += fmt.Sprintf(" -> <a href=%q>%s</a>", commentURLPrefix+req.parent.ID, escapeTelegramText(req.parent.User.Name))
msg += fmt.Sprintf(" -> <a href=%q>%s</a>", commentURLPrefix+req.parent.ID, ntf.EscapeTelegramText(req.parent.User.Name))
}
msg += fmt.Sprintf("\n\n%s", telegramSupportedHTML(req.Comment.Text))
msg += fmt.Sprintf("\n\n%s", ntf.TelegramSupportedHTML(req.Comment.Text))
if req.Comment.ParentID != "" {
msg += fmt.Sprintf("\n\n\"<i>%s</i>\"", telegramSupportedHTML(req.parent.Text))
msg += fmt.Sprintf("\n\n\"<i>%s</i>\"", ntf.TelegramSupportedHTML(req.parent.Text))
}
if req.Comment.PostTitle != "" {
msg += fmt.Sprintf("\n\n↦ <a href=%q>%s</a>", req.Comment.Locator.URL, escapeTelegramText(req.Comment.PostTitle))
msg += fmt.Sprintf("\n\n↦ <a href=%q>%s</a>", req.Comment.Locator.URL, ntf.EscapeTelegramText(req.Comment.PostTitle))
}
body := telegramMsg{Text: msg, ParseMode: "HTML"}
b, err := json.Marshal(body)
if err != nil {
return nil, err
}
return b, nil
}
// returns HTML with only tags allowed in Telegram HTML message payload, also trims ending newlines
// https://core.telegram.org/bots/api#html-style
func telegramSupportedHTML(htmlText string) string {
adjustedHTMLText := adjustHTMLTags(htmlText)
p := bluemonday.NewPolicy()
p.AllowElements("b", "strong", "i", "em", "u", "ins", "s", "strike", "del", "a", "code", "pre")
p.AllowAttrs("href").OnElements("a")
p.AllowAttrs("class").OnElements("code")
return strings.TrimRight(p.Sanitize(adjustedHTMLText), "\n")
}
// returns text sanitized of symbols not allowed inside other HTML tags in Telegram HTML message payload
// https://core.telegram.org/bots/api#html-style
func escapeTelegramText(text string) string {
// order is important
text = strings.ReplaceAll(text, "&", "&amp;")
text = strings.ReplaceAll(text, "<", "&lt;")
text = strings.ReplaceAll(text, ">", "&gt;")
return text
}
// telegram not allow h1-h6 tags
// replace these tags with a combination of <b> and <i> for visual distinction
func adjustHTMLTags(htmlText string) string {
buff := strings.Builder{}
tokenizer := html.NewTokenizer(strings.NewReader(htmlText))
for {
if tokenizer.Next() == html.ErrorToken {
return buff.String()
}
token := tokenizer.Token()
switch token.Type {
case html.StartTagToken, html.EndTagToken:
switch token.Data {
case "h1", "h2", "h3":
if token.Type == html.StartTagToken {
buff.WriteString("<b>")
}
if token.Type == html.EndTagToken {
buff.WriteString("</b>")
}
case "h4", "h5", "h6":
if token.Type == html.StartTagToken {
buff.WriteString("<i><b>")
}
if token.Type == html.EndTagToken {
buff.WriteString("</b></i>")
}
default:
buff.WriteString(token.String())
}
default:
buff.WriteString(token.String())
}
}
return msg
}
// SendVerification is not needed for telegram
@@ -231,132 +103,8 @@ func (t *Telegram) SendVerification(_ context.Context, _ VerificationRequest) er
return nil
}
// TelegramUpdate contains update information, which is used from whole telegram API response
type TelegramUpdate struct {
Result []struct {
UpdateID int `json:"update_id"`
Message struct {
Chat struct {
ID int `json:"id"`
Name string `json:"first_name"`
Type string `json:"type"`
} `json:"chat"`
Text string `json:"text"`
} `json:"message"`
} `json:"result"`
}
// GetBotUsername returns bot username
func (t *Telegram) GetBotUsername() string {
return t.username
}
// AddToken adds token
func (t *Telegram) AddToken(token, user, site string, expires time.Time) {
t.requests.Lock()
t.requests.data[token] = tgAuthRequest{
expires: expires,
user: user,
site: site,
}
t.requests.Unlock()
}
// CheckToken verifies incoming token, returns the user address if it's confirmed and empty string otherwise
func (t *Telegram) CheckToken(token, user string) (telegram, site string, err error) {
t.requests.RLock()
authRequest, ok := t.requests.data[token]
t.requests.RUnlock()
if !ok {
return "", "", errors.New("request is not found")
}
if time.Now().After(authRequest.expires) {
t.requests.Lock()
delete(t.requests.data, token)
t.requests.Unlock()
return "", "", errors.New("request expired")
}
if !authRequest.confirmed {
return "", "", errors.New("request is not verified yet")
}
if authRequest.user != user {
return "", "", errors.New("user does not match original requester")
}
// Delete request
t.requests.Lock()
delete(t.requests.data, token)
t.requests.Unlock()
return authRequest.telegramID, authRequest.site, nil
}
// Run starts processing login requests sent in Telegram, required for user notifications to work
// Blocks caller
func (t *Telegram) Run(ctx context.Context) {
atomic.AddInt32(&t.run, 1)
processUpdatedTicker := time.NewTicker(t.apiPollInterval)
cleanupTicker := time.NewTicker(t.expiredCleanupInterval)
for {
select {
case <-ctx.Done():
processUpdatedTicker.Stop()
cleanupTicker.Stop()
atomic.AddInt32(&t.run, -1)
return
case <-processUpdatedTicker.C:
updates, err := t.getUpdates(ctx)
if err != nil {
log.Printf("[WARN] Error while getting telegram updates: %v", err)
continue
}
t.processUpdates(ctx, updates)
case <-cleanupTicker.C:
now := time.Now()
t.requests.Lock()
for key, req := range t.requests.data {
if now.After(req.expires) {
delete(t.requests.data, key)
}
}
t.requests.Unlock()
}
}
}
// ProcessUpdate is alternative to Run, it processes provided plain text update from Telegram
// so that caller could get updates and send it not only there but to multiple sources
func (t *Telegram) ProcessUpdate(ctx context.Context, textUpdate string) error {
if atomic.LoadInt32(&t.run) != 0 {
return errors.New("Run goroutine should not be used with ProcessUpdate")
}
defer func() {
// as Run goroutine is not running, clean up old requests on each update
// even if we hit json decode error
now := time.Now()
t.requests.Lock()
for key, req := range t.requests.data {
if now.After(req.expires) {
delete(t.requests.data, key)
}
}
t.requests.Unlock()
}()
var updates TelegramUpdate
if err := json.Unmarshal([]byte(textUpdate), &updates); err != nil {
return errors.Wrap(err, "failed to decode provided telegram update")
}
t.processUpdates(ctx, &updates)
return nil
}
func (t *Telegram) String() string {
result := "telegram"
result := t.Telegram.String()
if t.AdminChannelID != "" {
result += " with admin notifications to " + t.AdminChannelID
}
@@ -365,134 +113,3 @@ func (t *Telegram) String() string {
}
return result
}
// getUpdates fetches incoming updates
func (t *Telegram) getUpdates(ctx context.Context) (*TelegramUpdate, error) {
url := `getUpdates?allowed_updates=["message"]`
if t.updateOffset != 0 {
url += fmt.Sprintf("&offset=%d", t.updateOffset)
}
var result TelegramUpdate
err := t.Request(ctx, url, nil, &result)
if err != nil {
return nil, errors.Wrap(err, "failed to fetch updates")
}
for _, u := range result.Result {
if u.UpdateID >= t.updateOffset {
t.updateOffset = u.UpdateID + 1
}
}
return &result, nil
}
// processUpdates processes a batch of updates from telegram servers
func (t *Telegram) processUpdates(ctx context.Context, updates *TelegramUpdate) {
for _, update := range updates.Result {
if update.Message.Chat.Type != "private" {
continue
}
if !strings.HasPrefix(update.Message.Text, "/start ") {
continue
}
token := strings.TrimPrefix(update.Message.Text, "/start ")
t.requests.RLock()
authRequest, ok := t.requests.data[token]
if !ok { // No such token
t.requests.RUnlock()
if t.ErrorMsg != "" {
if err := t.sendText(ctx, update.Message.Chat.ID, t.ErrorMsg); err != nil {
log.Printf("[WARN] failed to notify telegram peer: %v", err)
}
}
continue
}
t.requests.RUnlock()
authRequest.confirmed = true
authRequest.telegramID = strconv.Itoa(update.Message.Chat.ID)
t.requests.Lock()
t.requests.data[token] = authRequest
t.requests.Unlock()
if err := t.sendText(ctx, update.Message.Chat.ID, t.SuccessMsg); err != nil {
log.Printf("[ERROR] failed to notify telegram peer: %v", err)
}
}
}
// sendText sends a plain text message to telegram peer
func (t *Telegram) sendText(ctx context.Context, recipientID int, msg string) error {
url := fmt.Sprintf("sendMessage?chat_id=%d&text=%s", recipientID, neturl.PathEscape(msg))
return t.Request(ctx, url, nil, &struct{}{})
}
// botInfo returns info about configured bot
func (t *Telegram) botInfo(ctx context.Context) (*TelegramBotInfo, error) {
var resp = struct {
Result *TelegramBotInfo `json:"result"`
}{}
err := t.Request(ctx, "getMe", nil, &resp)
if err != nil {
return nil, err
}
if resp.Result == nil {
return nil, errors.New("received empty result")
}
return resp.Result, nil
}
// Request makes a request to the Telegram API and return the result
func (t *Telegram) Request(ctx context.Context, method string, b []byte, data interface{}) error {
return repeater.NewDefault(3, time.Millisecond*250).Do(ctx, func() error {
url := fmt.Sprintf("%s%s/%s", t.apiPrefix, t.Token, method)
var req *http.Request
var err error
if b == nil {
req, err = http.NewRequestWithContext(ctx, "GET", url, http.NoBody)
} else {
req, err = http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(b))
req.Header.Set("Content-Type", "application/json; charset=utf-8")
}
if err != nil {
return errors.Wrap(err, "failed to create request")
}
client := http.Client{Timeout: t.Timeout}
resp, err := client.Do(req)
if err != nil {
return errors.Wrap(err, "failed to send request")
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return t.parseError(resp.Body, resp.StatusCode)
}
if err = json.NewDecoder(resp.Body).Decode(data); err != nil {
return errors.Wrap(err, "failed to decode json response")
}
return nil
})
}
func (t *Telegram) parseError(r io.Reader, statusCode int) error {
tgErr := struct {
Description string `json:"description"`
}{}
if err := json.NewDecoder(r).Decode(&tgErr); err != nil {
return errors.Errorf("unexpected telegram API status code %d", statusCode)
}
return errors.Errorf("unexpected telegram API status code %d, error: %q", statusCode, tgErr.Description)
}
+30 -476
View File
@@ -2,507 +2,61 @@ package notify
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
ntf "github.com/go-pkgz/notify"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/umputun/remark42/backend/app/store"
)
func TestTelegram_New(t *testing.T) {
ts := mockTelegramServer(nil)
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "good-token",
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, tb.Timeout, time.Second*5)
assert.Equal(t, "remark_test", tb.AdminChannelID, "@ added")
_, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "empty-json",
apiPrefix: ts.URL + "/",
})
assert.EqualError(t, err, "can't retrieve bot info from Telegram API: received empty result")
st := time.Now()
_, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "non-json-resp",
Timeout: 2 * time.Second,
apiPrefix: ts.URL + "/",
})
func TestTelegram_NewError(t *testing.T) {
tb, err := NewTelegram(TelegramParams{})
assert.Error(t, err)
assert.Contains(t, err.Error(), "failed to decode json response:")
assert.True(t, time.Since(st) >= 250*3*time.Millisecond)
_, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "404",
Timeout: 2 * time.Second,
apiPrefix: ts.URL + "/",
})
assert.EqualError(t, err, "can't retrieve bot info from Telegram API: unexpected telegram API status code 404")
_, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "no-such-thing",
apiPrefix: "http://127.0.0.1:4321/",
})
require.Error(t, err)
assert.Contains(t, err.Error(), "can't retrieve bot info from Telegram API")
assert.Contains(t, err.Error(), "dial tcp 127.0.0.1:4321: connect: connection refused")
_, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "no-such-thing",
apiPrefix: "",
})
assert.Error(t, err, "empty api url not allowed")
_, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "good-token",
Timeout: 2 * time.Second,
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err, "0 timeout allowed as default")
tb, err = NewTelegram(TelegramParams{
AdminChannelID: "1234567890",
Token: "good-token",
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, "1234567890", tb.AdminChannelID, "no @ prefix")
}
func TestTelegram_GetBotUsername(t *testing.T) {
ts := mockTelegramServer(nil)
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "good-token",
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, "remark42_test_bot", tb.GetBotUsername())
assert.Nil(t, tb)
}
func TestTelegram_Send(t *testing.T) {
ts := mockTelegramServer(nil)
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
tb := Telegram{
AdminChannelID: "remark_test",
Token: "good-token",
UserNotifications: true,
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
assert.NotNil(t, tb)
c := store.Comment{Text: "some text", ParentID: "1", ID: "999", Locator: store.Locator{URL: "http://example.org/"}}
Telegram: &ntf.Telegram{}, // broken sender due to unset API
}
assert.Equal(t, "telegram notifications destination with admin notifications to remark_test with user notifications enabled", tb.String())
c := store.Comment{Text: "some text", ParentID: "1", ID: "999", PostTitle: "[test title]", Locator: store.Locator{URL: "http://example.org/"}}
c.User.Name = "from"
cp := store.Comment{Text: `<p>some parent text with a <a href="http://example.org">link</a> and special text:<br>& < > &</p>`}
cp.User.Name = "to"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
assert.NoError(t, err)
c.PostTitle = "test title"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
c.PostTitle = "[test title]"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
assert.NoError(t, err)
tb, err = NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "non-json-resp",
UserNotifications: true,
apiPrefix: ts.URL + "/",
})
assert.Nil(t, tb)
assert.Error(t, err, "should fail")
tb = &Telegram{
TelegramParams: TelegramParams{
AdminChannelID: "remark_test",
Token: "non-json-resp",
UserNotifications: true,
apiPrefix: ts.URL + "/",
}}
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
require.Error(t, err)
assert.Contains(t, err.Error(), "unexpected telegram API status code 404", "send on broken tg")
assert.Equal(t, "telegram with admin notifications to remark_test with user notifications enabled", tb.String())
// bad API URL
tb.apiPrefix = "http://non-existent"
err = tb.Send(context.TODO(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
err := tb.Send(context.Background(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
assert.Error(t, err)
assert.Contains(t, err.Error(), "2 errors occurred")
assert.Contains(t, err.Error(), "problem sending user telegram notification about comment ID 999 to \"test_user_channel\"")
assert.Contains(t, err.Error(), "problem sending admin telegram notification about comment ID 999 to remark_test")
// test buildMessage separately for message text
res, err := buildMessage(Request{Comment: c, parent: cp})
assert.NoError(t, err)
assert.Equal(t, `{"text":"\u003ca href=\"http://example.org/#remark42__comment-999\"\u003efrom\u003c/a\u003e -\u003e \u003ca href=\"http://example.org/#remark42__comment-\"\u003eto\u003c/a\u003e\n\n`+
`some text\n\n`+
`\"\u003ci\u003esome parent text with a \u003ca href=\"http://example.org\"\u003elink\u003c/a\u003e and special text:\u0026amp; \u0026lt; \u0026gt; \u0026amp;\u003c/i\u003e\"\n\n`+
`↦ \u003ca href=\"http://example.org/\"\u003e[test title]\u003c/a\u003e","parse_mode":"HTML"}`,
string(res))
res := tb.buildMessage(Request{Comment: c, parent: cp})
assert.Equal(t, `<a href="http://example.org/#remark42__comment-999">from</a> -> <a href="http://example.org/#remark42__comment-">to</a>
some text
"<i>some parent text with a <a href="http://example.org">link</a> and special text:&amp; &lt; &gt; &amp;</i>"
<a href="http://example.org/">[test title]</a>`,
res)
// special case for text with h1-h6 header
ch := store.Comment{Text: "<h1>Hello</h1><h6>World</h6>", ID: "555", Locator: store.Locator{URL: "http://example.org/"}}
ch.User.Name = "from"
res, err = buildMessage(Request{Comment: ch})
assert.NoError(t, err)
assert.Equal(t, `{"text":"\u003ca href=\"http://example.org/#remark42__comment-555\"\u003efrom\u003c/a\u003e\n\n`+
`\u003cb\u003eHello\u003c/b\u003e\u003ci\u003e\u003cb\u003eWorld\u003c/b\u003e\u003c/i\u003e`+
`","parse_mode":"HTML"}`,
string(res))
res = tb.buildMessage(Request{Comment: ch})
assert.Equal(t, `<a href="http://example.org/#remark42__comment-555">from</a>
<b>Hello</b><i><b>World</b></i>`,
res)
}
func TestTelegram_SendVerification(t *testing.T) {
ts := mockTelegramServer(nil)
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "good-token",
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
assert.NotNil(t, tb)
// empty VerificationRequest should return no error and no nothing, as well as any other
assert.NoError(t, tb.SendVerification(context.TODO(), VerificationRequest{}))
}
const getUpdatesResp = `{
"ok": true,
"result": [
{
"update_id": 998,
"message": {
"chat": {
"type": "group"
}
}
},
{
"update_id": 999,
"message": {
"text": "not starting with /start",
"chat": {
"type": "private"
}
}
},
{
"update_id": 1000,
"message": {
"message_id": 4,
"from": {
"id": 313131313,
"is_bot": false,
"first_name": "Joe",
"username": "joe123",
"language_code": "en"
},
"chat": {
"id": 313131313,
"first_name": "Joe",
"username": "joe123",
"type": "private"
},
"date": 1601665548,
"text": "/start token",
"entities": [
{
"offset": 0,
"length": 6,
"type": "bot_command"
}
]
}
}
]
}`
func TestTelegram_GetUpdatesFlow(t *testing.T) {
first := true
ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
if strings.Contains(r.URL.String(), "sendMessage") {
// respond normally to processUpdates attempt to send message back to user
_, _ = w.Write([]byte("{}"))
return
}
// responses to get updates calls to API
if first {
assert.Equal(t, "", r.URL.Query().Get("offset"))
first = false
} else {
assert.Equal(t, "1001", r.URL.Query().Get("offset"))
}
_, _ = w.Write([]byte(getUpdatesResp))
})
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "xxxsupersecretxxx",
UserNotifications: true,
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
// send request with no offset
upd, err := tb.getUpdates(context.Background())
assert.NoError(t, err)
assert.Len(t, upd.Result, 3)
assert.Equal(t, 1001, tb.updateOffset)
assert.Equal(t, "/start token", upd.Result[len(upd.Result)-1].Message.Text)
tb.AddToken("token", "user", "site", time.Now().Add(time.Minute))
_, _, err = tb.CheckToken("token", "user")
assert.Error(t, err)
tb.processUpdates(context.Background(), upd)
tgID, site, err := tb.CheckToken("token", "user")
assert.NoError(t, err)
assert.Equal(t, "313131313", tgID)
assert.Equal(t, "site", site)
// send request with offset
_, err = tb.getUpdates(context.Background())
assert.NoError(t, err)
}
func TestTelegram_ProcessUpdateFlow(t *testing.T) {
ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
// respond normally to processUpdates attempt to send message back to user
_, _ = w.Write([]byte("{}"))
})
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "xxxsupersecretxxx",
UserNotifications: true,
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
tb.AddToken("token", "user", "site", time.Now().Add(time.Minute))
tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
assert.Len(t, tb.requests.data, 2)
_, _, err = tb.CheckToken("token", "user")
assert.Error(t, err)
assert.NoError(t, tb.ProcessUpdate(context.Background(), getUpdatesResp))
assert.Len(t, tb.requests.data, 1, "expired token was cleaned up")
tgID, site, err := tb.CheckToken("token", "user")
assert.NoError(t, err)
assert.Len(t, tb.requests.data, 0, "token is deleted after successful check")
assert.Equal(t, "313131313", tgID)
assert.Equal(t, "site", site)
tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
assert.Len(t, tb.requests.data, 1)
assert.EqualError(t, tb.ProcessUpdate(context.Background(), ""), "failed to decode provided telegram update: unexpected end of JSON input")
assert.Len(t, tb.requests.data, 0, "expired token should be cleaned up despite the error")
}
const sendMessageResp = `{
"ok": true,
"result": {
"message_id": 100,
"from": {
"id": 666666666,
"is_bot": true,
"first_name": "Test auth bot",
"username": "TestAuthBot"
},
"chat": {
"id": 313131313,
"first_name": "Joe",
"username": "joe123",
"type": "private"
},
"date": 1602430546,
"text": "123"
}
}`
func TestTelegram_SendText(t *testing.T) {
ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, "123", r.URL.Query().Get("chat_id"))
assert.Equal(t, "hello there", r.URL.Query().Get("text"))
_, _ = w.Write([]byte(sendMessageResp))
})
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "xxxsupersecretxxx",
UserNotifications: true,
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
err = tb.sendText(context.Background(), 123, "hello there")
assert.NoError(t, err)
}
const errorResp = `{"ok":false,"error_code":400,"description":"Very bad request"}`
func TestTelegram_Error(t *testing.T) {
ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = w.Write([]byte(errorResp))
})
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "xxxsupersecretxxx",
UserNotifications: true,
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
_, err = tb.getUpdates(context.Background())
assert.EqualError(t, err, "failed to fetch updates: unexpected telegram API status code 400, error: \"Very bad request\"")
}
func TestTelegram_TokenVerification(t *testing.T) {
ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
if strings.Contains(r.URL.String(), "sendMessage") {
// respond normally to processUpdates attempt to send message back to user
_, _ = w.Write([]byte("{}"))
return
}
// responses to get updates calls to API
_, _ = w.Write([]byte(getUpdatesResp))
})
defer ts.Close()
tb, err := NewTelegram(TelegramParams{
AdminChannelID: "remark_test",
Token: "good-token",
apiPrefix: ts.URL + "/",
})
assert.NoError(t, err)
assert.NotNil(t, tb)
tb.AddToken("token", "user", "site", time.Now().Add(time.Minute))
assert.Len(t, tb.requests.data, 1)
// wrong token
tgID, site, err := tb.CheckToken("unknown token", "user")
assert.Empty(t, tgID)
assert.Empty(t, site)
assert.EqualError(t, err, "request is not found")
// right token and user, not verified yet
tgID, site, err = tb.CheckToken("token", "user")
assert.Empty(t, tgID)
assert.Empty(t, site)
assert.EqualError(t, err, "request is not verified yet")
// confirm request
authRequest, ok := tb.requests.data["token"]
assert.True(t, ok)
authRequest.confirmed = true
authRequest.telegramID = "telegramID"
tb.requests.data["token"] = authRequest
// wrong user
tgID, site, err = tb.CheckToken("token", "wrong user")
assert.Empty(t, tgID)
assert.Empty(t, site)
assert.EqualError(t, err, "user does not match original requester")
// successful check
tgID, site, err = tb.CheckToken("token", "user")
assert.NoError(t, err)
assert.Equal(t, "telegramID", tgID)
assert.Equal(t, "site", site)
// expired token
tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
tgID, site, err = tb.CheckToken("expired token", "user")
assert.Empty(t, tgID)
assert.Empty(t, site)
assert.EqualError(t, err, "request expired")
assert.Len(t, tb.requests.data, 0)
// expired token, cleaned up by the cleanup
tb.apiPollInterval = time.Millisecond * 15
tb.expiredCleanupInterval = time.Millisecond * 10
ctx, cancel := context.WithCancel(context.Background())
go tb.Run(ctx)
assert.Eventually(t, func() bool {
return tb.ProcessUpdate(ctx, "").Error() == "Run goroutine should not be used with ProcessUpdate"
}, time.Millisecond*100, time.Millisecond*10, "ProcessUpdate should not work same time as Run")
tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
tb.requests.RLock()
assert.Len(t, tb.requests.data, 1)
tb.requests.RUnlock()
time.Sleep(tb.expiredCleanupInterval * 2)
tb.requests.RLock()
assert.Len(t, tb.requests.data, 0)
tb.requests.RUnlock()
cancel()
// give enough time for Run() to finish
time.Sleep(tb.expiredCleanupInterval)
}
const getMeResp = `{"ok": true,
"result": {
"first_name": "comments_test",
"id": 707381019,
"is_bot": true,
"username": "remark42_test_bot"
}}`
func mockTelegramServer(h http.HandlerFunc) *httptest.Server {
if h != nil {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.Contains(r.URL.String(), "getMe") {
_, _ = w.Write([]byte(getMeResp))
return
}
h(w, r)
}))
}
router := chi.NewRouter()
router.Get("/good-token/getMe", func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(getMeResp))
})
router.Get("/empty-json/getMe", func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{}`))
})
router.Get("/non-json-resp/getMe", func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`not-a-json`))
})
router.Get("/404/getMe", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(404)
})
router.Post("/good-token/sendMessage", func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"ok": true}`))
})
return httptest.NewServer(router)
tb := Telegram{}
// empty VerificationRequest should return no error and do nothing, as well as any other
assert.NoError(t, tb.SendVerification(context.Background(), VerificationRequest{}))
}
+37 -66
View File
@@ -4,110 +4,81 @@ import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"strings"
"text/template"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
ntf "github.com/go-pkgz/notify"
)
const (
webhookDefaultTemplate = `{"text": "{{.Text}}"}`
)
// WebhookClient defines an interface of client for webhook
type WebhookClient interface {
Do(*http.Request) (*http.Response, error)
}
// WebhookParams contain settings for webhook notifications
type WebhookParams struct {
WebhookURL string
Template string
Headers []string
URL string
Template string
Headers []string
Timeout time.Duration
}
// Webhook implements notify.Destination for Webhook notifications
type Webhook struct {
WebhookParams
webhookClient WebhookClient
webhookTemplate *template.Template
*ntf.Webhook
url string
template *template.Template
}
// NewWebhook makes Webhook
func NewWebhook(client WebhookClient, params WebhookParams) (*Webhook, error) {
res := &Webhook{WebhookParams: params}
if res.WebhookURL == "" {
return nil, errors.New("webhook URL is required for webhook notifications")
func NewWebhook(params WebhookParams) (*Webhook, error) {
res := &Webhook{
Webhook: ntf.NewWebhook(ntf.WebhookParams{
Timeout: params.Timeout,
Headers: params.Headers,
}),
url: params.URL,
}
if res.Template == "" {
res.Template = webhookDefaultTemplate
if res.url == "" {
return nil, fmt.Errorf("webhook URL is required for webhook notifications")
}
payloadTmpl, err := template.New("webhook").Parse(res.Template)
if params.Template == "" {
params.Template = webhookDefaultTemplate
}
payloadTmpl, err := template.New("webhook").Parse(params.Template)
if err != nil {
return nil, errors.Wrap(err, "unable to parse webhook template")
return nil, fmt.Errorf("unable to parse webhook template: %w", err)
}
res.webhookClient = client
res.webhookTemplate = payloadTmpl
res.template = payloadTmpl
log.Printf("[DEBUG] create new webhook notifier for %s", res.WebhookURL)
log.Printf("[DEBUG] create new webhook notifier for %s", res.url)
return res, nil
}
// Send sends Webhook notification
func (t *Webhook) Send(ctx context.Context, req Request) error {
var payload bytes.Buffer
err := t.webhookTemplate.Execute(&payload, req.Comment)
if err != nil {
return errors.Wrap(err, "unable to compile webhook template")
}
httpReq, err := http.NewRequestWithContext(ctx, "POST", t.WebhookURL, &payload)
if err != nil {
return errors.Wrap(err, "unable to create webhook request")
}
for _, h := range t.Headers {
elems := strings.Split(h, ":")
if len(elems) != 2 {
continue
}
httpReq.Header.Set(strings.TrimSpace(elems[0]), strings.TrimSpace(elems[1]))
}
resp, err := t.webhookClient.Do(httpReq)
if err != nil {
return errors.Wrap(err, "webhook request failed")
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
errMsg := fmt.Sprintf("webhook request failed with non-OK status code: %d", resp.StatusCode)
respBody, e := io.ReadAll(resp.Body)
if e != nil {
return fmt.Errorf(errMsg)
}
return fmt.Errorf("%s, body: %s", errMsg, respBody)
}
func (w *Webhook) Send(ctx context.Context, req Request) error {
log.Printf("[DEBUG] send webhook notification, comment id %s", req.Comment.ID)
var payload bytes.Buffer
err := w.template.Execute(&payload, req.Comment)
if err != nil {
return fmt.Errorf("unable to compile webhook template: %w", err)
}
return nil
return w.Webhook.Send(ctx, w.url, payload.String())
}
// SendVerification is not implemented for Webhook
func (t *Webhook) SendVerification(_ context.Context, _ VerificationRequest) error {
func (w *Webhook) SendVerification(_ context.Context, _ VerificationRequest) error {
return nil
}
// String describes the webhook instance
func (t *Webhook) String() string {
return fmt.Sprintf("webhook notification to %s", t.WebhookURL)
func (w *Webhook) String() string {
return fmt.Sprintf("%s to %s", w.Webhook.String(), w.url)
}
+24 -95
View File
@@ -1,12 +1,9 @@
package notify
import (
"bytes"
"context"
"errors"
"io"
"net/http"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -14,72 +11,33 @@ import (
"github.com/umputun/remark42/backend/app/store"
)
type funcWebhookClient func(*http.Request) (*http.Response, error)
func (c funcWebhookClient) Do(r *http.Request) (*http.Response, error) {
return c(r)
}
var okWebhookClient = funcWebhookClient(func(*http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString("ok")),
}, nil
})
type errReader struct {
}
func (errReader) Read(p []byte) (n int, err error) {
return 0, errors.New("test error")
}
func TestWebhook_NewWebhook(t *testing.T) {
wh, err := NewWebhook(okWebhookClient, WebhookParams{
WebhookURL: "https://example.org/webhook",
Headers: []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="},
wh, err := NewWebhook(WebhookParams{
URL: "https://example.org/webhook",
Headers: []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="},
})
assert.NoError(t, err)
assert.NotNil(t, wh)
assert.Equal(t, "https://example.org/webhook", wh.WebhookURL)
assert.Equal(t, "https://example.org/webhook", wh.url)
assert.Equal(t, []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="}, wh.Headers)
assert.Equal(t, `{"text": "{{.Text}}"}`, wh.Template)
assert.NotNil(t, wh.template)
wh, err = NewWebhook(okWebhookClient, WebhookParams{
WebhookURL: "https://example.org/webhook",
Headers: []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="},
Template: "{{.Text}}",
})
assert.NoError(t, err)
assert.NotNil(t, wh)
assert.Equal(t, "{{.Text}}", wh.Template)
wh, err = NewWebhook(okWebhookClient, WebhookParams{})
wh, err = NewWebhook(WebhookParams{})
assert.Nil(t, wh)
assert.Error(t, err)
assert.Equal(t, "webhook URL is required for webhook notifications", err.Error())
wh, err = NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook", Template: "{{.Text"})
wh, err = NewWebhook(WebhookParams{URL: "https://example.org/webhook", Template: "{{.Text"})
assert.Nil(t, wh)
assert.Error(t, err)
assert.Contains(t, err.Error(), "unable to parse webhook template")
}
func TestWebhook_Send(t *testing.T) {
wh, err := NewWebhook(funcWebhookClient(func(r *http.Request) (*http.Response, error) {
assert.Len(t, r.Header, 1)
assert.Equal(t, r.Header.Get("Content-Type"), "application/json,text/plain")
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString("")),
}, nil
}), WebhookParams{
WebhookURL: "https://example.org/webhook",
Headers: []string{"Content-Type:application/json,text/plain", ""},
wh, err := NewWebhook(WebhookParams{
URL: "bad-url",
Headers: []string{"Content-Type:application/json,text/plain", ""},
})
assert.NoError(t, err)
assert.NotNil(t, wh)
@@ -87,74 +45,45 @@ func TestWebhook_Send(t *testing.T) {
c := store.Comment{Text: "some text", ParentID: "1", ID: "999"}
c.User.Name = "from"
err = wh.Send(context.TODO(), Request{Comment: c})
assert.NoError(t, err)
err = wh.Send(context.Background(), Request{Comment: c})
assert.Error(t, err)
wh, err = NewWebhook(okWebhookClient, WebhookParams{
WebhookURL: "https://example.org/webhook",
Template: "{{.InvalidProperty}}",
wh, err = NewWebhook(WebhookParams{
URL: "https://example.org/webhook",
Template: "{{.InvalidProperty}}",
})
assert.NoError(t, err)
err = wh.Send(context.TODO(), Request{Comment: c})
err = wh.Send(context.Background(), Request{Comment: c})
require.Error(t, err)
assert.Contains(t, err.Error(), "webhook template")
wh, err = NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook"})
wh, err = NewWebhook(WebhookParams{URL: "https://example.org/webhook"})
assert.NoError(t, err)
err = wh.Send(nil, Request{Comment: c}) // nolint
require.Error(t, err)
assert.Contains(t, err.Error(), "unable to create webhook request")
wh, err = NewWebhook(funcWebhookClient(func(*http.Request) (*http.Response, error) {
return nil, errors.New("request failed")
}), WebhookParams{WebhookURL: "https://not-existing-url.net"})
wh, err = NewWebhook(WebhookParams{URL: "https://not-existing-url.net"})
assert.NoError(t, err)
err = wh.Send(context.TODO(), Request{Comment: c})
err = wh.Send(context.Background(), Request{Comment: c})
require.Error(t, err)
assert.Contains(t, err.Error(), "webhook request failed")
wh, err = NewWebhook(funcWebhookClient(func(*http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusNotFound,
Body: io.NopCloser(bytes.NewBufferString("not found")),
}, nil
}), WebhookParams{
WebhookURL: "http:/example.org/invalid-url",
})
assert.NoError(t, err)
err = wh.Send(context.TODO(), Request{Comment: c})
require.Error(t, err)
assert.Contains(t, err.Error(), "non-OK status code: 404, body: not found")
wh, err = NewWebhook(funcWebhookClient(func(*http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusNotFound,
Body: io.NopCloser(errReader{}),
}, nil
}), WebhookParams{
WebhookURL: "http:/example.org/invalid-url",
})
assert.NoError(t, err)
err = wh.Send(context.TODO(), Request{Comment: c})
require.Error(t, err)
assert.Contains(t, err.Error(), "non-OK status code: 404")
assert.NotContains(t, err.Error(), "body")
}
func TestWebhook_SendVerification(t *testing.T) {
wh, err := NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook"})
wh, err := NewWebhook(WebhookParams{URL: "https://example.org/webhook"})
assert.NoError(t, err)
assert.NotNil(t, wh)
err = wh.SendVerification(context.TODO(), VerificationRequest{})
err = wh.SendVerification(context.Background(), VerificationRequest{})
assert.NoError(t, err)
}
func TestWebhook_String(t *testing.T) {
wh, err := NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook"})
wh, err := NewWebhook(WebhookParams{URL: "https://example.org/webhook", Timeout: time.Minute * 5})
assert.NoError(t, err)
assert.NotNil(t, wh)
str := wh.String()
assert.Equal(t, "webhook notification to https://example.org/webhook", str)
assert.Equal(t, "webhook notification with timeout 5m0s to https://example.org/webhook", str)
}
+2 -3
View File
@@ -11,8 +11,7 @@ import (
"time"
log "github.com/go-pkgz/lgr"
"github.com/umputun/remark42/backend/app/notify"
ntf "github.com/go-pkgz/notify"
)
type tgRequester interface {
@@ -45,7 +44,7 @@ func DispatchTelegramUpdates(ctx context.Context, requester tgRequester, receive
url += fmt.Sprintf("&offset=%d", updateOffset)
}
var update notify.TelegramUpdate
var update ntf.TelegramUpdate
err := requester.Request(ctx, url, nil, &update)
if err != nil {
+5 -6
View File
@@ -3,13 +3,12 @@ package providers
import (
"context"
"encoding/json"
"errors"
"fmt"
"testing"
"time"
ntf "github.com/go-pkgz/notify"
"github.com/stretchr/testify/assert"
"github.com/umputun/remark42/backend/app/notify"
)
func TestDispatchTelegramUpdates(t *testing.T) {
@@ -46,7 +45,7 @@ func (m *mockTGRequester) Request(_ context.Context, _ string, _ []byte, data in
assert.NoError(m.t, json.Unmarshal([]byte(getUpdatesResp), data))
return nil
}
return errors.New("test error")
return fmt.Errorf("test error")
}
type mockTGUpdatesReceiver struct {
@@ -59,7 +58,7 @@ func (m *mockTGUpdatesReceiver) String() string {
}
func (m *mockTGUpdatesReceiver) ProcessUpdate(_ context.Context, textUpdate string) error {
var result notify.TelegramUpdate
var result ntf.TelegramUpdate
err := json.Unmarshal([]byte(textUpdate), &result)
assert.NoError(m.t, err)
if m.hit < 2 {
@@ -68,5 +67,5 @@ func (m *mockTGUpdatesReceiver) ProcessUpdate(_ context.Context, textUpdate stri
return nil
}
assert.Nil(m.t, result.Result)
return errors.New("test error")
return fmt.Errorf("test error")
}
+8 -12
View File
@@ -1,7 +1,7 @@
package api
import (
"errors"
"fmt"
"net/http"
"path"
"time"
@@ -29,22 +29,21 @@ type admin struct {
type adminStore interface {
Delete(locator store.Locator, commentID string, mode store.DeleteMode) error
DeleteUser(siteID string, userID string, mode store.DeleteMode) error
DeleteUserDetail(siteID string, userID string, detail engine.UserDetail) error
DeleteUser(siteID, userID string, mode store.DeleteMode) error
DeleteUserDetail(siteID, userID string, detail engine.UserDetail) error
User(siteID, userID string, limit, skip int, user store.User) ([]store.Comment, error)
IsBlocked(siteID string, userID string) bool
SetBlock(siteID string, userID string, status bool, ttl time.Duration) error
IsBlocked(siteID, userID string) bool
SetBlock(siteID, userID string, status bool, ttl time.Duration) error
BlockedUsers(siteID string) ([]store.BlockedUser, error)
Info(locator store.Locator, readonlyAge int) (store.PostInfo, error)
SetTitle(locator store.Locator, commentID string) (comment store.Comment, err error)
SetVerified(siteID string, userID string, status bool) error
SetVerified(siteID, userID string, status bool) error
SetReadOnly(locator store.Locator, status bool) error
SetPin(locator store.Locator, commentID string, status bool) error
}
// DELETE /comment/{id}?site=siteID&url=post-url - removes comment
func (a *admin) deleteCommentCtrl(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
locator := store.Locator{SiteID: r.URL.Query().Get("site"), URL: r.URL.Query().Get("url")}
log.Printf("[INFO] delete comment %s", id)
@@ -61,7 +60,6 @@ func (a *admin) deleteCommentCtrl(w http.ResponseWriter, r *http.Request) {
// DELETE /user/{userid}?site=side-id - delete all user comments for requested userid
func (a *admin) deleteUserCtrl(w http.ResponseWriter, r *http.Request) {
userID := chi.URLParam(r, "userid")
siteID := r.URL.Query().Get("site")
log.Printf("[INFO] delete all user comments for %s, site %s", userID, siteID)
@@ -77,7 +75,6 @@ func (a *admin) deleteUserCtrl(w http.ResponseWriter, r *http.Request) {
// GET /user/{userid}?site=side-id - get user info for requested userid
func (a *admin) getUserInfoCtrl(w http.ResponseWriter, r *http.Request) {
userID := chi.URLParam(r, "userid")
siteID := r.URL.Query().Get("site")
log.Printf("[INFO] get user info for %s, site %s", userID, siteID)
@@ -94,7 +91,6 @@ func (a *admin) getUserInfoCtrl(w http.ResponseWriter, r *http.Request) {
// GET /deleteme?token=jwt - delete all user comments and details by user's request. Gets info about deleted used from provided token
// request made GET to allow direct click from the email sent by user
func (a *admin) deleteMeRequestCtrl(w http.ResponseWriter, r *http.Request) {
token := r.URL.Query().Get("token")
claims, err := a.authenticator.TokenService().Parse(token)
@@ -107,7 +103,7 @@ func (a *admin) deleteMeRequestCtrl(w http.ResponseWriter, r *http.Request) {
// deleteme set by deleteMeCtrl, this check just to make sure we not trying to delete with leaked token
if !claims.User.BoolAttr("delete_me") {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("forbidden"), "can't use provided token", rest.ErrNoAccess)
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("forbidden"), "can't use provided token", rest.ErrNoAccess)
return
}
@@ -187,7 +183,7 @@ func (a *admin) setReadOnlyCtrl(w http.ResponseWriter, r *http.Request) {
// don't allow to reset ro for posts turned to ro by ReadOnlyAge
if !roStatus {
if info, e := a.dataService.Info(locator, a.readOnlyAge); e == nil && isRoByAge(info) {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"),
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"),
"read-only due the age", rest.ErrActionRejected)
return
}
+48 -47
View File
@@ -38,7 +38,7 @@ func TestAdmin_Delete(t *testing.T) {
// check last comments
res, code := get(t, ts.URL+"/api/v1/last/2?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := []store.Comment{}
err := json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -65,10 +65,10 @@ func TestAdmin_Delete(t *testing.T) {
resp, err = sendReq(t, req, adminUmputunToken)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, code := getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr := store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -78,7 +78,7 @@ func TestAdmin_Delete(t *testing.T) {
time.Sleep(250 * time.Millisecond)
// check last comments updated
res, code = get(t, ts.URL+"/api/v1/last/2?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = []store.Comment{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -86,7 +86,7 @@ func TestAdmin_Delete(t *testing.T) {
// check count updated
res, code = get(t, ts.URL+"/api/v1/count?site=remark42&url=https://radio-t.com/blah")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
b := map[string]interface{}{}
err = json.Unmarshal([]byte(res), &b)
assert.NoError(t, err)
@@ -142,10 +142,10 @@ func TestAdmin_Title(t *testing.T) {
resp, err := sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, code := get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=%s/post1", ts.URL, id1, tss.URL))
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
cr := store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -177,11 +177,11 @@ func TestAdmin_DeleteUser(t *testing.T) {
resp, err := sendReq(t, req, adminUmputunToken)
assert.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
// all 3 comments here, but for id2 they deleted
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cmntWithInfo := commentsWithInfo{}
err = json.Unmarshal([]byte(res), &cmntWithInfo)
assert.NoError(t, err)
@@ -219,6 +219,7 @@ func TestAdmin_Pin(t *testing.T) {
pin := func(val int) int {
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut,
fmt.Sprintf("%s/api/v1/admin/pin/%s?site=remark42&url=https://radio-t.com/blah&pin=%d", ts.URL, id1, val), http.NoBody)
assert.NoError(t, err)
@@ -231,19 +232,19 @@ func TestAdmin_Pin(t *testing.T) {
}
code := pin(1)
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
body, code := get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr := store.Comment{}
err := json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
assert.True(t, cr.Pin)
code = pin(-1)
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
body, code = get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr = store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -286,7 +287,7 @@ func TestAdmin_Block(t *testing.T) {
// block permanently
code, body := block(1, "")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
j := R.JSON{}
err := json.Unmarshal(body, &j)
assert.NoError(t, err)
@@ -299,7 +300,7 @@ func TestAdmin_Block(t *testing.T) {
// get last to confirm one comment deleted
bodyStr, code := get(t, ts.URL+"/api/v1/last/10?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
pi := []store.PostInfo{}
assert.NoError(t, json.Unmarshal([]byte(bodyStr), &pi))
assert.Equal(t, 1, len(pi), "last status updated, one comment left")
@@ -317,7 +318,7 @@ func TestAdmin_Block(t *testing.T) {
assert.Equal(t, []store.PostInfo{{URL: "https://radio-t.com/blah", Count: 1}}, pi)
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -327,7 +328,7 @@ func TestAdmin_Block(t *testing.T) {
// unblock
code, body = block(-1, "")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
err = json.Unmarshal(body, &j)
assert.NoError(t, err)
assert.Equal(t, false, j["block"])
@@ -335,11 +336,11 @@ func TestAdmin_Block(t *testing.T) {
// block with ttl
makeTwoComments()
code, _ = block(1, "50ms")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
// get as regular user
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -351,7 +352,7 @@ func TestAdmin_Block(t *testing.T) {
// time
time.Sleep(50 * time.Millisecond)
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -385,7 +386,7 @@ func TestAdmin_BlockedList(t *testing.T) {
res, err := sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, res.Body.Close())
assert.Equal(t, 200, res.StatusCode)
assert.Equal(t, http.StatusOK, res.StatusCode)
// block user2
req, err = http.NewRequest(http.MethodPut,
@@ -394,13 +395,13 @@ func TestAdmin_BlockedList(t *testing.T) {
res, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, res.Body.Close())
assert.Equal(t, 200, res.StatusCode)
assert.Equal(t, http.StatusOK, res.StatusCode)
req, err = http.NewRequest("GET", ts.URL+"/api/v1/admin/blocked?site=remark42", http.NoBody)
require.NoError(t, err)
res, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.Equal(t, 200, res.StatusCode)
require.Equal(t, http.StatusOK, res.StatusCode)
users := []store.BlockedUser{}
err = json.NewDecoder(res.Body).Decode(&users)
assert.NoError(t, err)
@@ -417,7 +418,7 @@ func TestAdmin_BlockedList(t *testing.T) {
require.NoError(t, err)
res, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.Equal(t, 200, res.StatusCode)
require.Equal(t, http.StatusOK, res.StatusCode)
users = []store.BlockedUser{}
err = json.NewDecoder(res.Body).Decode(&users)
assert.NoError(t, err)
@@ -450,11 +451,11 @@ func TestAdmin_ReadOnly(t *testing.T) {
resp, err := sendReq(t, req, "") // non-admin user
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 401, resp.StatusCode)
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
resp, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
assert.NoError(t, err)
assert.True(t, info.ReadOnly)
@@ -478,7 +479,7 @@ func TestAdmin_ReadOnly(t *testing.T) {
resp, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
assert.NoError(t, err)
assert.False(t, info.ReadOnly)
@@ -508,12 +509,12 @@ func TestAdmin_ReadOnlyNoComments(t *testing.T) {
resp, err := sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
_, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
assert.Error(t, err)
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&format=tree")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -544,7 +545,7 @@ func TestAdmin_ReadOnlyWithAge(t *testing.T) {
resp, err := sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
assert.NoError(t, err)
assert.True(t, info.ReadOnly)
@@ -556,11 +557,10 @@ func TestAdmin_ReadOnlyWithAge(t *testing.T) {
resp, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 403, resp.StatusCode)
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
assert.NoError(t, err)
assert.True(t, info.ReadOnly)
}
func TestAdmin_Verify(t *testing.T) {
ts, srv, teardown := startupT(t)
@@ -586,12 +586,12 @@ func TestAdmin_Verify(t *testing.T) {
resp, err := sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
verified = srv.DataService.IsVerified("remark42", "user1")
assert.True(t, verified)
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -605,12 +605,12 @@ func TestAdmin_Verify(t *testing.T) {
resp, err = sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
verified = srv.DataService.IsVerified("remark42", "user1")
assert.False(t, verified)
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -632,7 +632,7 @@ func TestAdmin_ExportStream(t *testing.T) {
addComment(t, c2, ts)
body, code := getWithAdminAuth(t, ts.URL+"/api/v1/admin/export?site=remark42&mode=stream")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
assert.Equal(t, 3, strings.Count(body, "\n"))
assert.Equal(t, 2, strings.Count(body, "\"text\""))
t.Logf("%s", body)
@@ -656,7 +656,7 @@ func TestAdmin_ExportFile(t *testing.T) {
resp, err := sendReq(t, req, adminUmputunToken)
require.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
assert.Equal(t, "application/gzip", resp.Header.Get("Content-Type"))
ungzReader, err := gzip.NewReader(resp.Body)
@@ -720,6 +720,7 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
assert.NoError(t, err)
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, tkn), http.NoBody)
assert.NoError(t, err)
@@ -727,7 +728,7 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
resp, err := client.Do(req)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
_, err = srv.DataService.User("remark42", "user1", 0, 0, store.User{})
assert.EqualError(t, err, "no comments for user user1 in store")
@@ -735,7 +736,6 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
email, err = srv.DataService.GetUserEmail("remark42", "user1")
assert.NoError(t, err)
assert.Empty(t, email, "user1 email was deleted")
}
func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
@@ -754,13 +754,14 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
// try with bad token
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, "bad token"), http.NoBody)
assert.NoError(t, err)
req.SetBasicAuth("admin", "password")
resp, err := client.Do(req)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, 400, resp.StatusCode)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
// try with bad auth
claims := token.Claims{
@@ -788,7 +789,7 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
resp, err = client.Do(req)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, 403, resp.StatusCode)
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
// try bad user
badClaims := claims
@@ -801,7 +802,7 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
resp, err = client.Do(req)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, 400, resp.StatusCode, resp.Status)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode, resp.Status)
// try without deleteme flag
badClaims2 := claims
@@ -813,7 +814,7 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
req.SetBasicAuth("admin", "password")
resp, err = client.Do(req)
assert.NoError(t, err)
assert.Equal(t, 403, resp.StatusCode)
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
b, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
@@ -836,7 +837,7 @@ func TestAdmin_GetUserInfo(t *testing.T) {
body, code := getWithAdminAuth(t, fmt.Sprintf("%s/api/v1/admin/user/user1?site=remark42&url=https://radio-t.com/blah",
ts.URL))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
u := store.User{}
err = json.Unmarshal([]byte(body), &u)
assert.NoError(t, err)
@@ -844,8 +845,8 @@ func TestAdmin_GetUserInfo(t *testing.T) {
Admin: false, Blocked: false, Verified: false}, u)
_, code = get(t, fmt.Sprintf("%s/api/v1/admin/user/user1?site=remark42&url=https://radio-t.com/blah", ts.URL))
assert.Equal(t, 401, code, "no auth")
assert.Equal(t, http.StatusUnauthorized, code, "no auth")
_, code = getWithAdminAuth(t, fmt.Sprintf("%s/api/v1/admin/user/userX?site=remark42&url=https://radio-t.com/blah", ts.URL))
assert.Equal(t, 400, code, "no info about user")
assert.Equal(t, http.StatusBadRequest, code, "no info about user")
}
+8 -12
View File
@@ -5,7 +5,6 @@ import (
"context"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"sync"
@@ -15,7 +14,6 @@ import (
cache "github.com/go-pkgz/lcw"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/migrator"
"github.com/umputun/remark42/backend/app/rest"
@@ -44,11 +42,10 @@ type KeyStore interface {
// POST /import?secret=key&site=site-id&provider=disqus|remark|wordpress
// imports comments from post body.
func (m *Migrator) importCtrl(w http.ResponseWriter, r *http.Request) {
siteID := r.URL.Query().Get("site")
if m.isBusy(siteID) {
rest.SendErrorJSON(w, r, http.StatusConflict, errors.New("already running"),
rest.SendErrorJSON(w, r, http.StatusConflict, fmt.Errorf("already running"),
"import rejected", rest.ErrActionRejected)
return
}
@@ -71,7 +68,7 @@ func (m *Migrator) importFormCtrl(w http.ResponseWriter, r *http.Request) {
siteID := r.URL.Query().Get("site")
if m.isBusy(siteID) {
rest.SendErrorJSON(w, r, http.StatusConflict, errors.New("already running"),
rest.SendErrorJSON(w, r, http.StatusConflict, fmt.Errorf("already running"),
"import rejected", rest.ErrActionRejected)
return
}
@@ -132,7 +129,6 @@ func (m *Migrator) waitCtrl(w http.ResponseWriter, r *http.Request) {
// GET /export?site=site-id&secret=12345&?mode=file|stream
// exports all comments for siteID as gz file
func (m *Migrator) exportCtrl(w http.ResponseWriter, r *http.Request) {
siteID := r.URL.Query().Get("site")
var writer io.Writer = w
@@ -167,7 +163,7 @@ func (m *Migrator) remapCtrl(w http.ResponseWriter, r *http.Request) {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "remap failed, bad given rules", rest.ErrDecode)
return
}
defer r.Body.Close()
defer r.Body.Close() //nolint gosec // we don't care about response body
// start remap procedure with mapper
go func() {
@@ -175,7 +171,7 @@ func (m *Migrator) remapCtrl(w http.ResponseWriter, r *http.Request) {
defer m.setBusy(siteID, false)
// do export
fh, e := ioutil.TempFile("", "remark42_convert")
fh, e := os.CreateTemp("", "remark42_convert")
if e != nil {
log.Printf("[WARN] failed to make temp file %+v", e)
return
@@ -253,17 +249,17 @@ func (m *Migrator) runImport(siteID, provider, tmpfile string) {
// saveTemp reads from reader and saves to temp file
func (m *Migrator) saveTemp(r io.Reader) (string, error) {
tmpfile, err := ioutil.TempFile("", "remark42_import")
tmpfile, err := os.CreateTemp("", "remark42_import")
if err != nil {
return "", errors.Wrap(err, "can't make temp file")
return "", fmt.Errorf("can't make temp file: %w", err)
}
if _, err = io.Copy(tmpfile, r); err != nil {
return "", errors.Wrap(err, "can't copy to temp file")
return "", fmt.Errorf("can't copy to temp file: %w", err)
}
if err = tmpfile.Close(); err != nil {
return "", errors.Wrap(err, "can't close temp file")
return "", fmt.Errorf("can't close temp file: %w", err)
}
return tmpfile.Name(), nil
+20 -9
View File
@@ -34,6 +34,7 @@ func TestMigrator_Import(t *testing.T) {
"votes":{},"time":"2018-04-30T01:37:00.861387771-05:00"}`)
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -92,6 +93,7 @@ func TestMigrator_ImportFromWP(t *testing.T) {
r := strings.NewReader(strings.Replace(xmlTestWP, "'", "`", -1))
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=wordpress", r)
assert.NoError(t, err)
req.Header.Add("Content-Type", "application/xml; charset=utf-8")
@@ -120,6 +122,7 @@ func TestMigrator_ImportFromCommento(t *testing.T) {
"provider":"sso:example.com","joinDate":"2021-03-19T19:27:25.954285Z","isModerator":false}]}`)
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=commento", r)
assert.NoError(t, err)
req.Header.Add("Content-Type", "application/json; charset=utf-8")
@@ -150,6 +153,7 @@ func TestMigrator_ImportRejected(t *testing.T) {
"votes":{},"time":"2018-04-30T01:37:00.861387771-05:00"}`)
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native&secret=XYZ", r)
assert.NoError(t, err)
resp, err := client.Do(req)
@@ -172,6 +176,7 @@ func TestMigrator_ImportDouble(t *testing.T) {
}
r := strings.NewReader(`{"version":1}` + strings.Join(recs, "\n")) // reader with 10k records
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -182,6 +187,7 @@ func TestMigrator_ImportDouble(t *testing.T) {
assert.Equal(t, http.StatusAccepted, resp.StatusCode)
client = &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err = http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -208,6 +214,7 @@ func TestMigrator_ImportWaitExpired(t *testing.T) {
}
r := strings.NewReader(`{"version":1}` + strings.Join(recs, "\n")) // reader with `nRecs` records
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -218,6 +225,7 @@ func TestMigrator_ImportWaitExpired(t *testing.T) {
assert.Equal(t, http.StatusAccepted, resp.StatusCode)
client = &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err = http.NewRequest("GET", ts.URL+"/api/v1/admin/wait?site=remark42&timeout=5ms", http.NoBody)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -245,6 +253,7 @@ func TestMigrator_Export(t *testing.T) {
// import comments first
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -260,7 +269,7 @@ func TestMigrator_Export(t *testing.T) {
req.SetBasicAuth("admin", "password")
resp, err = client.Do(req)
require.NoError(t, err)
require.Equal(t, 200, resp.StatusCode)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "application/gzip", resp.Header.Get("Content-Type"))
ungzReader, err := gzip.NewReader(resp.Body)
@@ -278,7 +287,7 @@ func TestMigrator_Export(t *testing.T) {
req.SetBasicAuth("admin", "password")
resp, err = client.Do(req)
require.NoError(t, err)
require.Equal(t, 200, resp.StatusCode)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "text/plain; charset=utf-8", resp.Header.Get("Content-Type"))
body, err := io.ReadAll(resp.Body)
@@ -325,7 +334,7 @@ func TestMigrator_Remap(t *testing.T) {
// check that comments created as expected
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo/")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
comments := commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
require.NoError(t, err)
@@ -333,7 +342,7 @@ func TestMigrator_Remap(t *testing.T) {
require.False(t, comments.Info.ReadOnly)
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo-another/")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
require.NoError(t, err)
@@ -350,7 +359,7 @@ func TestMigrator_Remap(t *testing.T) {
// after remap finished we should find comments from new urls
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://www.remark42.com/demo/")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
require.NoError(t, err)
@@ -358,7 +367,7 @@ func TestMigrator_Remap(t *testing.T) {
require.False(t, comments.Info.ReadOnly)
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://www.remark42.com/demo-another/")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
require.NoError(t, err)
@@ -367,14 +376,14 @@ func TestMigrator_Remap(t *testing.T) {
// should find nothing from previous url
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo/")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
require.NoError(t, err)
require.Equal(t, 0, comments.Info.Count)
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo-another/")
require.Equal(t, 200, code)
require.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
require.NoError(t, err)
@@ -387,6 +396,7 @@ func TestMigrator_RemapReject(t *testing.T) {
// without admin credentials
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
rules := strings.NewReader(`https://remark42.com/* https://www.remark42.com/*`)
req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/remap?site=remark42", rules)
require.NoError(t, err)
@@ -398,12 +408,13 @@ func TestMigrator_RemapReject(t *testing.T) {
func waitForMigrationCompletion(t *testing.T, ts *httptest.Server) {
client := &http.Client{Timeout: 10 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("GET", ts.URL+"/api/v1/admin/wait?site=remark42", http.NoBody)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
resp, err := client.Do(req)
require.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
b, err := io.ReadAll(resp.Body)
require.NoError(t, err)
defer resp.Body.Close()
+122 -84
View File
@@ -3,14 +3,19 @@ package api
import (
"bytes"
"context"
"embed"
"encoding/json"
"fmt"
"io/fs"
"net/http"
"net/mail"
"os"
"regexp"
"strings"
"sync"
"time"
"github.com/didip/tollbooth/v6"
"github.com/didip/tollbooth/v7"
"github.com/didip/tollbooth_chi"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
@@ -21,8 +26,6 @@ import (
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/go-pkgz/rest/logger"
"github.com/pkg/errors"
"github.com/rakyll/statik/fs"
"github.com/umputun/remark42/backend/app/notify"
"github.com/umputun/remark42/backend/app/rest"
@@ -30,7 +33,6 @@ import (
"github.com/umputun/remark42/backend/app/store"
"github.com/umputun/remark42/backend/app/store/image"
"github.com/umputun/remark42/backend/app/store/service"
"github.com/umputun/remark42/backend/app/templates"
)
// Rest is a rest access server
@@ -49,6 +51,7 @@ type Rest struct {
AnonVote bool
WebRoot string
WebFS embed.FS
RemarkURL string
ReadOnlyAge int
SharedSecret string
@@ -56,15 +59,16 @@ type Rest struct {
Low int
Critical int
}
UpdateLimiter float64
EmailNotifications bool
TelegramBotUsername string
EmojiEnabled bool
SimpleView bool
ProxyCORS bool
SendJWTHeader bool
AllowedAncestors []string // sets Content-Security-Policy "frame-ancestors ..."
SubscribersOnly bool
UpdateLimiter float64
EmailNotifications bool
TelegramNotifications bool
EmojiEnabled bool
SimpleView bool
ProxyCORS bool
SendJWTHeader bool
AllowedAncestors []string // sets Content-Security-Policy "frame-ancestors ..."
SubscribersOnly bool
DisableSignature bool // prevent signature from being added to headers
SSLConfig SSLConfig
httpsServer *http.Server
@@ -95,7 +99,6 @@ type commentsWithInfo struct {
// Run the lister and request's router, activate rest server
func (s *Rest) Run(address string, port int) {
if address == "*" {
address = ""
}
@@ -191,7 +194,10 @@ func (s *Rest) makeHTTPServer(address string, port int, router http.Handler) *ht
func (s *Rest) routes() chi.Router {
router := chi.NewRouter()
router.Use(middleware.Throttle(1000), middleware.RealIP, R.Recoverer(log.Default()))
router.Use(R.AppInfo("remark42", "umputun", s.Version), R.Ping)
if !s.DisableSignature {
router.Use(R.AppInfo("remark42", "umputun", s.Version))
}
router.Use(R.Ping)
s.pubRest, s.privRest, s.adminRest, s.rssRest = s.controllerGroups() // assign controllers for groups
@@ -221,7 +227,8 @@ func (s *Rest) routes() chi.Router {
router.Group(func(r chi.Router) {
r.Use(middleware.Timeout(5 * time.Second))
r.Use(logInfoWithBody, tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)), middleware.NoCache)
r.Use(logInfoWithBody, tollbooth_chi.LimitHandler(tollbooth.NewLimiter(2, nil)), middleware.NoCache)
r.Use(validEmailAuth()) // reject suspicious email logins
r.Mount("/auth", authHandler)
})
@@ -235,7 +242,6 @@ func (s *Rest) routes() chi.Router {
// api routes
router.Route("/api/v1", func(rapi chi.Router) {
rapi.Group(func(rava chi.Router) {
rava.Use(middleware.Timeout(5 * time.Second))
rava.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(100, nil)))
@@ -263,7 +269,6 @@ func (s *Rest) routes() chi.Router {
rrss.Get("/site", s.rssRest.siteCommentsCtrl)
rrss.Get("/reply", s.rssRest.repliesCtrl)
})
})
// open routes, cached
@@ -272,6 +277,7 @@ func (s *Rest) routes() chi.Router {
ropen.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
ropen.Use(authMiddleware.Trace, logInfoWithBody)
ropen.Get("/picture/{user}/{id}", s.pubRest.loadPictureCtrl)
ropen.Get("/qr/telegram", s.pubRest.telegramQrCtrl)
})
// protected routes, require auth
@@ -337,33 +343,29 @@ func (s *Rest) routes() chi.Router {
rauth.Use(logger.New(logger.Log(log.Default()), logger.Prefix("[DEBUG]"), logger.IPfn(ipFn)).Handler)
rauth.Post("/picture", s.privRest.savePictureCtrl)
})
})
// open routes on root level
router.Group(func(rroot chi.Router) {
rroot.Use(middleware.Timeout(10 * time.Second))
rroot.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(50, nil)))
rroot.Get("/index.html", s.pubRest.getStartedCtrl)
rroot.Get("/robots.txt", s.pubRest.robotsCtrl)
rroot.Get("/email/unsubscribe.html", s.privRest.emailUnsubscribeCtrl)
rroot.Post("/email/unsubscribe.html", s.privRest.emailUnsubscribeCtrl)
})
// file server for static content from /web
addFileServer(router, "/web", http.Dir(s.WebRoot), s.Version)
// file server for static content from s.WebRoot on path /web
addFileServer(router, s.WebFS, s.WebRoot, s.Version)
return router
}
func (s *Rest) controllerGroups() (public, private, admin, rss) {
pubGrp := public{
dataService: s.DataService,
cache: s.Cache,
imageService: s.ImageService,
commentFormatter: s.CommentFormatter,
readOnlyAge: s.ReadOnlyAge,
webRoot: s.WebRoot,
}
privGrp := private{
@@ -377,7 +379,6 @@ func (s *Rest) controllerGroups() (public, private, admin, rss) {
telegramService: s.TelegramService,
remarkURL: s.RemarkURL,
anonVote: s.AnonVote,
templates: templates.NewFS(),
}
admGrp := admin{
@@ -413,44 +414,44 @@ func (s *Rest) configCtrl(w http.ResponseWriter, r *http.Request) {
emails, _ := s.DataService.AdminStore.Email(siteID)
cnf := struct {
Version string `json:"version"`
EditDuration int `json:"edit_duration"`
AdminEdit bool `json:"admin_edit"`
MaxCommentSize int `json:"max_comment_size"`
Admins []string `json:"admins"`
AdminEmail string `json:"admin_email"`
Auth []string `json:"auth_providers"`
AnonVote bool `json:"anon_vote"`
LowScore int `json:"low_score"`
CriticalScore int `json:"critical_score"`
PositiveScore bool `json:"positive_score"`
ReadOnlyAge int `json:"readonly_age"`
MaxImageSize int `json:"max_image_size"`
EmailNotifications bool `json:"email_notifications"`
TelegramBotUsername string `json:"telegram_bot_username"`
EmojiEnabled bool `json:"emoji_enabled"`
SimpleView bool `json:"simple_view"`
SendJWTHeader bool `json:"send_jwt_header"`
SubscribersOnly bool `json:"subscribers_only"`
Version string `json:"version"`
EditDuration int `json:"edit_duration"`
AdminEdit bool `json:"admin_edit"`
MaxCommentSize int `json:"max_comment_size"`
Admins []string `json:"admins"`
AdminEmail string `json:"admin_email"`
Auth []string `json:"auth_providers"`
AnonVote bool `json:"anon_vote"`
LowScore int `json:"low_score"`
CriticalScore int `json:"critical_score"`
PositiveScore bool `json:"positive_score"`
ReadOnlyAge int `json:"readonly_age"`
MaxImageSize int `json:"max_image_size"`
EmailNotifications bool `json:"email_notifications"`
TelegramNotifications bool `json:"telegram_notifications"`
EmojiEnabled bool `json:"emoji_enabled"`
SimpleView bool `json:"simple_view"`
SendJWTHeader bool `json:"send_jwt_header"`
SubscribersOnly bool `json:"subscribers_only"`
}{
Version: s.Version,
EditDuration: int(s.DataService.EditDuration.Seconds()),
AdminEdit: s.DataService.AdminEdits,
MaxCommentSize: s.DataService.MaxCommentSize,
Admins: admins,
AdminEmail: emails,
LowScore: s.ScoreThresholds.Low,
CriticalScore: s.ScoreThresholds.Critical,
PositiveScore: s.DataService.PositiveScore,
ReadOnlyAge: s.ReadOnlyAge,
MaxImageSize: s.ImageService.MaxSize,
EmailNotifications: s.EmailNotifications,
TelegramBotUsername: s.TelegramBotUsername,
EmojiEnabled: s.EmojiEnabled,
AnonVote: s.AnonVote,
SimpleView: s.SimpleView,
SendJWTHeader: s.SendJWTHeader,
SubscribersOnly: s.SubscribersOnly,
Version: s.Version,
EditDuration: int(s.DataService.EditDuration.Seconds()),
AdminEdit: s.DataService.AdminEdits,
MaxCommentSize: s.DataService.MaxCommentSize,
Admins: admins,
AdminEmail: emails,
LowScore: s.ScoreThresholds.Low,
CriticalScore: s.ScoreThresholds.Critical,
PositiveScore: s.DataService.PositiveScore,
ReadOnlyAge: s.ReadOnlyAge,
MaxImageSize: s.ImageService.MaxSize,
EmailNotifications: s.EmailNotifications,
TelegramNotifications: s.TelegramNotifications,
EmojiEnabled: s.EmojiEnabled,
AnonVote: s.AnonVote,
SimpleView: s.SimpleView,
SendJWTHeader: s.SendJWTHeader,
SubscribersOnly: s.SubscribersOnly,
}
cnf.Auth = []string{}
@@ -465,35 +466,28 @@ func (s *Rest) configCtrl(w http.ResponseWriter, r *http.Request) {
render.JSON(w, r, cnf)
}
// serves static files from /web or embedded by statik
func addFileServer(r chi.Router, path string, root http.FileSystem, version string) {
// serves static files from the webRoot directory or files embedded into the compiled binary if that directory is absent
func addFileServer(r chi.Router, embedFS embed.FS, webRoot, version string) {
var webFS http.Handler
statikFS, err := fs.New()
if err != nil {
log.Printf("[DEBUG] no embedded assets loaded, %s", err)
log.Printf("[INFO] run file server for %s, path %s", root, path)
webFS = http.FileServer(root)
if _, err := os.Stat(webRoot); err == nil {
log.Printf("[INFO] run file server from %s from the disk", webRoot)
webFS = http.FileServer(http.Dir(webRoot))
} else {
log.Printf("[INFO] run file server for %s, embedded", root)
webFS = http.FileServer(statikFS)
log.Printf("[INFO] run file server, embedded")
var contentFS, _ = fs.Sub(embedFS, "web")
webFS = http.FileServer(http.FS(contentFS))
}
origPath := path
webFS = http.StripPrefix(path, webFS)
if path != "/" && path[len(path)-1] != '/' {
r.Get(path, http.RedirectHandler(path+"/", http.StatusMovedPermanently).ServeHTTP)
path += "/"
}
path += "*"
webFS = http.StripPrefix("/web", webFS)
r.Get("/web", http.RedirectHandler("/web/", http.StatusMovedPermanently).ServeHTTP)
r.With(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(20, nil)),
middleware.Timeout(10*time.Second),
cacheControl(time.Hour, version),
).Get(path, func(w http.ResponseWriter, r *http.Request) {
).Get("/web/*", func(w http.ResponseWriter, r *http.Request) {
// don't show dirs, just serve files
if strings.HasSuffix(r.URL.Path, "/") && len(r.URL.Path) > 1 && r.URL.Path != (origPath+"/") {
if strings.HasSuffix(r.URL.Path, "/") && len(r.URL.Path) > 1 && r.URL.Path != ("/web/") {
http.NotFound(w, r)
return
}
@@ -506,7 +500,7 @@ func encodeJSONWithHTML(v interface{}) ([]byte, error) {
enc := json.NewEncoder(buf)
enc.SetEscapeHTML(false)
if err := enc.Encode(v); err != nil {
return nil, errors.Wrap(err, "json encoding failed")
return nil, fmt.Errorf("json encoding failed: %w", err)
}
return buf.Bytes(), nil
}
@@ -592,7 +586,6 @@ func matchSiteID(next http.Handler) http.Handler {
// cacheControl is a middleware setting cache expiration. Using url+version as etag
func cacheControl(expiration time.Duration, version string) func(http.Handler) http.Handler {
etag := func(r *http.Request, version string) string {
s := version + ":" + r.URL.String()
return store.EncodeID(s)
@@ -653,6 +646,52 @@ func subscribersOnly(enable bool) func(http.Handler) http.Handler {
}
}
// validEmailAuth is a middleware for auth endpoints for email method.
// it rejects login request if user, site or email are suspicious
func validEmailAuth() func(http.Handler) http.Handler {
reUser := regexp.MustCompile(`^[\p{L}\d\s_]{4,64}$`) // matches ui side validation, adding min/max limitation
reSite := regexp.MustCompile(`^[a-zA-Z\d\s_.-]{1,64}$`)
return func(h http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/auth/email/login" {
// not email login, skip the check
h.ServeHTTP(w, r)
return
}
if u := r.URL.Query().Get("user"); u != "" {
if !reUser.MatchString(u) {
log.Printf("[WARN] suspicious user rejected: %s", u)
http.Error(w, "Access denied", http.StatusForbidden)
return
}
}
if a := r.URL.Query().Get("address"); a != "" {
if _, err := mail.ParseAddress(a); err != nil {
log.Printf("[WARN] suspicious address rejected: %s", a)
http.Error(w, "Access denied", http.StatusForbidden)
return
}
}
if s := r.URL.Query().Get("site"); s != "" {
if !reSite.MatchString(s) {
log.Printf("[WARN] suspicious site rejected: %s", s)
http.Error(w, "Access denied", http.StatusForbidden)
return
}
}
h.ServeHTTP(w, r)
}
return http.HandlerFunc(fn)
}
}
func parseError(err error, defaultCode int) (code int) {
code = defaultCode
@@ -672,7 +711,6 @@ func parseError(err error, defaultCode int) (code int) {
code = rest.ErrCommentEditExpired
case strings.HasPrefix(err.Error(), "parent comment with reply can't be edited"):
code = rest.ErrCommentEditChanged
}
return code
+108 -73
View File
@@ -22,7 +22,6 @@ import (
R "github.com/go-pkgz/rest"
"github.com/golang-jwt/jwt"
"github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/notify"
"github.com/umputun/remark42/backend/app/rest"
@@ -44,7 +43,6 @@ type private struct {
telegramService telegramService
remarkURL string
anonVote bool
templates templates.FileReader
}
// telegramService is a subset of Telegram service used for setting up user telegram notifications
@@ -60,15 +58,15 @@ type privStore interface {
Vote(req service.VoteReq) (comment store.Comment, err error)
Get(locator store.Locator, commentID string, user store.User) (store.Comment, error)
User(siteID, userID string, limit, skip int, user store.User) ([]store.Comment, error)
GetUserEmail(siteID string, userID string) (string, error)
SetUserEmail(siteID string, userID string, value string) (string, error)
GetUserTelegram(siteID string, userID string) (string, error)
SetUserTelegram(siteID string, userID string, value string) (string, error)
DeleteUserDetail(siteID string, userID string, detail engine.UserDetail) error
GetUserEmail(siteID, userID string) (string, error)
SetUserEmail(siteID, userID, value string) (string, error)
GetUserTelegram(siteID, userID string) (string, error)
SetUserTelegram(siteID, userID, value string) (string, error)
DeleteUserDetail(siteID, userID string, detail engine.UserDetail) error
ValidateComment(c *store.Comment) error
IsVerified(siteID string, userID string) bool
IsVerified(siteID, userID string) bool
IsReadOnly(locator store.Locator) bool
IsBlocked(siteID string, userID string) bool
IsBlocked(siteID, userID string) bool
Info(locator store.Locator, readonlyAge int) (store.PostInfo, error)
}
@@ -92,11 +90,11 @@ func (s *private) previewCommentCtrl(w http.ResponseWriter, r *http.Request) {
comment = s.commentFormatter.Format(comment)
comment.Sanitize()
// check if images are valid
for _, id := range s.imageService.ExtractPictures(comment.Text) {
// check if images are valid, omit proxied images as they are lazy-loaded
for _, id := range s.imageService.ExtractNonProxiedPictures(comment.Text) {
err := s.imageService.ResetCleanupTimer(id)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't renew staged picture cleanup timer", rest.ErrImgNotFound)
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't load picture from the comment", rest.ErrImgNotFound)
return
}
}
@@ -106,7 +104,6 @@ func (s *private) previewCommentCtrl(w http.ResponseWriter, r *http.Request) {
// POST /comment - adds comment, resets all immutable fields
func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
comment := store.Comment{}
if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &comment); err != nil {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't bind comment", rest.ErrDecode)
@@ -132,8 +129,8 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
}
comment = s.commentFormatter.Format(comment)
// check if images are valid
for _, id := range s.imageService.ExtractPictures(comment.Text) {
// check if images are valid, omit proxied images as they are lazy-loaded
for _, id := range s.imageService.ExtractNonProxiedPictures(comment.Text) {
_, err := s.imageService.Load(id)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't load picture from the comment", rest.ErrImgNotFound)
@@ -143,12 +140,12 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
// check if user blocked
if s.dataService.IsBlocked(comment.Locator.SiteID, comment.User.ID) {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "user blocked", rest.ErrUserBlocked)
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "user blocked", rest.ErrUserBlocked)
return
}
if s.isReadOnly(comment.Locator) {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "old post, read-only", rest.ErrReadOnly)
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "old post, read-only", rest.ErrReadOnly)
return
}
@@ -175,7 +172,7 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
s.notifyService.Submit(notify.Request{Comment: finalComment})
}
log.Printf("[DEBUG] created commend %+v", finalComment)
log.Printf("[DEBUG] created comment %+v", finalComment)
render.Status(r, http.StatusCreated)
render.JSON(w, r, &finalComment)
@@ -183,7 +180,6 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
// PUT /comment/{id}?site=siteID&url=post-url - update comment
func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
edit := struct {
Text string
Summary string
@@ -191,7 +187,7 @@ func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
}{}
if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &edit); err != nil {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't bind comment", rest.ErrDecode)
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't read comment details from body", rest.ErrDecode)
return
}
@@ -209,7 +205,7 @@ func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
}
if currComment.User.ID != user.ID {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"),
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"),
"can not edit comments for other users", rest.ErrNoAccess)
return
}
@@ -270,13 +266,13 @@ func (s *private) voteCtrl(w http.ResponseWriter, r *http.Request) {
vote := r.URL.Query().Get("vote") == "1"
if s.isReadOnly(locator) {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "old post, read-only", rest.ErrReadOnly)
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "old post, read-only", rest.ErrReadOnly)
return
}
// check if user blocked
if s.dataService.IsBlocked(locator.SiteID, user.ID) {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "user blocked", rest.ErrUserBlocked)
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "user blocked", rest.ErrUserBlocked)
return
}
@@ -311,28 +307,59 @@ func (s *private) getEmailCtrl(w http.ResponseWriter, r *http.Request) {
}
// sendEmailConfirmationCtrl gets address and siteID from query, makes confirmation token and sends it to user.
// GET /email/subscribe?site=siteID&address=someone@example.com
// In case user is logged in with the same email, and auto_confirm is true, confirm it right away.
// In case of quick confirmation, "updated" is set to true, otherwise - to false.
// POST /email/subscribe with site and address in json body
//
//nolint:dupl // too hard to deduplicate that logic, as then it's tricky to use SendErrorJSON
func (s *private) sendEmailConfirmationCtrl(w http.ResponseWriter, r *http.Request) {
user := rest.MustGetUserInfo(r)
address := r.URL.Query().Get("address")
siteID := r.URL.Query().Get("site")
if address == "" {
subscribe := struct {
Site string
Address string
autoConfirm bool
}{autoConfirm: true}
if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &subscribe); err != nil {
if err != io.EOF {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't parse request body", rest.ErrDecode)
return
}
// old behavior fallback, reading from the query params. Auto confirm is false in this case.
subscribe.Address = r.URL.Query().Get("address")
subscribe.Site = r.URL.Query().Get("site")
subscribe.autoConfirm = false
}
if subscribe.Address == "" {
rest.SendErrorJSON(w, r, http.StatusBadRequest,
errors.New("missing parameter"), "address parameter is required", rest.ErrInternal)
fmt.Errorf("missing parameter"), "address parameter is required", rest.ErrInternal)
return
}
existingAddress, err := s.dataService.GetUserEmail(siteID, user.ID)
if err != nil {
log.Printf("[WARN] can't read email for %s, %v", user.ID, err)
existingAddress, getErr := s.dataService.GetUserEmail(subscribe.Site, user.ID)
if getErr != nil {
log.Printf("[WARN] can't read email for %s, %v", user.ID, getErr)
}
if address == existingAddress {
if subscribe.Address == existingAddress {
rest.SendErrorJSON(w, r, http.StatusConflict,
errors.New("already verified"), "email address is already verified for this user", rest.ErrInternal)
fmt.Errorf("already verified"), "email address is already verified for this user", rest.ErrInternal)
return
}
// in case the user logged in with the same email as they try to subscribe with, confirm it right away
// this behavior is different from the previous one and is hidden behind the autoConfirm flag,
// which is true for the new API, and false for the old one
//
// nolint:gosec // this is not used for security purposes
if subscribe.autoConfirm &&
strings.HasPrefix(user.ID, "email_") &&
strings.TrimPrefix(user.ID, "email_") == token.HashID(sha1.New(), subscribe.Address) {
s.setEmail(w, r, user.ID, subscribe.Site, subscribe.Address)
return
}
claims := token.Claims{
Handshake: &token.Handshake{ID: user.ID + "::" + address},
Handshake: &token.Handshake{ID: user.ID + "::" + subscribe.Address},
StandardClaims: jwt.StandardClaims{
Audience: r.URL.Query().Get("site"),
ExpiresAt: time.Now().Add(30 * time.Minute).Unix(),
@@ -349,14 +376,14 @@ func (s *private) sendEmailConfirmationCtrl(w http.ResponseWriter, r *http.Reque
s.notifyService.SubmitVerification(
notify.VerificationRequest{
SiteID: siteID,
SiteID: subscribe.Site,
User: user.Name,
Email: address,
Email: subscribe.Address,
Token: tkn,
},
)
render.JSON(w, r, R.JSON{"user": user, "address": address})
render.JSON(w, r, R.JSON{"user": user, "address": subscribe.Address, "updated": false})
}
// telegramSubscribeCtrl generates and verifies telegram notification request
@@ -366,7 +393,7 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
if s.telegramService == nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError,
errors.New("not enabled"), "telegram notifications are not enabled", rest.ErrActionRejected)
fmt.Errorf("not enabled"), "telegram notifications are not enabled", rest.ErrActionRejected)
return
}
@@ -375,13 +402,13 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
// GET /telegram/subscribe?site=siteID (No token supplied)
siteID := r.URL.Query().Get("site")
if siteID == "" {
rest.SendErrorJSON(w, r, http.StatusBadRequest, errors.New("missing parameter"), "site parameter is required", rest.ErrInternal)
rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "site parameter is required", rest.ErrInternal)
return
}
// we don't care as much if we can't retrieve the current value of that field for the user, so ignore the error
if existingAddress, _ := s.dataService.GetUserTelegram(siteID, user.ID); existingAddress != "" {
rest.SendErrorJSON(w, r, http.StatusConflict,
errors.New("already subscribed"), "telegram subscription is already set for this user, delete if first to re-subscribe", rest.ErrActionRejected)
fmt.Errorf("already subscribed"), "telegram subscription is already set for this user, delete if first to re-subscribe", rest.ErrActionRejected)
return
}
// Generate and send token
@@ -403,7 +430,7 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
var address, siteID string
address, siteID, err := s.telegramService.CheckToken(queryToken, user.ID)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't set telegram for user", rest.ErrInternal)
rest.SendErrorJSON(w, r, http.StatusNotFound, err, "request is not verified yet", rest.ErrInternal)
return
}
@@ -420,45 +447,60 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
}
// setConfirmedEmailCtrl uses provided token parameter (generated by sendEmailConfirmationCtrl) to set email and add it to user token
// PUT /email/confirm?site=siteID&tkn=jwt
// POST /email/confirm with site and token in json body
func (s *private) setConfirmedEmailCtrl(w http.ResponseWriter, r *http.Request) {
tkn := r.URL.Query().Get("tkn")
if tkn == "" {
rest.SendErrorJSON(w, r, http.StatusBadRequest, errors.New("missing parameter"), "token parameter is required", rest.ErrInternal)
user := rest.MustGetUserInfo(r)
confirm := struct {
Site string
Token string
}{}
if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &confirm); err != nil {
if err != io.EOF {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't parse request body", rest.ErrDecode)
return
}
// old behavior fallback, reading from the query params
confirm.Token = r.URL.Query().Get("tkn")
confirm.Site = r.URL.Query().Get("site")
}
if confirm.Token == "" {
rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "token parameter is required", rest.ErrInternal)
return
}
user := rest.MustGetUserInfo(r)
siteID := r.URL.Query().Get("site")
confClaims, err := s.authenticator.TokenService().Parse(tkn)
confClaims, err := s.authenticator.TokenService().Parse(confirm.Token)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal)
return
}
if s.authenticator.TokenService().IsExpired(confClaims) {
rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("expired"), "failed to verify confirmation token", rest.ErrInternal)
rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("expired"), "failed to verify confirmation token", rest.ErrInternal)
return
}
// Handshake.ID is user.ID + "::" + address
elems := strings.Split(confClaims.Handshake.ID, "::")
if len(elems) != 2 || elems[0] != user.ID {
rest.SendErrorJSON(w, r, http.StatusBadRequest, errors.New(confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal)
rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("%s", confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal)
return
}
address := elems[1]
s.setEmail(w, r, user.ID, confirm.Site, address)
}
log.Printf("[DEBUG] set email for user %s", user.ID)
func (s *private) setEmail(w http.ResponseWriter, r *http.Request, userID, siteID, address string) {
log.Printf("[DEBUG] set email for user %s", userID)
val, err := s.dataService.SetUserEmail(siteID, user.ID, address)
val, err := s.dataService.SetUserEmail(siteID, userID, address)
if err != nil {
code := parseError(err, rest.ErrInternal)
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't set email for user", code)
return
}
// update User.Email from the token
// update User.Email field
claims, _, err := s.authenticator.TokenService().Get(r)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal)
@@ -476,29 +518,26 @@ func (s *private) setConfirmedEmailCtrl(w http.ResponseWriter, r *http.Request)
func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
tkn := r.URL.Query().Get("tkn")
if tkn == "" {
rest.SendErrorHTML(w, r, http.StatusBadRequest,
errors.New("missing parameter"), "token parameter is required", rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "token parameter is required", rest.ErrInternal)
return
}
siteID := r.URL.Query().Get("site")
confClaims, err := s.authenticator.TokenService().Parse(tkn)
if err != nil {
rest.SendErrorHTML(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal)
return
}
if s.authenticator.TokenService().IsExpired(confClaims) {
rest.SendErrorHTML(w, r, http.StatusForbidden,
errors.New("expired"), "failed to verify confirmation token", rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusForbidden, fmt.Errorf("expired"), "failed to verify confirmation token", rest.ErrInternal)
return
}
// Handshake.ID is user.ID + "::" + address
elems := strings.Split(confClaims.Handshake.ID, "::")
if len(elems) != 2 {
rest.SendErrorHTML(w, r, http.StatusBadRequest,
errors.New(confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusBadRequest, fmt.Errorf("%s", confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal)
return
}
userID := elems[0]
@@ -511,14 +550,11 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
log.Printf("[WARN] can't read email for %s, %v", userID, err)
}
if existingAddress == "" {
rest.SendErrorHTML(w, r, http.StatusConflict,
errors.New("user is not subscribed"), "user does not have active email subscription", rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusConflict, fmt.Errorf("user is not subscribed"), "user does not have active email subscription", rest.ErrInternal)
return
}
if address != existingAddress {
rest.SendErrorHTML(w, r, http.StatusBadRequest,
errors.New("wrong email unsubscription"), "email address in request does not match known for this user",
rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusBadRequest, fmt.Errorf("wrong email unsubscription"), "email address in request does not match known for this user", rest.ErrInternal)
return
}
@@ -526,7 +562,7 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
if err = s.dataService.DeleteUserDetail(siteID, userID, engine.UserEmail); err != nil {
code := parseError(err, rest.ErrInternal)
rest.SendErrorHTML(w, r, http.StatusBadRequest, err, "can't delete email for user", code, s.templates)
rest.SendErrorHTML(w, r, http.StatusBadRequest, err, "can't delete email for user", code)
return
}
// clean User.Email from the token, if user has the token
@@ -537,7 +573,7 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
if claims.User != nil && claims.User.Email != "" {
claims.User.Email = ""
if _, err = s.authenticator.TokenService().Set(w, claims); err != nil {
rest.SendErrorHTML(w, r, http.StatusInternalServerError, err, "failed to set token", rest.ErrInternal, s.templates)
rest.SendErrorHTML(w, r, http.StatusInternalServerError, err, "failed to set token", rest.ErrInternal)
return
}
}
@@ -549,13 +585,13 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
}
}
MustRead := func(path string) string {
file, err := s.templates.ReadFile(path)
file, err := templates.Read(path)
if err != nil {
panic(err)
}
return string(file)
}
tmplstr := MustRead("unsubscribe.html.tmpl")
tmplstr := MustRead("email_unsubscribe.html.tmpl")
tmpl := template.Must(template.New("unsubscribe").Parse(tmplstr))
msg := bytes.Buffer{}
MustExecute(tmpl, &msg, nil)
@@ -657,7 +693,6 @@ func (s *private) userAllDataCtrl(w http.ResponseWriter, r *http.Request) {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, merr, "can't write user info", rest.ErrInternal)
return
}
}
// POST /deleteme?site_id=site - requesting delete of all user info
@@ -730,11 +765,11 @@ func (s *private) isReadOnly(locator store.Locator) bool {
func randToken() (string, error) {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
return "", errors.Wrap(err, "can't get random")
return "", fmt.Errorf("can't get random: %w", err)
}
s := sha1.New() //nolint:gosec // not used for security
if _, err := s.Write(b); err != nil {
return "", errors.Wrap(err, "can't write randoms to sha1")
return "", fmt.Errorf("can't write randoms to sha1: %w", err)
}
return fmt.Sprintf("%x", s.Sum(nil)), nil
}
+413 -63
View File
@@ -6,11 +6,11 @@ import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"mime/multipart"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
@@ -25,6 +25,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/umputun/remark42/backend/app/notify"
"github.com/umputun/remark42/backend/app/rest/proxy"
"github.com/umputun/remark42/backend/app/store"
"github.com/umputun/remark42/backend/app/store/image"
)
@@ -55,6 +56,111 @@ func TestRest_Create(t *testing.T) {
assert.True(t, len(c["id"].(string)) > 8)
}
// based on issue https://github.com/umputun/remark42/issues/1292
func TestRest_CreateFilteredCode(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
resp, err := post(t, ts.URL+"/api/v1/comment",
`{"text": "`+"`foo<bar>`"+`", "locator":{"url": "https://radio-t.com/blah1", "site": "remark42"}}`)
assert.NoError(t, err)
b, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
require.Equal(t, http.StatusCreated, resp.StatusCode, string(b))
assert.NoError(t, resp.Body.Close())
c := R.JSON{}
err = json.Unmarshal(b, &c)
require.NoError(t, err, string(b))
loc := c["locator"].(map[string]interface{})
assert.Equal(t, "remark42", loc["site"])
assert.Equal(t, "https://radio-t.com/blah1", loc["url"])
assert.Equal(t, "`foo<bar>`", c["orig"])
assert.Contains(t, c["text"], "foo")
assert.Contains(t, c["text"], "bar")
assert.True(t, len(c["id"].(string)) > 8)
}
// based on issue https://github.com/umputun/remark42/issues/1631
func TestRest_CreateAndPreviewWithImage(t *testing.T) {
ts, srv, teardown := startupT(t)
ts.Close()
defer teardown()
srv.ImageService.ProxyAPI = srv.RemarkURL + "/api/v1/img"
srv.ImageProxy = &proxy.Image{
HTTP2HTTPS: true,
CacheExternal: true,
RoutePath: "/api/v1/img",
RemarkURL: srv.RemarkURL,
ImageService: srv.ImageService,
}
srv.CommentFormatter = store.NewCommentFormatter(srv.ImageProxy)
// need to recreate the server with new ImageProxy, otherwise old one will be used
ts = httptest.NewServer(srv.routes())
defer ts.Close()
var pngRead bool
// server with the test PNG image
pngServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, e := io.Copy(w, gopherPNG())
assert.NoError(t, e)
pngRead = true
}))
defer pngServer.Close()
t.Run("create", func(t *testing.T) {
resp, err := post(t, ts.URL+"/api/v1/comment",
`{"text": "![](`+pngServer.URL+`/gopher.png)", "locator":{"url": "https://radio-t.com/blah1", "site": "remark42"}}`)
assert.NoError(t, err)
b, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
require.Equal(t, http.StatusCreated, resp.StatusCode, string(b))
assert.NoError(t, resp.Body.Close())
c := R.JSON{}
err = json.Unmarshal(b, &c)
require.NoError(t, err, string(b))
assert.NotContains(t, c["text"], pngServer.URL)
assert.Contains(t, c["text"], srv.RemarkURL)
loc := c["locator"].(map[string]interface{})
assert.Equal(t, "remark42", loc["site"])
assert.Equal(t, "https://radio-t.com/blah1", loc["url"])
assert.True(t, len(c["id"].(string)) > 8)
assert.Equal(t, false, pngRead, "original image is not yet accessed by server")
})
t.Run("preview", func(t *testing.T) {
resp, err := post(t, ts.URL+"/api/v1/preview",
`{"text": "![](`+pngServer.URL+`/gopher.png)", "locator":{"url": "https://radio-t.com/blah1", "site": "remark42"}}`)
assert.NoError(t, err)
b, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode, string(b))
assert.NoError(t, resp.Body.Close())
assert.NotContains(t, string(b), pngServer.URL)
assert.Contains(t, string(b), srv.RemarkURL)
assert.Equal(t, false, pngRead, "original image is not yet accessed by server")
// retrieve the image from the cache
imgURL := strings.Split(strings.Split(string(b), "src=\"")[1], "\"")[0]
// replace srv.RemarkURL with ts.URL
imgURL = strings.ReplaceAll(imgURL, srv.RemarkURL, ts.URL)
resp, err = http.Get(imgURL)
assert.NoError(t, err)
b, err = io.ReadAll(resp.Body)
assert.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode, string(b))
assert.NoError(t, resp.Body.Close())
// compare image to original gopher png after decoding from base64
assert.Equal(t, gopher, base64.StdEncoding.EncodeToString(b))
assert.Equal(t, true, pngRead, "original image accessed to be shown to user")
})
}
func TestRest_CreateOldPost(t *testing.T) {
ts, srv, teardown := startupT(t)
defer teardown()
@@ -142,8 +248,26 @@ func TestRest_CreateWithRestrictedWord(t *testing.T) {
assert.Equal(t, "invalid comment", c["details"])
}
func TestRest_CreateRejected(t *testing.T) {
func TestRest_CreateRelativeURL(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
// check that it's not possible to click insert URL button and not alter the URL in it (which is `url` by default)
relativeURLText := `{"text": "here is a link with relative URL: [google.com](url)", "locator":{"url": "https://radio-t.com/blah1", "site": "remark42"}}`
resp, err := post(t, ts.URL+"/api/v1/comment", relativeURLText)
assert.NoError(t, err)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
b, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
c := R.JSON{}
err = json.Unmarshal(b, &c)
assert.NoError(t, err)
assert.Equal(t, "links should start with mailto:, http:// or https://", c["error"])
assert.Equal(t, "invalid comment", c["details"])
}
func TestRest_CreateRejected(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
body := `{"text": "test 123", "locator":{"url": "https://radio-t.com/blah1", "site": "remark42"}}`
@@ -152,10 +276,11 @@ func TestRest_CreateRejected(t *testing.T) {
resp, err := http.Post(ts.URL+"/api/v1/comment", "", strings.NewReader(body))
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 401, resp.StatusCode)
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
// try with wrong aud
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/comment", strings.NewReader(body))
require.NoError(t, err)
req.Header.Add("X-JWT", devTokenBadAud)
@@ -224,7 +349,7 @@ func TestRest_CreateAndGet(t *testing.T) {
// get created comment by id as admin
res, code := getWithAdminAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah1", ts.URL, id))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comment := store.Comment{}
err = json.Unmarshal([]byte(res), &comment)
assert.NoError(t, err)
@@ -236,7 +361,7 @@ func TestRest_CreateAndGet(t *testing.T) {
// get created comment by id as non-admin
res, code = getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah1", ts.URL, id))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comment = store.Comment{}
err = json.Unmarshal([]byte(res), &comment)
assert.NoError(t, err)
@@ -252,6 +377,7 @@ func TestRest_Update(t *testing.T) {
id := addComment(t, c1, ts)
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"text":"updated text", "summary":"my edit"}`))
assert.NoError(t, err)
@@ -260,7 +386,7 @@ func TestRest_Update(t *testing.T) {
assert.NoError(t, err)
body, err := io.ReadAll(b.Body)
assert.NoError(t, err)
assert.Equal(t, 200, b.StatusCode, string(body))
assert.Equal(t, http.StatusOK, b.StatusCode, string(body))
assert.NoError(t, b.Body.Close())
// comments returned by update
@@ -275,7 +401,7 @@ func TestRest_Update(t *testing.T) {
// read updated comment
res, code := getWithAdminAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah1", ts.URL, id))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
c3 := store.Comment{}
err = json.Unmarshal([]byte(res), &c3)
assert.NoError(t, err)
@@ -305,6 +431,7 @@ func TestRest_UpdateDelete(t *testing.T) {
// delete a comment
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"delete": true, "summary":"removed by user"}`))
require.NoError(t, err)
@@ -313,7 +440,7 @@ func TestRest_UpdateDelete(t *testing.T) {
require.NoError(t, err)
body, err := io.ReadAll(b.Body)
require.NoError(t, err)
assert.Equal(t, 200, b.StatusCode, string(body))
assert.Equal(t, http.StatusOK, b.StatusCode, string(body))
assert.NoError(t, b.Body.Close())
// comments returned by update
@@ -325,7 +452,7 @@ func TestRest_UpdateDelete(t *testing.T) {
// read updated comment
res, code := getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah1", ts.URL, id))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
c3 := store.Comment{}
err = json.Unmarshal([]byte(res), &c3)
assert.NoError(t, err)
@@ -347,6 +474,109 @@ func TestRest_UpdateDelete(t *testing.T) {
{URL: "https://radio-t.com/blah2", Count: 0}}, j)
}
func TestRest_DeleteChildThenParent(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
p := store.Comment{Text: "test test #1",
Locator: store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah1"}}
idP := addComment(t, p, ts)
c1 := store.Comment{Text: "test test #1", ParentID: idP,
Locator: store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah1"}}
idC1 := addComment(t, c1, ts)
c2 := store.Comment{Text: "test test #1", ParentID: idP,
Locator: store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah1"}}
idC2 := addComment(t, c2, ts)
// check multi count equals two
resp, err := post(t, ts.URL+"/api/v1/counts?site=remark42", `["https://radio-t.com/blah1"]`)
require.NoError(t, err)
assert.Equal(t, http.StatusOK, resp.StatusCode)
bb, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
j := []store.PostInfo{}
err = json.Unmarshal(bb, &j)
assert.NoError(t, err)
assert.Equal(t, []store.PostInfo{{URL: "https://radio-t.com/blah1", Count: 3}}, j)
// update a parent comment fails after child is created
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+idP+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"text": "updated text", "summary":"updated by user"}`))
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
b, err := client.Do(req)
require.NoError(t, err)
body, err := io.ReadAll(b.Body)
require.NoError(t, err)
assert.Equal(t, http.StatusBadRequest, b.StatusCode, string(body))
assert.NoError(t, b.Body.Close())
// delete first child comment
req, err = http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+idC1+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"delete": true, "summary":"removed by user"}`))
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
b, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(b.Body)
require.NoError(t, err)
assert.Equal(t, http.StatusOK, b.StatusCode, string(body))
assert.NoError(t, b.Body.Close())
// delete a parent comment, fails as one comment child still present
defer client.CloseIdleConnections()
req, err = http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+idP+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"delete": true, "summary":"removed by user"}`))
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
b, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(b.Body)
require.NoError(t, err)
assert.Equal(t, http.StatusBadRequest, b.StatusCode, string(body))
assert.NoError(t, b.Body.Close())
// delete second child comment, as an admin to check both deletion methods
req, err = http.NewRequest(http.MethodDelete,
fmt.Sprintf("%s/api/v1/admin/comment/%s?site=remark42&url=https://radio-t.com/blah1", ts.URL, idC2), http.NoBody)
require.NoError(t, err)
requireAdminOnly(t, req)
resp, err = sendReq(t, req, adminUmputunToken)
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, http.StatusOK, resp.StatusCode)
// delete a parent comment, shouldn't fail after children are deleted
defer client.CloseIdleConnections()
req, err = http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+idP+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"delete": true, "summary":"removed by user"}`))
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
b, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(b.Body)
require.NoError(t, err)
assert.Equal(t, http.StatusOK, b.StatusCode, string(body))
assert.NoError(t, b.Body.Close())
// check multi count decremented to zero
resp, err = post(t, ts.URL+"/api/v1/counts?site=remark42", `["https://radio-t.com/blah1"]`)
require.NoError(t, err)
assert.Equal(t, http.StatusOK, resp.StatusCode)
bb, err = io.ReadAll(resp.Body)
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
j = []store.PostInfo{}
err = json.Unmarshal(bb, &j)
assert.NoError(t, err)
assert.Equal(t, []store.PostInfo{{URL: "https://radio-t.com/blah1", Count: 0}}, j)
}
func TestRest_UpdateNotOwner(t *testing.T) {
ts, srv, teardown := startupT(t)
defer teardown()
@@ -357,6 +587,7 @@ func TestRest_UpdateNotOwner(t *testing.T) {
assert.NoError(t, err)
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id1+
"?site=remark42&url=https://radio-t.com/blah1", strings.NewReader(`{"text":"updated text", "summary":"my edit"}`))
assert.NoError(t, err)
@@ -366,10 +597,9 @@ func TestRest_UpdateNotOwner(t *testing.T) {
body, err := io.ReadAll(b.Body)
assert.NoError(t, err)
assert.NoError(t, b.Body.Close())
assert.Equal(t, 403, b.StatusCode, string(body), "update from non-owner")
assert.Equal(t, http.StatusForbidden, b.StatusCode, string(body), "update from non-owner")
assert.Equal(t, `{"code":3,"details":"can not edit comments for other users","error":"rejected"}`+"\n", string(body))
client = http.Client{}
req, err = http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id1+
"?site=remark42&url=https://radio-t.com/blah1", strings.NewReader(`ERRR "text":"updated text", "summary":"my"}`))
assert.NoError(t, err)
@@ -377,7 +607,7 @@ func TestRest_UpdateNotOwner(t *testing.T) {
b, err = client.Do(req)
assert.NoError(t, err)
assert.NoError(t, b.Body.Close())
assert.Equal(t, 400, b.StatusCode, string(body), "update is not json")
assert.Equal(t, http.StatusBadRequest, b.StatusCode, string(body), "update is not json")
}
func TestRest_UpdateWrongAud(t *testing.T) {
@@ -389,6 +619,7 @@ func TestRest_UpdateWrongAud(t *testing.T) {
id := addComment(t, c1, ts)
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"text":"updated text", "summary":"my edit"}`))
assert.NoError(t, err)
@@ -408,6 +639,7 @@ func TestRest_UpdateWithRestrictedWords(t *testing.T) {
id := addComment(t, c1, ts)
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id+"?site=remark42&url=https://radio-t.com/blah1",
strings.NewReader(`{"text":"What the duck is that?", "summary":"my edit"}`))
assert.NoError(t, err)
@@ -420,7 +652,7 @@ func TestRest_UpdateWithRestrictedWords(t *testing.T) {
c := R.JSON{}
err = json.Unmarshal(body, &c)
assert.NoError(t, err)
assert.Equal(t, 400, b.StatusCode, string(body))
assert.Equal(t, http.StatusBadRequest, b.StatusCode, string(body))
assert.Equal(t, "comment contains restricted words", c["error"])
assert.Equal(t, "invalid comment", c["details"])
}
@@ -439,6 +671,7 @@ func TestRest_Vote(t *testing.T) {
vote := func(val int) int {
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut,
fmt.Sprintf("%s/api/v1/vote/%s?site=remark42&url=https://radio-t.com/blah&vote=%d", ts.URL, id1, val), http.NoBody)
assert.NoError(t, err)
@@ -449,10 +682,10 @@ func TestRest_Vote(t *testing.T) {
return resp.StatusCode
}
assert.Equal(t, 200, vote(1), "first vote allowed")
assert.Equal(t, 400, vote(1), "second vote rejected")
assert.Equal(t, http.StatusOK, vote(1), "first vote allowed")
assert.Equal(t, http.StatusBadRequest, vote(1), "second vote rejected")
body, code := getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr := store.Comment{}
err := json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -461,27 +694,27 @@ func TestRest_Vote(t *testing.T) {
assert.Equal(t, map[string]bool(nil), cr.Votes, "hidden")
assert.Equal(t, map[string]store.VotedIPInfo(nil), cr.VotedIPs, "hidden")
assert.Equal(t, 200, vote(-1), "opposite vote allowed")
assert.Equal(t, http.StatusOK, vote(-1), "opposite vote allowed")
body, code = getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr = store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
assert.Equal(t, 0, cr.Score)
assert.Equal(t, 0, cr.Vote)
assert.Equal(t, 200, vote(-1), "opposite vote allowed one more time")
assert.Equal(t, http.StatusOK, vote(-1), "opposite vote allowed one more time")
body, code = getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr = store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
assert.Equal(t, -1, cr.Score)
assert.Equal(t, -1, cr.Vote)
assert.Equal(t, 400, vote(-1), "dbl vote not allowed")
assert.Equal(t, http.StatusBadRequest, vote(-1), "dbl vote not allowed")
body, code = getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr = store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -489,7 +722,7 @@ func TestRest_Vote(t *testing.T) {
assert.Equal(t, -1, cr.Vote)
body, code = get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr = store.Comment{}
err = json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -503,7 +736,7 @@ func TestRest_Vote(t *testing.T) {
assert.NoError(t, err)
resp, err := sendReq(t, req, adminUmputunToken)
assert.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
cr = store.Comment{}
err = json.NewDecoder(resp.Body).Decode(&cr)
assert.NoError(t, err)
@@ -528,6 +761,7 @@ func TestRest_AnonVote(t *testing.T) {
vote := func(val int) int {
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut,
fmt.Sprintf("%s/api/v1/vote/%s?site=remark42&url=https://radio-t.com/blah&vote=%d", ts.URL, id1, val), http.NoBody)
assert.NoError(t, err)
@@ -540,6 +774,7 @@ func TestRest_AnonVote(t *testing.T) {
getWithAnonAuth := func(url string) (body string, code int) {
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("GET", url, http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", anonToken)
@@ -551,13 +786,13 @@ func TestRest_AnonVote(t *testing.T) {
return string(b), r.StatusCode
}
assert.Equal(t, 403, vote(1), "vote is disallowed with anonVote false")
assert.Equal(t, http.StatusForbidden, vote(1), "vote is disallowed with anonVote false")
srv.privRest.anonVote = true
assert.Equal(t, 200, vote(1), "first vote allowed")
assert.Equal(t, 400, vote(1), "second vote rejected")
assert.Equal(t, http.StatusOK, vote(1), "first vote allowed")
assert.Equal(t, http.StatusBadRequest, vote(1), "second vote rejected")
body, code := getWithAnonAuth(fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
cr := store.Comment{}
err := json.Unmarshal([]byte(body), &cr)
assert.NoError(t, err)
@@ -567,17 +802,10 @@ func TestRest_AnonVote(t *testing.T) {
assert.Equal(t, map[string]store.VotedIPInfo(nil), cr.VotedIPs)
}
type MockFS struct{}
func (fs *MockFS) ReadFile(path string) ([]byte, error) {
return []byte(fmt.Sprintf("template %s", path)), nil
}
func TestRest_EmailAndTelegram(t *testing.T) {
ts, srv, teardown := startupT(t)
defer teardown()
srv.privRest.templates = &MockFS{}
srv.privRest.telegramService = &mockTelegram{site: "remark42"}
// issue good token
@@ -601,19 +829,25 @@ func TestRest_EmailAndTelegram(t *testing.T) {
responseCode int
noAuth bool
cookieEmail string
body string
}{
{description: "issue delete request without auth", url: "/api/v1/email", method: http.MethodDelete, responseCode: http.StatusUnauthorized, noAuth: true},
{description: "issue delete request without site_id", url: "/api/v1/email", method: http.MethodDelete, responseCode: http.StatusBadRequest},
{description: "delete non-existent user email", url: "/api/v1/email?site=remark42", method: http.MethodDelete, responseCode: http.StatusOK},
{description: "set user email, token not set", url: "/api/v1/email/confirm?site=remark42", method: http.MethodPost, responseCode: http.StatusBadRequest},
{description: "send email confirmation without address", url: "/api/v1/email/subscribe?site=remark42", method: http.MethodPost, responseCode: http.StatusBadRequest},
{description: "send email confirmation", url: "/api/v1/email/subscribe?site=remark42&address=good@example.com", method: http.MethodPost, responseCode: http.StatusOK},
{description: "set user email, token is good", url: fmt.Sprintf("/api/v1/email/confirm?site=remark42&tkn=%s", goodToken), method: http.MethodPost, responseCode: http.StatusOK, cookieEmail: "good@example.com"},
{description: "set user email, token not set", url: "/api/v1/email/confirm", method: http.MethodPost, responseCode: http.StatusBadRequest, body: `{"site":"remark42"}`},
{description: "set user email, token not set, old query param", url: "/api/v1/email/confirm?site=remark42", method: http.MethodPost, responseCode: http.StatusBadRequest},
{description: "send email confirmation without address", url: "/api/v1/email/subscribe", method: http.MethodPost, responseCode: http.StatusBadRequest, body: `{"site":"remark42"}`},
{description: "send email confirmation without address, old query param", url: "/api/v1/email/subscribe?site=remark42", method: http.MethodPost, responseCode: http.StatusBadRequest},
{description: "send email confirmation", url: "/api/v1/email/subscribe", method: http.MethodPost, responseCode: http.StatusOK, body: `{"site":"remark42","address":"good@example.com"}`},
{description: "send email confirmation, old query param", url: "/api/v1/email/subscribe?site=remark42&address=good@example.com", method: http.MethodPost, responseCode: http.StatusOK},
{description: "set user email, token is good", url: "/api/v1/email/confirm", method: http.MethodPost, responseCode: http.StatusOK, cookieEmail: "good@example.com", body: fmt.Sprintf(`{"site":"remark42","token":%q}`, goodToken)},
{description: "set user email, token is good, old query param", url: fmt.Sprintf("/api/v1/email/confirm?site=remark42&tkn=%s", goodToken), method: http.MethodPost, responseCode: http.StatusOK, cookieEmail: "good@example.com"},
{description: "send confirmation with same address", url: "/api/v1/email/subscribe?site=remark42&address=good@example.com", method: http.MethodPost, responseCode: http.StatusConflict},
{description: "get user email", url: "/api/v1/email?site=remark42", method: http.MethodGet, responseCode: http.StatusOK},
{description: "delete user email", url: "/api/v1/email?site=remark42", method: http.MethodDelete, responseCode: http.StatusOK},
{description: "send another confirmation", url: "/api/v1/email/subscribe?site=remark42&address=good@example.com", method: http.MethodPost, responseCode: http.StatusOK},
{description: "set user email, token is good", url: fmt.Sprintf("/api/v1/email/confirm?site=remark42&tkn=%s", goodToken), method: http.MethodPost, responseCode: http.StatusOK, cookieEmail: "good@example.com"},
{description: "set user email, token is good", url: "/api/v1/email/confirm", method: http.MethodPost, responseCode: http.StatusOK, cookieEmail: "good@example.com", body: fmt.Sprintf(`{"site":"remark42","token":%q}`, goodToken)},
{description: "set user email, token is good, old query param", url: fmt.Sprintf("/api/v1/email/confirm?site=remark42&tkn=%s", goodToken), method: http.MethodPost, responseCode: http.StatusOK, cookieEmail: "good@example.com"},
{description: "unsubscribe user, no token", url: "/email/unsubscribe.html?site=remark42", method: http.MethodPost, responseCode: http.StatusBadRequest},
{description: "unsubscribe user, wrong token", url: "/email/unsubscribe.html?site=remark42&tkn=jwt", method: http.MethodGet, responseCode: http.StatusForbidden},
{description: "unsubscribe user, good token", url: fmt.Sprintf("/email/unsubscribe.html?site=remark42&tkn=%s", goodToken), method: http.MethodPost, responseCode: http.StatusOK},
@@ -630,10 +864,15 @@ func TestRest_EmailAndTelegram(t *testing.T) {
{description: "set user telegram, token is good", url: "/api/v1/telegram/subscribe?site=remark42&tkn=good_token", method: http.MethodGet, responseCode: http.StatusOK},
}
client := http.Client{}
defer client.CloseIdleConnections()
for _, x := range testData {
x := x
t.Run(x.description, func(t *testing.T) {
req, err := http.NewRequest(x.method, ts.URL+x.url, http.NoBody)
reqBody := io.NopCloser(strings.NewReader(x.body))
if x.body == "" {
reqBody = http.NoBody
}
req, err := http.NewRequest(x.method, ts.URL+x.url, reqBody)
require.NoError(t, err)
if !x.noAuth {
req.Header.Add("X-JWT", devToken)
@@ -665,6 +904,7 @@ func TestRest_EmailNotification(t *testing.T) {
defer srv.privRest.notifyService.Close()
client := http.Client{}
defer client.CloseIdleConnections()
// create new comment from dev user
req, err := http.NewRequest("POST", ts.URL+"/api/v1/comment", strings.NewReader(
@@ -708,7 +948,11 @@ func TestRest_EmailNotification(t *testing.T) {
assert.Empty(t, mockDestination.Get()[1].Emails)
// send confirmation token for email
req, err = http.NewRequest(http.MethodPost, ts.URL+"/api/v1/email/subscribe?site=remark42&address=good@example.com", http.NoBody)
req, err = http.NewRequest(
http.MethodPost,
ts.URL+"/api/v1/email/subscribe",
io.NopCloser(strings.NewReader(`{"site": "remark42", "address": "good@example.com"}`)),
)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err = client.Do(req)
@@ -723,8 +967,31 @@ func TestRest_EmailNotification(t *testing.T) {
assert.Equal(t, "good@example.com", mockDestination.GetVerify()[0].Email)
verificationToken := mockDestination.GetVerify()[0].Token
// get user information to verify lack of the subscription
req, err = http.NewRequest(
http.MethodGet,
ts.URL+"/api/v1/user?site=remark42",
http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
var clearUser store.User
err = json.Unmarshal(body, &clearUser)
assert.NoError(t, err)
assert.Equal(t, store.User{Name: "developer one", ID: "dev", EmailSubscription: false,
Picture: "http://example.com/pic.png", IP: "127.0.0.1", SiteID: "remark42"}, clearUser)
// verify email
req, err = http.NewRequest(http.MethodPost, ts.URL+fmt.Sprintf("/api/v1/email/confirm?site=remark42&tkn=%s", verificationToken), http.NoBody)
req, err = http.NewRequest(
http.MethodPost,
ts.URL+"/api/v1/email/confirm",
io.NopCloser(strings.NewReader(fmt.Sprintf(`{"site": "remark42", "token": %q}`, verificationToken))),
)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err = client.Do(req)
@@ -735,7 +1002,10 @@ func TestRest_EmailNotification(t *testing.T) {
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
// get user information to verify the subscription
req, err = http.NewRequest(http.MethodGet, ts.URL+"/api/v1/user?site=remark42", http.NoBody)
req, err = http.NewRequest(
http.MethodGet,
ts.URL+"/api/v1/user?site=remark42",
http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err = client.Do(req)
@@ -744,11 +1014,11 @@ func TestRest_EmailNotification(t *testing.T) {
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
var user store.User
err = json.Unmarshal(body, &user)
var subscribedUser store.User
err = json.Unmarshal(body, &subscribedUser)
assert.NoError(t, err)
assert.Equal(t, store.User{Name: "developer one", ID: "dev", EmailSubscription: true,
Picture: "http://example.com/pic.png", IP: "127.0.0.1", SiteID: "remark42"}, user)
Picture: "http://example.com/pic.png", IP: "127.0.0.1", SiteID: "remark42"}, subscribedUser)
// create child comment from another user, email notification expected
req, err = http.NewRequest("POST", ts.URL+"/api/v1/comment", strings.NewReader(fmt.Sprintf(
@@ -799,6 +1069,80 @@ func TestRest_EmailNotification(t *testing.T) {
time.Sleep(time.Millisecond * 30)
require.Equal(t, 4, len(mockDestination.Get()))
assert.Empty(t, mockDestination.Get()[3].Emails)
// confirm email via subscribe call with query params, old behavior, email notification is expected
req, err = http.NewRequest(
http.MethodPost,
ts.URL+"/api/v1/email/subscribe?site=remark42&address=good@example.com",
http.NoBody,
)
require.NoError(t, err)
req.Header.Add("X-JWT", emailUserToken)
resp, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
// wait for mock notification Submit to kick off
time.Sleep(time.Millisecond * 30)
require.Equal(t, 2, len(mockDestination.GetVerify()), "verification email was sent")
// get email user information to verify there is no subscription yet
req, err = http.NewRequest(
http.MethodGet,
ts.URL+"/api/v1/user?site=remark42",
http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", emailUserToken)
resp, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
var unsubscribedEmailUser store.User
err = json.Unmarshal(body, &unsubscribedEmailUser)
assert.NoError(t, err)
assert.Equal(t, store.User{Name: "good@example.com test user", ID: "email_f5dfe9d2e6bd75fc74ea5fabf273b45b5baeb195", EmailSubscription: false,
Picture: "http://example.com/pic.png", IP: "127.0.0.1", SiteID: "remark42"}, unsubscribedEmailUser)
// confirm email via subscribe call, no email notification is expected
req, err = http.NewRequest(
http.MethodPost,
ts.URL+"/api/v1/email/subscribe",
io.NopCloser(strings.NewReader(`{"site": "remark42", "address": "good@example.com"}`)),
)
require.NoError(t, err)
req.Header.Add("X-JWT", emailUserToken)
resp, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
// wait for mock notification Submit to kick off
time.Sleep(time.Millisecond * 30)
require.Equal(t, 2, len(mockDestination.GetVerify()), "no new verification email was sent")
// get email user information to verify the subscription happened without the confirmation call
req, err = http.NewRequest(
http.MethodGet,
ts.URL+"/api/v1/user?site=remark42",
http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", emailUserToken)
resp, err = client.Do(req)
require.NoError(t, err)
body, err = io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusOK, resp.StatusCode, string(body))
var subscribedEmailUser store.User
err = json.Unmarshal(body, &subscribedEmailUser)
assert.NoError(t, err)
assert.Equal(t, store.User{Name: "good@example.com test user", ID: "email_f5dfe9d2e6bd75fc74ea5fabf273b45b5baeb195", EmailSubscription: true,
Picture: "http://example.com/pic.png", IP: "127.0.0.1", SiteID: "remark42"}, subscribedEmailUser)
}
func TestRest_TelegramNotification(t *testing.T) {
@@ -810,6 +1154,7 @@ func TestRest_TelegramNotification(t *testing.T) {
defer srv.privRest.notifyService.Close()
client := http.Client{}
defer client.CloseIdleConnections()
// create new comment from dev user
req, err := http.NewRequest("POST", ts.URL+"/api/v1/comment", strings.NewReader(
@@ -893,8 +1238,8 @@ func TestRest_TelegramNotification(t *testing.T) {
body, err = io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusInternalServerError, resp.StatusCode, string(body))
require.Equal(t, `{"code":0,"details":"can't set telegram for user","error":"not verified"}`+"\n", string(body))
require.Equal(t, http.StatusNotFound, resp.StatusCode, string(body))
require.Equal(t, `{"code":0,"details":"request is not verified yet","error":"not verified"}`+"\n", string(body))
mockTlgrm.notVerified = false
@@ -1016,12 +1361,13 @@ func TestRest_UserAllData(t *testing.T) {
require.NoError(t, err)
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("GET", ts.URL+"/api/v1/userdata?site=remark42", http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err := client.Do(req)
require.NoError(t, err)
require.Equal(t, 200, resp.StatusCode)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "application/gzip", resp.Header.Get("Content-Type"))
ungzReader, err := gzip.NewReader(resp.Body)
@@ -1050,7 +1396,7 @@ func TestRest_UserAllData(t *testing.T) {
resp, err = client.Do(req)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, 401, resp.StatusCode)
require.Equal(t, http.StatusUnauthorized, resp.StatusCode)
}
func TestRest_UserAllDataManyComments(t *testing.T) {
@@ -1068,12 +1414,13 @@ func TestRest_UserAllDataManyComments(t *testing.T) {
require.NoError(t, err)
}
client := &http.Client{Timeout: 1 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("GET", ts.URL+"/api/v1/userdata?site=remark42", http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err := client.Do(req)
require.NoError(t, err)
require.Equal(t, 200, resp.StatusCode)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "application/gzip", resp.Header.Get("Content-Type"))
ungzReader, err := gzip.NewReader(resp.Body)
@@ -1092,12 +1439,13 @@ func TestRest_DeleteMe(t *testing.T) {
defer teardown()
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPost, fmt.Sprintf("%s/api/v1/deleteme?site=remark42", ts.URL), http.NoBody)
assert.NoError(t, err)
req.Header.Add("X-JWT", devToken)
resp, err := client.Do(req)
assert.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, resp.Body.Close())
assert.NoError(t, err)
@@ -1118,7 +1466,7 @@ func TestRest_DeleteMe(t *testing.T) {
assert.NoError(t, err)
resp, err = client.Do(req)
assert.NoError(t, err)
assert.Equal(t, 401, resp.StatusCode)
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
assert.NoError(t, resp.Body.Close())
}
@@ -1138,13 +1486,14 @@ func TestRest_SavePictureCtrl(t *testing.T) {
require.NoError(t, bodyWriter.Close())
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPost, fmt.Sprintf("%s/api/v1/picture", ts.URL), bodyBuf)
require.NoError(t, err)
req.Header.Add("Content-Type", contentType)
req.Header.Add("X-JWT", devToken)
resp, err := client.Do(req)
assert.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
@@ -1159,7 +1508,7 @@ func TestRest_SavePictureCtrl(t *testing.T) {
id := savePic("picture.png")
resp, err := http.Get(fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, id))
require.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
@@ -1170,34 +1519,34 @@ func TestRest_SavePictureCtrl(t *testing.T) {
resp, err = http.Get(fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, id))
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
assert.Equal(t, "image/png", resp.Header.Get("Content-Type"))
id = savePic("picture.jpg")
resp, err = http.Get(fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, id))
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
assert.Equal(t, "image/png", resp.Header.Get("Content-Type"))
id = savePic("picture.blah")
resp, err = http.Get(fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, id))
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
assert.Equal(t, "image/png", resp.Header.Get("Content-Type"))
resp, err = http.Get(fmt.Sprintf("%s/api/v1/picture/blah/pic.blah", ts.URL))
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 400, resp.StatusCode)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
}
func TestRest_CreateWithPictures(t *testing.T) {
ts, svc, teardown := startupT(t)
defer func() {
teardown()
os.RemoveAll("/tmp/remark42")
assert.NoError(t, os.RemoveAll("/tmp/remark42"))
}()
lgr.Setup(lgr.Debug, lgr.CallerFile, lgr.CallerFunc)
@@ -1229,13 +1578,14 @@ func TestRest_CreateWithPictures(t *testing.T) {
contentType := bodyWriter.FormDataContentType()
require.NoError(t, bodyWriter.Close())
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPost, fmt.Sprintf("%s/api/v1/picture", ts.URL), bodyBuf)
require.NoError(t, err)
req.Header.Add("Content-Type", contentType)
req.Header.Add("X-JWT", devToken)
resp, err := client.Do(req)
assert.NoError(t, err)
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
@@ -1288,7 +1638,7 @@ func (m *mockTelegram) GetBotUsername() string {
func (m *mockTelegram) CheckToken(string, string) (telegram, site string, err error) {
if m.notVerified {
return "", "", errors.New("not verified")
return "", "", fmt.Errorf("not verified")
}
return "good_telegram", m.site, nil
}
+44 -22
View File
@@ -4,10 +4,9 @@ import (
"bytes"
"crypto/sha1" // nolint
"encoding/base64"
"fmt"
"io"
"net/http"
"os"
"path"
"strconv"
"strings"
"time"
@@ -17,7 +16,7 @@ import (
cache "github.com/go-pkgz/lcw"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/pkg/errors"
"github.com/skip2/go-qrcode"
"github.com/umputun/remark42/backend/app/rest"
"github.com/umputun/remark42/backend/app/store"
@@ -31,7 +30,6 @@ type public struct {
readOnlyAge int
commentFormatter *store.CommentFormatter
imageService *image.Service
webRoot string
}
type pubStore interface {
@@ -42,7 +40,7 @@ type pubStore interface {
User(siteID, userID string, limit, skip int, user store.User) ([]store.Comment, error)
UserCount(siteID, userID string) (int, error)
Count(locator store.Locator) (int, error)
List(siteID string, limit int, skip int) ([]store.PostInfo, error)
List(siteID string, limit, skip int) ([]store.PostInfo, error)
Info(locator store.Locator, readonlyAge int) (store.PostInfo, error)
ValidateComment(c *store.Comment) error
@@ -173,7 +171,6 @@ func (s *public) lastCommentsCtrl(w http.ResponseWriter, r *http.Request) {
// GET /id/{id}?site=siteID&url=post-url - gets a comment by id
func (s *public) commentByIDCtrl(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
siteID := r.URL.Query().Get("site")
url := r.URL.Query().Get("url")
@@ -194,7 +191,6 @@ func (s *public) commentByIDCtrl(w http.ResponseWriter, r *http.Request) {
// GET /comments?site=siteID&user=id&limit=123&skip=10 - returns comments for given userID
func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
userID := r.URL.Query().Get("user")
siteID := r.URL.Query().Get("site")
@@ -209,8 +205,8 @@ func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
limit, skip := getNumWithDef("limit"), getNumWithDef("skip")
resp := struct {
Comments []store.Comment `json:"comments,omitempty"`
Count int `json:"count,omitempty"`
Comments []store.Comment `json:"comments"`
Count int `json:"count"`
}{}
log.Printf("[DEBUG] get comments for userID %s, %s", userID, siteID)
@@ -219,6 +215,10 @@ func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
data, err := s.cache.Get(key, func() ([]byte, error) {
comments, e := s.dataService.User(siteID, userID, limit, skip, rest.GetUserOrEmpty(r))
if e != nil {
if strings.Contains(e.Error(), "no comments for user") { // store returns this error when no comments found
resp.Comments, resp.Count = []store.Comment{}, 0
return encodeJSONWithHTML(resp)
}
return nil, e
}
comments = filterComments(comments, func(c store.Comment) bool { return !c.Deleted })
@@ -234,7 +234,6 @@ func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get comment by user id", rest.ErrCommentNotFound)
return
}
if err = R.RenderJSONFromBytes(w, r, data); err != nil {
log.Printf("[WARN] can't render found comments for user %s", userID)
}
@@ -287,7 +286,6 @@ func (s *public) countMultiCtrl(w http.ResponseWriter, r *http.Request) {
// GET /list?site=siteID&limit=50&skip=10 - list posts with comments
func (s *public) listCtrl(w http.ResponseWriter, r *http.Request) {
siteID := r.URL.Query().Get("site")
limit, skip := 0, 0
@@ -344,16 +342,6 @@ func (s *public) loadPictureCtrl(w http.ResponseWriter, r *http.Request) {
}
}
// GET /index.html - respond to /index.html with the content of getstarted.html under /web root
func (s *public) getStartedCtrl(w http.ResponseWriter, r *http.Request) {
data, err := os.ReadFile(path.Join(s.webRoot, "getstarted.html"))
if err != nil {
w.WriteHeader(http.StatusNotFound)
return
}
render.HTML(w, r, string(data))
}
// GET /robots.txt
func (s *public) robotsCtrl(w http.ResponseWriter, r *http.Request) {
allowed := []string{"/find", "/last", "/id", "/count", "/counts", "/list", "/config", "/user",
@@ -364,6 +352,40 @@ func (s *public) robotsCtrl(w http.ResponseWriter, r *http.Request) {
render.PlainText(w, r, "User-agent: *\nDisallow: /auth/\nDisallow: /api/\n"+strings.Join(allowed, "\n")+"\n")
}
// GET /qr/telegram - generates QR for provided URL, used for Telegram auth and notifications subscription. The first
// step of both is the user opening a link to Telegram and writing bot a message, and the user might try to log in
// from a computer but have Telegram installed only on the mobile device.
// Provided text should start with https://t.me/.
func (s *public) telegramQrCtrl(w http.ResponseWriter, r *http.Request) {
text := r.URL.Query().Get("url")
if text == "" {
rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "text parameter is required", rest.ErrInternal)
return
}
if !strings.HasPrefix(text, "https://t.me/") {
rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("wrong parameter"), "text parameter should start with https://t.me/", rest.ErrInternal)
return
}
q, err := qrcode.New(text, qrcode.Medium)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't generate QR", rest.ErrInternal)
return
}
q.DisableBorder = true
png, err := q.PNG(256)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't generate QR", rest.ErrInternal)
return
}
w.Header().Set("Content-Type", "image/png")
if _, err = w.Write(png); err != nil {
log.Printf("[WARN] can't render qr, %v", err)
}
}
func (s *public) applyView(comments []store.Comment, view string) []store.Comment {
if strings.EqualFold(view, "user") {
projection := make([]store.Comment, 0, len(comments))
@@ -388,7 +410,7 @@ func (s *public) parseSince(r *http.Request) (time.Time, error) {
if since := r.URL.Query().Get("since"); since != "" {
unixTS, e := strconv.ParseInt(since, 10, 64)
if e != nil {
return time.Time{}, errors.Wrap(e, "can't translate since parameter")
return time.Time{}, fmt.Errorf("can't translate since parameter: %w", e)
}
sinceTS = time.Unix(unixTS/1000, 1000000*(unixTS%1000)) // since param in msec timestamp
}
+115 -53
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"io"
"net/http"
"os"
"strings"
"testing"
"time"
@@ -22,9 +23,32 @@ func TestRest_Ping(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
res, code := get(t, ts.URL+"/api/v1/ping")
assert.Equal(t, "pong", res)
assert.Equal(t, 200, code)
resp, err := http.Get(ts.URL + "/api/v1/ping")
require.NoError(t, err)
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, "pong", string(body))
assert.Equal(t, http.StatusOK, resp.StatusCode)
assert.Equal(t, "remark42", resp.Header.Get("App-Name"))
}
func TestRest_PingNoSignature(t *testing.T) {
ts, _, teardown := startupT(t, func(srv *Rest) {
srv.DisableSignature = true
})
defer teardown()
resp, err := http.Get(ts.URL + "/api/v1/ping")
require.NoError(t, err)
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, "pong", string(body))
assert.Equal(t, http.StatusOK, resp.StatusCode)
assert.Equal(t, "", resp.Header.Get("App-Name"))
}
func TestRest_Preview(t *testing.T) {
@@ -42,7 +66,7 @@ func TestRest_Preview(t *testing.T) {
resp, err = post(t, ts.URL+"/api/v1/preview", "bad")
assert.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, 400, resp.StatusCode)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
resp, err = post(t, ts.URL+"/api/v1/preview", fmt.Sprintf(`{"text": "![non-existent.jpg](%s/api/v1/picture/dev_user/bad_picture)", "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`, srv.RemarkURL))
assert.NoError(t, err)
@@ -52,14 +76,13 @@ func TestRest_Preview(t *testing.T) {
assert.NoError(t, resp.Body.Close())
assert.Contains(t,
string(b),
"{\"code\":20,\"details\":\"can't renew staged picture cleanup timer\","+
"\"error\":\"can't get image stats for dev_user/bad_picture: stat",
`{"code":20,"details":"can't load picture from the comment",`+
`"error":"can't get image stats for dev_user/bad_picture: stat`,
)
assert.Contains(t,
string(b),
"/pics-remark42/staging/dev_user/62/bad_picture: no such file or directory\"}\n",
)
}
func TestRest_PreviewWithWrongImage(t *testing.T) {
@@ -74,8 +97,8 @@ func TestRest_PreviewWithWrongImage(t *testing.T) {
assert.NoError(t, resp.Body.Close())
assert.Contains(t,
string(b),
"{\"code\":20,\"details\":\"can't renew staged picture cleanup timer\","+
"\"error\":\"can't get image stats for dev_user/bad_picture: stat ",
`{"code":20,"details":"can't load picture from the comment",`+
`"error":"can't get image stats for dev_user/bad_picture: stat `,
)
assert.Contains(t,
string(b),
@@ -108,11 +131,11 @@ BKT
assert.NoError(t, err)
assert.Equal(t,
`<h1>h1</h1>
<pre class="chroma">func TestRest_Preview(t *testing.T) {
srv, ts := prep(t)
require.NotNil(t, srv)
}
</pre>`,
<pre class="chroma"><code><span class="line"><span class="cl">func TestRest_Preview(t *testing.T) {
</span></span><span class="line"><span class="cl">srv, ts := prep(t)
</span></span><span class="line"><span class="cl"> require.NotNil(t, srv)
</span></span><span class="line"><span class="cl">}
</span></span></code></pre>`,
string(b))
assert.NoError(t, resp.Body.Close())
}
@@ -134,8 +157,8 @@ BKT
assert.Equal(t, http.StatusOK, resp.StatusCode)
b, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, `<pre class="chroma"><span class="kd">func</span> <span class="nf">main</span><span class="p">(</span><span class="nx">aa</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">int</span> <span class="p">{</span><span class="k">return</span> <span class="mi">0</span><span class="p">}</span>
</pre>`, string(b))
assert.Equal(t, `<pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">(</span><span class="nx">aa</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">int</span> <span class="p">{</span><span class="k">return</span> <span class="mi">0</span><span class="p">}</span>
</span></span></code></pre>`, string(b))
assert.NoError(t, resp.Body.Close())
}
@@ -144,7 +167,7 @@ func TestRest_Find(t *testing.T) {
defer teardown()
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := commentsWithInfo{}
err := json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -162,7 +185,7 @@ func TestRest_Find(t *testing.T) {
// get sorted by +time
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=+time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -178,7 +201,7 @@ func TestRest_Find(t *testing.T) {
// get sorted by -time
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=-time")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
require.Equal(t, 2, len(comments.Comments), "should have 2 comments")
@@ -188,7 +211,7 @@ func TestRest_Find(t *testing.T) {
// get in tree mode
tree := service.Tree{}
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&format=tree")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &tree)
assert.NoError(t, err)
require.Equal(t, 1, len(tree.Nodes))
@@ -215,14 +238,14 @@ func TestRest_FindAge(t *testing.T) {
tree := service.Tree{}
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&format=tree")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &tree)
assert.NoError(t, err)
assert.Equal(t, "https://radio-t.com/blah1", tree.Info.URL)
assert.False(t, tree.Info.ReadOnly, "post is fresh")
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah2&format=tree")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &tree)
assert.NoError(t, err)
assert.Equal(t, "https://radio-t.com/blah2", tree.Info.URL)
@@ -246,6 +269,7 @@ func TestRest_FindReadOnly(t *testing.T) {
// set post to read-only
client := http.Client{}
defer client.CloseIdleConnections()
req, err := http.NewRequest(http.MethodPut,
fmt.Sprintf("%s/api/v1/admin/readonly?site=remark42&url=https://radio-t.com/blah1&ro=1", ts.URL), http.NoBody)
assert.NoError(t, err)
@@ -256,7 +280,7 @@ func TestRest_FindReadOnly(t *testing.T) {
tree := service.Tree{}
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&format=tree")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &tree)
require.NoError(t, err)
assert.Equal(t, "https://radio-t.com/blah1", tree.Info.URL)
@@ -264,7 +288,7 @@ func TestRest_FindReadOnly(t *testing.T) {
tree = service.Tree{}
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah2&format=tree")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &tree)
require.NoError(t, err)
assert.Equal(t, "https://radio-t.com/blah2", tree.Info.URL)
@@ -276,7 +300,7 @@ func TestRest_FindUserView(t *testing.T) {
defer teardown()
res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&view=user")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := commentsWithInfo{}
err := json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -294,7 +318,7 @@ func TestRest_FindUserView(t *testing.T) {
// get sorted by +time with view=user
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=+time&view=user")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -311,7 +335,7 @@ func TestRest_FindUserView(t *testing.T) {
srv.Cache.Flush(cache.FlusherRequest{})
res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=+time&view=user")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = commentsWithInfo{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -324,7 +348,7 @@ func TestRest_Last(t *testing.T) {
defer teardown()
res, code := get(t, ts.URL+"/api/v1/last/2?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
assert.Equal(t, "[]\n", res, "empty last should return empty list")
c1 := store.Comment{Text: "test test #1", ParentID: "p1",
@@ -341,7 +365,7 @@ func TestRest_Last(t *testing.T) {
id2 := addComment(t, c2, ts)
res, code = get(t, ts.URL+"/api/v1/last/2?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments := []store.Comment{}
err := json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -350,7 +374,7 @@ func TestRest_Last(t *testing.T) {
assert.Equal(t, id2, comments[0].ID)
res, code = get(t, fmt.Sprintf("%s/api/v1/last/2?site=remark42&since=%d", ts.URL, ts1))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = []store.Comment{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -359,7 +383,7 @@ func TestRest_Last(t *testing.T) {
assert.Equal(t, id2, comments[0].ID)
res, code = get(t, fmt.Sprintf("%s/api/v1/last/2?site=remark42&since=%d", ts.URL, ts2))
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
comments = []store.Comment{}
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
@@ -367,13 +391,13 @@ func TestRest_Last(t *testing.T) {
assert.Equal(t, id2, comments[0].ID)
res, code = get(t, ts.URL+"/api/v1/last/5?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
assert.Equal(t, 3, len(comments), "should have 3 comments")
res, code = get(t, ts.URL+"/api/v1/last/X?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
assert.Equal(t, 3, len(comments), "should have 3 comments")
@@ -382,13 +406,13 @@ func TestRest_Last(t *testing.T) {
assert.NoError(t, err)
srv.Cache.Flush(cache.FlusherRequest{})
res, code = get(t, ts.URL+"/api/v1/last/5?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(res), &comments)
assert.NoError(t, err)
assert.Equal(t, 2, len(comments), "should have 2 comments")
_, code = get(t, ts.URL+"/api/v1/last/2?site=remark42-BLAH")
assert.Equal(t, 500, code)
assert.Equal(t, http.StatusInternalServerError, code)
}
func TestRest_FindUserComments(t *testing.T) {
@@ -412,12 +436,12 @@ func TestRest_FindUserComments(t *testing.T) {
err := srv.DataService.Delete(c2.Locator, id, store.SoftDelete)
assert.NoError(t, err)
_, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=blah")
assert.Equal(t, 400, code, "noting for user blah")
comments, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=blah")
assert.Equal(t, http.StatusOK, code, "noting for user blah")
assert.Equal(t, `{"comments":[],"count":0}`+"\n", comments)
{
res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=dev")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
resp := struct {
Comments []store.Comment
@@ -436,7 +460,7 @@ func TestRest_FindUserComments(t *testing.T) {
{
res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=dev&skip=1&limit=2")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
resp := struct {
Comments []store.Comment
@@ -458,7 +482,7 @@ func TestRest_UserInfo(t *testing.T) {
defer teardown()
body, code := getWithDevAuth(t, ts.URL+"/api/v1/user?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
user := store.User{}
err := json.Unmarshal([]byte(body), &user)
assert.NoError(t, err)
@@ -482,20 +506,20 @@ func TestRest_Count(t *testing.T) {
addComment(t, c2, ts)
body, code := get(t, ts.URL+"/api/v1/count?site=remark42&url=https://radio-t.com/blah1")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
j := R.JSON{}
err := json.Unmarshal([]byte(body), &j)
assert.NoError(t, err)
assert.Equal(t, 3.0, j["count"])
body, code = get(t, ts.URL+"/api/v1/count?site=remark42&url=https://radio-t.com/blah2")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
err = json.Unmarshal([]byte(body), &j)
assert.NoError(t, err)
assert.Equal(t, 2.0, j["count"])
_, code = get(t, ts.URL+"/api/v1/count?site=remark42-BLAH&url=https://radio-t.com/blah1XXX")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
}
func TestRest_Counts(t *testing.T) {
@@ -529,7 +553,7 @@ func TestRest_Counts(t *testing.T) {
resp, err = post(t, ts.URL+"/api/v1/counts?site=radio-XXX", `{}`)
require.NoError(t, err)
assert.Equal(t, 400, resp.StatusCode)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
assert.NoError(t, resp.Body.Close())
}
@@ -549,7 +573,7 @@ func TestRest_List(t *testing.T) {
addComment(t, c2, ts)
body, code := get(t, ts.URL+"/api/v1/list?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
pi := []store.PostInfo{}
err := json.Unmarshal([]byte(body), &pi)
assert.NoError(t, err)
@@ -559,7 +583,7 @@ func TestRest_List(t *testing.T) {
assert.Equal(t, 3, pi[1].Count)
_, code = get(t, ts.URL+"/api/v1/list?site=remark42-BLAH")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
}
func TestRest_ListWithSkipAndLimit(t *testing.T) {
@@ -582,7 +606,7 @@ func TestRest_ListWithSkipAndLimit(t *testing.T) {
addComment(t, c3, ts)
body, code := get(t, ts.URL+"/api/v1/list?site=remark42&skip=1&limit=2")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
pi := []store.PostInfo{}
err := json.Unmarshal([]byte(body), &pi)
assert.NoError(t, err)
@@ -598,7 +622,7 @@ func TestRest_Config(t *testing.T) {
defer teardown()
body, code := get(t, ts.URL+"/api/v1/config?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
j := R.JSON{}
err := json.Unmarshal([]byte(body), &j)
assert.NoError(t, err)
@@ -615,6 +639,44 @@ func TestRest_Config(t *testing.T) {
assert.Equal(t, false, j["admin_edit"].(bool))
}
func TestRest_QR(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
// missing parameter
body, code := get(t, ts.URL+"/api/v1/qr/telegram")
assert.Equal(t, http.StatusBadRequest, code)
assert.Equal(t, "{\"code\":0,\"details\":\"text parameter is required\",\"error\":\"missing parameter\"}\n", body)
// too long request to build the qr
body, code = get(t, ts.URL+"/api/v1/qr/telegram?url=https://t.me/"+strings.Repeat("string", 1000))
assert.Equal(t, http.StatusInternalServerError, code)
assert.Equal(t, "{\"code\":0,\"details\":\"can't generate QR\",\"error\":\"content too long to encode\"}\n", body)
// wrong request
body, code = get(t, ts.URL+"/api/v1/qr/telegram?url=nonsense")
assert.Equal(t, http.StatusBadRequest, code)
assert.Equal(t, "{\"code\":0,\"details\":\"text parameter should start with https://t.me/\",\"error\":\"wrong parameter\"}\n", body)
// correct request
r, err := http.Get(ts.URL + "/api/v1/qr/telegram?url=https://t.me/BotFather")
require.NoError(t, err)
bdy, err := io.ReadAll(r.Body)
require.NoError(t, err)
require.NoError(t, r.Body.Close())
require.NotEmpty(t, bdy)
assert.Equal(t, "image/png", r.Header.Get("Content-Type"))
assert.Equal(t, http.StatusOK, r.StatusCode)
// compare the image
fh, err := os.Open("testdata/qr_test.png")
defer func() { assert.NoError(t, fh.Close()) }()
assert.NoError(t, err)
img, err := io.ReadAll(fh)
assert.NoError(t, err)
assert.Equal(t, img, bdy)
}
func TestRest_Info(t *testing.T) {
ts, srv, teardown := startupT(t)
defer teardown()
@@ -637,7 +699,7 @@ func TestRest_Info(t *testing.T) {
require.NoError(t, err)
body, code := get(t, ts.URL+"/api/v1/info?site=remark42&url=https://radio-t.com/blah1")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
info := store.PostInfo{}
err = json.Unmarshal([]byte(body), &info)
@@ -647,9 +709,9 @@ func TestRest_Info(t *testing.T) {
assert.Equal(t, exp, info)
_, code = get(t, ts.URL+"/api/v1/info?site=remark42&url=https://radio-t.com/blah-no")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
_, code = get(t, ts.URL+"/api/v1/info?site=remark42-no&url=https://radio-t.com/blah-no")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
}
func TestRest_Robots(t *testing.T) {
@@ -657,7 +719,7 @@ func TestRest_Robots(t *testing.T) {
defer teardown()
body, code := get(t, ts.URL+"/robots.txt")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
assert.Equal(t, "User-agent: *\nDisallow: /auth/\nDisallow: /api/\nAllow: /api/v1/find\n"+
"Allow: /api/v1/last\nAllow: /api/v1/id\nAllow: /api/v1/count\nAllow: /api/v1/counts\n"+
"Allow: /api/v1/list\nAllow: /api/v1/config\nAllow: /api/v1/user\nAllow: /api/v1/img\n"+
+68 -47
View File
@@ -4,7 +4,6 @@ import (
"bytes"
"crypto/tls"
"encoding/json"
"errors"
"fmt"
"io"
"math/rand"
@@ -38,10 +37,15 @@ import (
"github.com/umputun/remark42/backend/app/store/service"
)
// To generate a token, enter one of the tokens here into https://jwt.io, change the secret to one you're using in your test
// ("secret" in case of startupT), and alter the fields you want to be changed.
var devToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6ImRldmVsb3BlciBvbmUiLCJpZCI6ImRldiIsInBpY3R1cmUiOiJodHRwOi8vZXhhbXBsZS5jb20vcGljLnBuZyIsImlwIjoiMTI3LjAuMC4xIiwiZW1haWwiOiJtZUBleGFtcGxlLmNvbSJ9fQ.aKUAXiZxXypgV7m1wEOgUcyPOvUDXHDi3A06YWKbcLg`
var anonToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6ImFub255bW91cyB0ZXN0IHVzZXIiLCJpZCI6ImFub255bW91c190ZXN0X3VzZXIiLCJwaWN0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL3BpYy5wbmciLCJpcCI6IjEyNy4wLjAuMSIsImVtYWlsIjoiYW5vbkBleGFtcGxlLmNvbSJ9fQ.gAae2WMxZNZE5ebVboptPEyQ7Nk6EQxciNnGJ_mPOuU`
var emailUserToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6Imdvb2RAZXhhbXBsZS5jb20gdGVzdCB1c2VyIiwiaWQiOiJlbWFpbF9mNWRmZTlkMmU2YmQ3NWZjNzRlYTVmYWJmMjczYjQ1YjViYWViMTk1IiwicGljdHVyZSI6Imh0dHA6Ly9leGFtcGxlLmNvbS9waWMucG5nIiwiaXAiOiIxMjcuMC4wLjEiLCJlbWFpbCI6Imdvb2RAZXhhbXBsZS5jb20ifX0.vH2HN1JpuXL8okTJq1A-zGHQ-l2ILcwxvDDEmu2zwks`
var devTokenBadAud = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0Ml9iYWQiLCJleHAiOjM3ODkxOTE4MjIsImp0aSI6InJhbmRvbSBpZCIsImlzcyI6InJlbWFyazQyIiwibmJmIjoxNTIxODg0MjIyLCJ1c2VyIjp7Im5hbWUiOiJkZXZlbG9wZXIgb25lIiwiaWQiOiJkZXYiLCJwaWN0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL3BpYy5wbmciLCJpcCI6IjEyNy4wLjAuMSIsImVtYWlsIjoibWVAZXhhbXBsZS5jb20ifX0.FuTTocVtcxr4VjpfIICvU2yOb3su28VkDzj94H9Q3xY`
var adminUmputunToken = `eyJhbGciOiJIUzI1NiJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6MTk1NDU5Nzk4MCwianRpIjoiOTdhMmUwYWM0ZGM3ZDVmNjkyNmQ1ZTg2MjBhY2VmOWE0MGMwIiwiaWF0IjoxNDU0NTk3NjgwLCJpc3MiOiJyZW1hcms0MiIsInVzZXIiOnsibmFtZSI6IlVtcHV0dW4iLCJpZCI6ImdpdGh1Yl9lZjBmNzA2YTciLCJwaWN0dXJlIjoiaHR0cHM6Ly9yZW1hcms0Mi5yYWRpby10LmNvbS9hcGkvdjEvYXZhdGFyL2NiNDJmZjQ5M2FkZTY5NmQ4OGEzYTU5MGYxMzZhZTllMzRkZTdjMWIuaW1hZ2UiLCJhdHRycyI6eyJhZG1pbiI6dHJ1ZSwiYmxvY2tlZCI6ZmFsc2V9fX0.dZiOjWHguo9f42XCMooMcv4EmYFzifl_-LEvPZHCtks`
@@ -56,29 +60,11 @@ func TestRest_FileServer(t *testing.T) {
assert.NoError(t, err)
body, code := get(t, ts.URL+"/web/"+testHTMLName)
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
assert.Equal(t, "some html", body)
_ = os.Remove(testHTMLFile)
}
func TestRest_GetStarted(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
getStartedHTML := os.TempDir() + "/getstarted.html"
err := os.WriteFile(getStartedHTML, []byte("some html blah"), 0o700)
assert.NoError(t, err)
body, code := get(t, ts.URL+"/index.html")
assert.Equal(t, 200, code)
assert.Equal(t, "some html blah", body)
_ = os.Remove(getStartedHTML)
_, code = get(t, ts.URL+"/index.html")
assert.Equal(t, 404, code)
}
func TestRest_Shutdown(t *testing.T) {
srv := Rest{Authenticator: &auth.Service{}, ImageProxy: &proxy.Image{}}
done := make(chan bool)
@@ -148,17 +134,18 @@ func TestRest_RunStaticSSLMode(t *testing.T) {
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
defer client.CloseIdleConnections()
resp, err := client.Get(fmt.Sprintf("http://localhost:%d/blah?param=1", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 307, resp.StatusCode)
assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
assert.Equal(t, fmt.Sprintf("https://localhost:%d/blah?param=1", sslPort), resp.Header.Get("Location"))
resp, err = client.Get(fmt.Sprintf("https://localhost:%d/ping", sslPort))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, "pong", string(body))
@@ -192,18 +179,18 @@ func TestRest_RunAutocertModeHTTPOnly(t *testing.T) {
return http.ErrUseLastResponse
},
}
defer client.CloseIdleConnections()
resp, err := client.Get(fmt.Sprintf("http://localhost:%d/blah?param=1", port))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 307, resp.StatusCode)
assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
assert.Equal(t, fmt.Sprintf("https://localhost:%d/blah?param=1", sslPort), resp.Header.Get("Location"))
srv.Shutdown()
}
func TestRest_rejectAnonUser(t *testing.T) {
ts := httptest.NewServer(fakeAuth(rejectAnonUser(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "Hello")
}))))
@@ -247,7 +234,6 @@ func Test_URLKey(t *testing.T) {
assert.Equal(t, tt.key, URLKey(r))
})
}
}
func Test_URLKeyWithUser(t *testing.T) {
@@ -273,7 +259,6 @@ func Test_URLKeyWithUser(t *testing.T) {
assert.Equal(t, tt.key, URLKeyWithUser(r))
})
}
}
func TestRest_parseError(t *testing.T) {
@@ -281,13 +266,13 @@ func TestRest_parseError(t *testing.T) {
err error
res int
}{
{errors.New("can not vote for his own comment"), rest.ErrVoteSelf},
{errors.New("already voted for"), rest.ErrVoteDbl},
{errors.New("maximum number of votes exceeded for comment"), rest.ErrVoteMax},
{errors.New("minimal score reached for comment"), rest.ErrVoteMinScore},
{errors.New("too late to edit"), rest.ErrCommentEditExpired},
{errors.New("parent comment with reply can't be edited"), rest.ErrCommentEditChanged},
{errors.New("blah blah"), rest.ErrInternal},
{fmt.Errorf("can not vote for his own comment"), rest.ErrVoteSelf},
{fmt.Errorf("already voted for"), rest.ErrVoteDbl},
{fmt.Errorf("maximum number of votes exceeded for comment"), rest.ErrVoteMax},
{fmt.Errorf("minimal score reached for comment"), rest.ErrVoteMinScore},
{fmt.Errorf("too late to edit"), rest.ErrCommentEditExpired},
{fmt.Errorf("parent comment with reply can't be edited"), rest.ErrCommentEditChanged},
{fmt.Errorf("blah blah"), rest.ErrInternal},
}
for n, tt := range tbl {
@@ -300,7 +285,6 @@ func TestRest_parseError(t *testing.T) {
}
func TestRest_cacheControl(t *testing.T) {
tbl := []struct {
url string
version string
@@ -317,7 +301,7 @@ func TestRest_cacheControl(t *testing.T) {
for i, tt := range tbl {
tt := tt
t.Run(strconv.Itoa(i), func(t *testing.T) {
req := httptest.NewRequest("GET", tt.url, nil)
req := httptest.NewRequest("GET", tt.url, http.NoBody)
w := httptest.NewRecorder()
h := cacheControl(tt.exp, tt.version)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
@@ -328,14 +312,11 @@ func TestRest_cacheControl(t *testing.T) {
t.Logf("%+v", resp.Header)
assert.Equal(t, `"`+tt.etag+`"`, resp.Header.Get("Etag"))
assert.Equal(t, `max-age=`+strconv.Itoa(int(tt.exp.Seconds()))+", no-cache", resp.Header.Get("Cache-Control"))
})
}
}
func TestRest_frameAncestors(t *testing.T) {
tbl := []struct {
hosts []string
header string
@@ -348,7 +329,7 @@ func TestRest_frameAncestors(t *testing.T) {
for i, tt := range tbl {
tt := tt
t.Run(strconv.Itoa(i), func(t *testing.T) {
req := httptest.NewRequest("GET", "http://example.com", nil)
req := httptest.NewRequest("GET", "http://example.com", http.NoBody)
w := httptest.NewRecorder()
h := frameAncestors(tt.hosts)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
@@ -358,14 +339,11 @@ func TestRest_frameAncestors(t *testing.T) {
assert.NoError(t, resp.Body.Close())
t.Logf("%+v", resp.Header)
assert.Equal(t, tt.header, resp.Header.Get("Content-Security-Policy"))
})
}
}
func TestRest_subscribersOnly(t *testing.T) {
paidSubUser := &token.User{}
paidSubUser.SetPaidSub(true)
@@ -385,7 +363,7 @@ func TestRest_subscribersOnly(t *testing.T) {
for i, tt := range tbl {
tt := tt
t.Run(strconv.Itoa(i), func(t *testing.T) {
req := httptest.NewRequest("GET", "http://example.com", nil)
req := httptest.NewRequest("GET", "http://example.com", http.NoBody)
if tt.setUser {
req = token.SetUserInfo(req, tt.user)
}
@@ -399,6 +377,38 @@ func TestRest_subscribersOnly(t *testing.T) {
}
}
func Test_validEmailAuth(t *testing.T) {
tbl := []struct {
req string
status int
}{
{"/auth/email/login?site=remark42&address=umputun%example.com&user=someone", http.StatusOK},
{"/auth/email/login?site=site-with-dash_and_underscore-and.dot&address=umputun%example.com&user=someone", http.StatusOK},
{"/auth/email/login?site=remark42&address=umputun%example.com&user=someone+blah", http.StatusOK},
{"/auth/email/login?site=remark42&address=umputun%example.com&user=Евгений+Умпутун", http.StatusOK},
{"/auth/email/login?site=remark42&address=umputun%example.com&user=12", http.StatusForbidden},
{"/auth/email/login?site=remark42&address=umputun%example.com&user=..blah+blah", http.StatusForbidden},
{"/auth/email/login?site=remark42&address=umputun%example.com&user=someonelooong+loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong", http.StatusForbidden},
{"/auth/twitter/login?site=remark42&address=umputun%example.com&user=..blah+blah", http.StatusOK},
{"/auth/email/login?site=remark42&address=umputun%example.com", http.StatusOK},
{"/auth/email/login?site=remark42&address=umputun+example.com&user=someone", http.StatusForbidden},
{"/auth/email/login?site=bad!site&address=umputun%example.com&user=someone", http.StatusForbidden},
{"/auth/email/login?site=loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongsite&address=umputun%example.com&user=someone", http.StatusForbidden},
}
for i, tt := range tbl {
t.Run(strconv.Itoa(i), func(t *testing.T) {
req := httptest.NewRequest("GET", "http://example.com"+tt.req, http.NoBody)
w := httptest.NewRecorder()
h := validEmailAuth()(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
h.ServeHTTP(w, req)
resp := w.Result()
assert.Equal(t, tt.status, resp.StatusCode)
assert.NoError(t, resp.Body.Close())
})
}
}
// randomPath pick a file or folder name which is not in use for sure
func randomPath(tempDir, basename, suffix string) (string, error) {
for i := 0; i < 10; i++ {
@@ -409,10 +419,12 @@ func randomPath(tempDir, basename, suffix string) (string, error) {
return fname, nil
}
}
return "", errors.New("cannot create temp file")
return "", fmt.Errorf("cannot create temp file in %s", tempDir)
}
func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
// startupT runs fully configured testing server
// srvHook is an optional func to set some Rest param after the creation but prior to Run
func startupT(t *testing.T, srvHook ...func(srv *Rest)) (ts *httptest.Server, srv *Rest, teardown func()) {
tmp := os.TempDir()
testDB, err := randomPath(tmp, "test-remark", ".db")
require.NoError(t, err)
@@ -477,6 +489,10 @@ func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
}
srv.ScoreThresholds.Low, srv.ScoreThresholds.Critical = -5, -10
for _, h := range srvHook {
h(srv)
}
ts = httptest.NewServer(srv.routes())
teardown = func() {
@@ -515,6 +531,7 @@ func get(t *testing.T, url string) (response string, statusCode int) {
func sendReq(_ *testing.T, r *http.Request, tkn string) (*http.Response, error) {
client := http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
if tkn != "" {
r.Header.Set("X-JWT", tkn)
}
@@ -523,6 +540,7 @@ func sendReq(_ *testing.T, r *http.Request, tkn string) (*http.Response, error)
func getWithDevAuth(t *testing.T, url string) (body string, code int) {
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("GET", url, http.NoBody)
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
@@ -536,6 +554,7 @@ func getWithDevAuth(t *testing.T, url string) (body string, code int) {
func getWithAdminAuth(t *testing.T, url string) (response string, statusCode int) {
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("GET", url, http.NoBody)
require.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -548,6 +567,7 @@ func getWithAdminAuth(t *testing.T, url string) (response string, statusCode int
}
func post(t *testing.T, url, body string) (*http.Response, error) {
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", url, strings.NewReader(body))
assert.NoError(t, err)
req.SetBasicAuth("admin", "password")
@@ -559,6 +579,7 @@ func addComment(t *testing.T, c store.Comment, ts *httptest.Server) string {
require.NoError(t, err, "can't marshal comment %+v", c)
client := &http.Client{Timeout: 5 * time.Second}
defer client.CloseIdleConnections()
req, err := http.NewRequest("POST", ts.URL+"/api/v1/comment", bytes.NewBuffer(b))
require.NoError(t, err)
req.Header.Add("X-JWT", devToken)
@@ -580,12 +601,12 @@ func requireAdminOnly(t *testing.T, req *http.Request) {
resp, err := sendReq(t, req, "") // no-auth user
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 401, resp.StatusCode)
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
resp, err = sendReq(t, req, devToken) // non-admin user
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
assert.Equal(t, 403, resp.StatusCode)
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
}
func chooseRandomUnusedPort() (port int) {
+1 -4
View File
@@ -8,7 +8,6 @@ import (
cache "github.com/go-pkgz/lcw"
log "github.com/go-pkgz/lgr"
"github.com/gorilla/feeds"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/rest"
"github.com/umputun/remark42/backend/app/store"
@@ -102,10 +101,9 @@ func (s *rss) repliesCtrl(w http.ResponseWriter, r *http.Request) {
key := cache.NewKey(siteID).ID(URLKey(r)).Scopes(siteID, lastCommentsScope)
data, err := s.cache.Get(key, func() (res []byte, e error) {
replies, userName, e := s.dataService.UserReplies(siteID, userID, maxRssItems, maxReplyDuration)
if e != nil {
return nil, errors.Wrap(e, "can't get last comments")
return nil, fmt.Errorf("can't get last comments: %w", e)
}
feed, e := s.toRssFeed(siteID, replies, "replies to "+userName)
@@ -128,7 +126,6 @@ func (s *rss) repliesCtrl(w http.ResponseWriter, r *http.Request) {
}
func (s *rss) toRssFeed(url string, comments []store.Comment, description string) (string, error) {
if description == "" {
description = "comment updates"
}
+8 -7
View File
@@ -2,6 +2,7 @@ package api
import (
"fmt"
"net/http"
"regexp"
"strings"
"testing"
@@ -31,7 +32,7 @@ func TestServer_RssPost(t *testing.T) {
pubDate := time.Now().Format(time.RFC1123Z)
res, code := get(t, ts.URL+"/api/v1/rss/post?site=remark42&url=https://radio-t.com/blah1")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
t.Log(res)
expected := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
@@ -55,7 +56,7 @@ func TestServer_RssPost(t *testing.T) {
assert.Equal(t, expected, res)
_, code = get(t, ts.URL+"/api/v1/rss/post?site=remark42-bad&url=https://radio-t.com/blah1")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
}
func TestServer_RssSite(t *testing.T) {
@@ -86,7 +87,7 @@ func TestServer_RssSite(t *testing.T) {
require.NoError(t, err)
res, code := get(t, ts.URL+"/api/v1/rss/site?site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
t.Log(res)
expected := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
@@ -118,7 +119,7 @@ func TestServer_RssSite(t *testing.T) {
assert.Equal(t, expected, res)
_, code = get(t, ts.URL+"/api/v1/rss/site?site=bad-radio-t")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
}
func TestServer_RssWithReply(t *testing.T) {
@@ -149,7 +150,7 @@ func TestServer_RssWithReply(t *testing.T) {
require.NoError(t, err)
res, code := get(t, ts.URL+"/api/v1/rss/post?site=remark42&url=https://radio-t.com/blah10")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
t.Log(res)
expected := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
@@ -236,7 +237,7 @@ func TestServer_RssReplies(t *testing.T) {
// replies to c1 (user1). Must be [c3, c2]
res, code := get(t, ts.URL+"/api/v1/rss/reply?user=user1&site=remark42")
assert.Equal(t, 200, code)
assert.Equal(t, http.StatusOK, code)
t.Log(res)
expected := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
@@ -266,7 +267,7 @@ func TestServer_RssReplies(t *testing.T) {
assert.Equal(t, expected, res)
_, code = get(t, ts.URL+"/api/v1/rss/reply?user=user1&site=remark42-bad")
assert.Equal(t, 400, code)
assert.Equal(t, http.StatusBadRequest, code)
}
func waitOnSecChange() {
-4
View File
@@ -111,15 +111,11 @@ func (s *Rest) getRemarkHost() string {
func (s *Rest) makeTLSConfig() *tls.Config {
return &tls.Config{
PreferServerCipherSuites: true,
CipherSuites: []uint16{
tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
// tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
// tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
// tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
},
MinVersion: tls.VersionTLS12,
CurvePreferences: []tls.CurveID{
+6 -4
View File
@@ -30,12 +30,13 @@ func TestSSL_Redirect(t *testing.T) {
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
}
defer client.CloseIdleConnections()
// check http to https redirect response
resp, err := client.Get(ts.URL + "/blah?param=1")
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 307, resp.StatusCode)
assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
assert.Equal(t, "https://localhost:443/blah?param=1", resp.Header.Get("Location"))
}
@@ -59,12 +60,13 @@ func TestSSL_ACME_HTTPChallengeRouter(t *testing.T) {
return http.ErrUseLastResponse
},
}
defer client.CloseIdleConnections()
// check http to https redirect response
resp, err := client.Get(ts.URL + "/blah?param=1")
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 307, resp.StatusCode)
assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
assert.Equal(t, "https://localhost:443/blah?param=1", resp.Header.Get("Location"))
// check acme http challenge
@@ -74,7 +76,7 @@ func TestSSL_ACME_HTTPChallengeRouter(t *testing.T) {
resp, err = client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 404, resp.StatusCode)
assert.Equal(t, http.StatusNotFound, resp.StatusCode)
err = m.Cache.Put(context.Background(), "token123+http-01", []byte("token"))
assert.NoError(t, err)
@@ -82,7 +84,7 @@ func TestSSL_ACME_HTTPChallengeRouter(t *testing.T) {
resp, err = client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
assert.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, "token", string(body))
Binary file not shown.

After

Width:  |  Height:  |  Size: 468 B

+3 -3
View File
@@ -48,9 +48,9 @@ type errTmplData struct {
Details string
}
// SendErrorHTML makes html body with provided template and responds with provided http status code,
// SendErrorHTML makes html body from error_response.html.tmpl template and responds with provided http status code,
// error code is not included in render as it is intended for UI developers and not for the users
func SendErrorHTML(w http.ResponseWriter, r *http.Request, httpStatusCode int, err error, details string, errCode int, t templates.FileReader) {
func SendErrorHTML(w http.ResponseWriter, r *http.Request, httpStatusCode int, err error, details string, errCode int) {
// MustExecute behaves like template.Execute, but panics if an error occurs.
MustExecute := func(tmpl *template.Template, wr io.Writer, data interface{}) {
if err = tmpl.Execute(wr, data); err != nil {
@@ -58,7 +58,7 @@ func SendErrorHTML(w http.ResponseWriter, r *http.Request, httpStatusCode int, e
}
}
MustRead := func(path string) string {
file, e := t.ReadFile(path)
file, e := templates.Read(path)
if e != nil {
panic(e)
}
+6 -15
View File
@@ -1,7 +1,6 @@
package rest
import (
"errors"
"fmt"
"io"
"net/http"
@@ -15,11 +14,10 @@ import (
)
func TestSendErrorJSON(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/error" {
t.Log("http err request", r.URL)
SendErrorJSON(w, r, 500, errors.New("error 500"), "error details 123456", 123)
SendErrorJSON(w, r, 500, fmt.Errorf("error 500"), "error details 123456", 123)
return
}
w.WriteHeader(404)
@@ -33,23 +31,16 @@ func TestSendErrorJSON(t *testing.T) {
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, 500, resp.StatusCode)
assert.Equal(t, http.StatusInternalServerError, resp.StatusCode)
assert.Equal(t, `{"code":123,"details":"error details 123456","error":"error 500"}`+"\n", string(body))
}
type MockFS struct{}
func (fs *MockFS) ReadFile(path string) ([]byte, error) {
return []byte(fmt.Sprintf("{{.Error}}{{.Details}} %s", path)), nil
}
func TestSendErrorHTML(t *testing.T) {
fs := &MockFS{}
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/error" {
t.Log("http err request", r.URL)
SendErrorHTML(w, r, 500, errors.New("error 500"), "error details 123456", 987, fs)
SendErrorHTML(w, r, 500, fmt.Errorf("error 500"), "error details 123456", 987)
return
}
w.WriteHeader(404)
@@ -63,7 +54,7 @@ func TestSendErrorHTML(t *testing.T) {
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
assert.Equal(t, 500, resp.StatusCode)
assert.Equal(t, http.StatusInternalServerError, resp.StatusCode)
assert.NotContains(t, string(body), `987`, "user html should not contain internal error code")
assert.Contains(t, string(body), `error details 123456`)
@@ -75,7 +66,7 @@ func TestErrorDetailsMsg(t *testing.T) {
req, err := http.NewRequest("GET", "https://example.com/test?k1=v1&k2=v2", http.NoBody)
require.NoError(t, err)
req.RemoteAddr = "1.2.3.4"
msg := errDetailsMsg(req, 500, errors.New("error 500"), "error details 123456", 123)
msg := errDetailsMsg(req, 500, fmt.Errorf("error 500"), "error details 123456", 123)
assert.Contains(t, msg, "error details 123456 - error 500 - 500 (123) - https://example.com/test?k1=v1&k2=v2 - [app/rest/httperrors_test.go:")
// error line in the middle of the message is not checked
assert.Contains(t, msg, " rest.TestErrorDetailsMsg]")
@@ -90,7 +81,7 @@ func TestErrorDetailsMsgWithUser(t *testing.T) {
req.RemoteAddr = "127.0.0.1:1234"
req = SetUserInfo(req, store.User{Name: "test", ID: "id"})
require.NoError(t, err)
msg := errDetailsMsg(req, 500, errors.New("error 500"), "error details 123456", 34567)
msg := errDetailsMsg(req, 500, fmt.Errorf("error 500"), "error details 123456", 34567)
assert.Contains(t, msg, "error details 123456 - error 500 - 500 (34567) - test/id - https://example.com/test?k1=v1&k2=v2 - [app/rest/httperrors_test.go:")
// error line in the middle of the message is not checked
assert.Contains(t, msg, " rest.TestErrorDetailsMsgWithUser]")
+8 -7
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"encoding/base64"
"fmt"
"io"
"net/http"
"strings"
@@ -12,7 +13,6 @@ import (
"github.com/PuerkitoBio/goquery"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/repeater"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/rest"
"github.com/umputun/remark42/backend/app/store/image"
@@ -54,7 +54,7 @@ func (p Image) Convert(commentHTML string) string {
func (p Image) extract(commentHTML string, imgSrcPred func(string) bool) ([]string, error) {
doc, err := goquery.NewDocumentFromReader(strings.NewReader(commentHTML))
if err != nil {
return nil, errors.Wrap(err, "can't create document")
return nil, fmt.Errorf("can't create document: %w", err)
}
result := []string{}
doc.Find("img").Each(func(i int, s *goquery.Selection) {
@@ -145,28 +145,29 @@ func (p Image) downloadImage(ctx context.Context, imgURL string) ([]byte, error)
defer cancel()
client := http.Client{Timeout: 30 * time.Second}
defer client.CloseIdleConnections()
var resp *http.Response
err := repeater.NewDefault(5, time.Second).Do(ctx, func() error {
var e error
req, e := http.NewRequest("GET", imgURL, http.NoBody)
if e != nil {
return errors.Wrapf(e, "failed to make request for %s", imgURL)
return fmt.Errorf("failed to make request for %s: %w", imgURL, e)
}
resp, e = client.Do(req.WithContext(ctx)) //nolint:bodyclose // need a refactor to fix that
return e
})
if err != nil {
return nil, errors.Wrapf(err, "can't download image %s", imgURL)
return nil, fmt.Errorf("can't download image %s: %w", imgURL, err)
}
defer resp.Body.Close()
defer resp.Body.Close() //nolint gosec // we don't care about response body
if resp.StatusCode != http.StatusOK {
return nil, errors.Errorf("got unsuccessful response status %d while fetching %s", resp.StatusCode, imgURL)
return nil, fmt.Errorf("got unsuccessful response status %d while fetching %s", resp.StatusCode, imgURL)
}
imgData, err := io.ReadAll(resp.Body)
if err != nil {
return nil, errors.Errorf("unable to read image body")
return nil, fmt.Errorf("unable to read image body")
}
return imgData, nil
}
+26 -25
View File
@@ -12,7 +12,6 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/umputun/remark42/backend/app/store/image"
@@ -95,7 +94,8 @@ func TestImage_Replace(t *testing.T) {
}
func TestImage_Routes(t *testing.T) {
imageStore := image.MockStore{}
// no image supposed to be cached
imageStore := image.StoreMock{LoadFunc: func(id string) ([]byte, error) { return nil, nil }}
img := Image{
HTTP2HTTPS: true,
RemarkURL: "https://demo.remark42.com",
@@ -110,9 +110,6 @@ func TestImage_Routes(t *testing.T) {
encodedImgURL := base64.URLEncoding.EncodeToString([]byte(httpSrv.URL + "/image/img1.png"))
// no image supposed to be cached
imageStore.On("Load", mock.Anything).Times(2).Return(nil, nil)
resp, err := http.Get(ts.URL + "/?src=" + encodedImgURL)
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
@@ -131,10 +128,11 @@ func TestImage_Routes(t *testing.T) {
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
assert.Equal(t, 2, len(imageStore.LoadCalls()))
}
func TestImage_DisabledCachingAndHTTP2HTTPS(t *testing.T) {
imageStore := image.MockStore{}
imageStore := image.StoreMock{LoadFunc: func(id string) ([]byte, error) { return nil, nil }}
img := Image{
RemarkURL: "https://demo.remark42.com",
RoutePath: "/api/v1/proxy",
@@ -148,8 +146,6 @@ func TestImage_DisabledCachingAndHTTP2HTTPS(t *testing.T) {
encodedImgURL := base64.URLEncoding.EncodeToString([]byte(httpSrv.URL + "/image/img1.png"))
imageStore.On("Load", mock.Anything).Once().Return(nil, nil)
resp, err := http.Get(ts.URL + "/?src=" + encodedImgURL)
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
@@ -157,11 +153,18 @@ func TestImage_DisabledCachingAndHTTP2HTTPS(t *testing.T) {
assert.Equal(t, "1462", resp.Header["Content-Length"][0])
assert.Equal(t, "image/png", resp.Header["Content-Type"][0])
imageStore.AssertCalled(t, "Load", mock.Anything)
assert.Equal(t, 1, len(imageStore.LoadCalls()))
}
func TestImage_RoutesCachingImage(t *testing.T) {
imageStore := image.MockStore{}
imageStore := image.StoreMock{
LoadFunc: func(id string) ([]byte, error) {
return nil, nil
},
SaveFunc: func(id string, img []byte) error {
return nil
},
}
img := Image{
CacheExternal: true,
RemarkURL: "https://demo.remark42.com",
@@ -177,9 +180,6 @@ func TestImage_RoutesCachingImage(t *testing.T) {
imgURL := httpSrv.URL + "/image/img1.png"
encodedImgURL := base64.URLEncoding.EncodeToString([]byte(imgURL))
imageStore.On("Load", mock.Anything).Once().Return(nil, nil)
imageStore.On("Save", mock.Anything, mock.Anything).Once().Return(nil)
resp, err := http.Get(ts.URL + "/?src=" + encodedImgURL)
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
@@ -187,12 +187,18 @@ func TestImage_RoutesCachingImage(t *testing.T) {
assert.Equal(t, "1462", resp.Header["Content-Length"][0])
assert.Equal(t, "image/png", resp.Header["Content-Type"][0])
imageStore.AssertCalled(t, "Load", mock.Anything)
imageStore.AssertCalled(t, "Save", "cached_images/4b84b15bff6ee5796152495a230e45e3d7e947d9-"+image.Sha1Str(imgURL), gopherPNGBytes())
assert.Equal(t, 1, len(imageStore.LoadCalls()))
assert.Equal(t, 1, len(imageStore.SaveCalls()))
assert.Equal(t, "cached_images/4b84b15bff6ee5796152495a230e45e3d7e947d9-"+image.Sha1Str(imgURL), imageStore.SaveCalls()[0].ID)
assert.Equal(t, gopherPNGBytes(), imageStore.SaveCalls()[0].Img)
}
func TestImage_RoutesUsingCachedImage(t *testing.T) {
imageStore := image.MockStore{}
// In order to validate that cached data used cache "will return" some other data from what http server would
testImage := []byte(fmt.Sprintf("%256s", "X"))
imageStore := image.StoreMock{LoadFunc: func(id string) ([]byte, error) {
return testImage, nil
}}
img := Image{
CacheExternal: true,
RemarkURL: "https://demo.remark42.com",
@@ -207,10 +213,6 @@ func TestImage_RoutesUsingCachedImage(t *testing.T) {
encodedImgURL := base64.URLEncoding.EncodeToString([]byte(httpSrv.URL + "/image/img1.png"))
// In order to validate that cached data used cache "will return" some other data from what http server would
testImage := []byte(fmt.Sprintf("%256s", "X"))
imageStore.On("Load", mock.Anything).Once().Return(testImage, nil)
resp, err := http.Get(ts.URL + "/?src=" + encodedImgURL)
require.NoError(t, err)
assert.NoError(t, resp.Body.Close())
@@ -219,11 +221,12 @@ func TestImage_RoutesUsingCachedImage(t *testing.T) {
assert.Equal(t, "text/plain; charset=utf-8", resp.Header["Content-Type"][0],
"if you save text you receive text/plain in response, that's only fair option you got")
imageStore.AssertCalled(t, "Load", mock.Anything)
assert.Equal(t, 1, len(imageStore.LoadCalls()))
}
func TestImage_RoutesTimedOut(t *testing.T) {
imageStore := image.MockStore{}
// no image supposed to be cached
imageStore := image.StoreMock{LoadFunc: func(id string) ([]byte, error) { return nil, nil }}
img := Image{
HTTP2HTTPS: true,
RemarkURL: "https://demo.remark42.com",
@@ -239,9 +242,6 @@ func TestImage_RoutesTimedOut(t *testing.T) {
encodedImgURL := base64.URLEncoding.EncodeToString([]byte(httpSrv.URL + "/image/img-slow.png"))
// no image supposed to be cached
imageStore.On("Load", mock.Anything).Once().Return(nil, nil)
resp, err := http.Get(ts.URL + "/?src=" + encodedImgURL)
require.NoError(t, err)
assert.Equal(t, http.StatusNotFound, resp.StatusCode)
@@ -250,6 +250,7 @@ func TestImage_RoutesTimedOut(t *testing.T) {
require.NoError(t, err)
t.Log(string(b))
assert.True(t, strings.Contains(string(b), "deadline exceeded"))
assert.Equal(t, 1, len(imageStore.LoadCalls()))
}
func TestImage_ConvertProxyMode(t *testing.T) {
+2 -3
View File
@@ -1,10 +1,10 @@
package rest
import (
"fmt"
"net/http"
"github.com/go-pkgz/auth/token"
"github.com/pkg/errors"
"github.com/umputun/remark42/backend/app/store"
)
@@ -21,10 +21,9 @@ func MustGetUserInfo(r *http.Request) store.User {
// GetUserInfo returns user from request context
func GetUserInfo(r *http.Request) (user store.User, err error) {
u, err := token.GetUserInfo(r)
if err != nil {
return store.User{}, errors.Wrap(err, "can't extract user info from the token")
return store.User{}, fmt.Errorf("can't extract user info from the token: %w", err)
}
return store.User{
+3 -3
View File
@@ -2,7 +2,7 @@
package admin
import (
"errors"
"fmt"
"strings"
log "github.com/go-pkgz/lgr"
@@ -50,7 +50,7 @@ func NewStaticKeyStore(key string) *StaticStore {
// Key returns static key, same for all sites
func (s *StaticStore) Key(_ string) (key string, err error) {
if s.key == "" {
return "", errors.New("empty key for static key store")
return "", fmt.Errorf("empty key for static key store")
}
return s.key, nil
}
@@ -78,5 +78,5 @@ func (s *StaticStore) Enabled(site string) (ok bool, err error) {
return false, nil
}
// OnEvent doesn nothing for StaticStore
// OnEvent does nothing for StaticStore
func (s *StaticStore) OnEvent(_ string, _ EventType) error { return nil }
+18 -10
View File
@@ -15,7 +15,7 @@ type Comment struct {
ID string `json:"id" bson:"_id"`
ParentID string `json:"pid"`
Text string `json:"text"`
Orig string `json:"orig,omitempty"`
Orig string `json:"orig,omitempty"` // important: never render this as HTML! It's not sanitized.
User User `json:"user"`
Locator Locator `json:"locator"`
Score int `json:"score"`
@@ -112,25 +112,28 @@ func (c *Comment) SetDeleted(mode DeleteMode) {
}
}
// Sanitize clean dangerous html/js from the comment
// Sanitize clean dangerous html/js from the comment.
// Comment.Orig which is used to store the original comment text is not sanitized
// as we expect to never render it as HTML and render Comment.Text instead
func (c *Comment) Sanitize() {
p := bluemonday.UGCPolicy()
p.AllowAttrs("class").Matching(regexp.MustCompile("^chroma$")).OnElements("pre")
// special case for embedding the quotes from Twitter
p.AllowAttrs("class").Matching(regexp.MustCompile("^twitter-tweet$")).OnElements("blockquote")
// this is list of <span> tag classes which could be produced by chroma code renderer
// source: https://github.com/alecthomas/chroma/blob/022b6f4fc2c4aa819aac18363c8de3f70619200b/types.go#L221-L316
const codeSpanClassRegex = "^(chroma|ln|lnt|hl|lntable|lntd|w|err|x|esc|k|kc" +
// source: https://github.com/alecthomas/chroma/blob/c263f6f/types.go#L209-L306
const codeSpanClassRegex = "^(bg|chroma|line|ln|lnt|hl|lntable|lntd|lnlinks|cl|w|err|x|k|kc" +
"|kd|kn|kp|kr|kt|n|na|nb|bp|nc|no|nd|ni|ne|nf|fm|py|nl|nn|nx|nt|nv|vc|vg" +
"|vi|vm|l|ld|s|sa|sb|sc|dl|sd|s2|se|sh|si|sx|sr|s1|ss|m|mb|mf|mh|mi|il" +
"|mo|o|ow|p|c|ch|cm|cp|cpf|c1|cs|g|gd|ge|gr|gh|gi|go|gp|gs|gu|gt|gl)$"
p.AllowAttrs("class").Matching(regexp.MustCompile(codeSpanClassRegex)).OnElements("span")
p.AllowAttrs("loading").Matching(regexp.MustCompile("^(lazy|eager)$")).OnElements("img")
c.Text = p.Sanitize(c.Text)
c.Orig = p.Sanitize(c.Orig)
c.User.ID = template.HTMLEscapeString(c.User.ID)
c.User.Name = c.escapeHTMLWithSome(c.User.Name)
c.User.Name = c.SanitizeText(c.User.Name)
c.User.Picture = c.SanitizeAsURL(c.User.Picture)
c.Locator.URL = c.SanitizeAsURL(c.Locator.URL)
c.PostTitle = p.Sanitize(c.PostTitle)
c.PostTitle = c.SanitizeText(c.PostTitle)
}
// Snippet from comment's text
@@ -143,7 +146,7 @@ func (c *Comment) Snippet(limit int) string {
if size < limit {
return cleanText
}
snippet := []rune(cleanText)[:size]
snippet := []rune(cleanText)[:limit]
// go back in snippet and found the first space
for i := len(snippet) - 1; i >= 0; i-- {
if snippet[i] == ' ' {
@@ -151,6 +154,10 @@ func (c *Comment) Snippet(limit int) string {
break
}
}
// Don't add a space if comment is just a one single word which has been truncated.
if len(snippet) == limit {
return string(snippet) + "..."
}
return string(snippet) + " ..."
}
@@ -169,13 +176,14 @@ func (c *Comment) SanitizeAsURL(inp string) string {
func (c *Comment) escapeHTMLWithSome(inp string) string {
res := template.HTMLEscapeString(inp)
res = strings.Replace(res, "&amp;", "&", -1)
res = strings.Replace(res, "&#34;", "\"", -1)
res = strings.Replace(res, "&#39;", "'", -1)
res = strings.Replace(res, "&amp;", "&", -1)
return res
}
// SanitizeText used to sanitize any input string
func (c *Comment) SanitizeText(inp string) string {
return c.escapeHTMLWithSome(bluemonday.UGCPolicy().Sanitize(inp))
clean := bluemonday.UGCPolicy().Sanitize(inp)
return c.escapeHTMLWithSome(clean)
}
+24 -11
View File
@@ -9,7 +9,6 @@ import (
)
func TestComment_Sanitize(t *testing.T) {
tbl := []struct {
inp Comment
out Comment
@@ -36,6 +35,10 @@ func TestComment_Sanitize(t *testing.T) {
User: User{ID: "id", Name: "xyz-123"},
},
},
{
inp: Comment{Text: "blah `123`"},
out: Comment{Text: "blah `123`"},
},
{
inp: Comment{Text: "blah & & 123 &mdash; &mdash;"},
out: Comment{Text: `blah &amp; &amp; 123 — —`},
@@ -44,9 +47,17 @@ func TestComment_Sanitize(t *testing.T) {
inp: Comment{Text: "blah & & 123 — —"},
out: Comment{Text: `blah &amp; &amp; 123 — —`},
},
{
inp: Comment{Text: "blah `123`", User: User{Name: "name \"xxx\" `yyy`"}},
out: Comment{Text: "blah `123`", User: User{Name: "name \"xxx\" `yyy`"}},
},
{
inp: Comment{Text: "blah & & 123", User: User{Name: "name <> & ' ` \""}},
out: Comment{Text: `blah &amp; &amp; 123`, User: User{Name: "name &lt;&gt; & ' ` \""}},
out: Comment{Text: `blah &amp; &amp; 123`, User: User{Name: "name &lt;&gt; &amp; ' ` \""}},
},
{
inp: Comment{Text: "blah & & 123", User: User{Name: `name <a href="javascript:alert('XSS1')" onmouseover="alert('XSS2')">XSS</a>`}},
out: Comment{Text: `blah &amp; &amp; 123`, User: User{Name: "name XSS"}},
},
{
@@ -77,11 +88,17 @@ func TestComment_Sanitize(t *testing.T) {
inp: Comment{Text: "blah blah", PostTitle: "<script>alert()</script>something"},
out: Comment{Text: "blah blah", PostTitle: "something"},
},
{
inp: Comment{Text: `<blockquote class="twitter-tweet"><p lang="es" dir="ltr">Silicon iMac Concept<a href="https://t.co/7ga95QxVXn">https://t.co/7ga95QxVXn</a> by <a href="https://twitter.com/marcsheep?ref_src=twsrc%5Etfw">@marcsheep</a> <a href="https://t.co/ULnVpG8w55">pic.twitter.com/ULnVpG8w55</a></p>&mdash; Andreas Storm (@avstorm) <a href="https://twitter.com/avstorm/status/1325693387798933504?ref_src=twsrc%5Etfw">November 9, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>`, PostTitle: "Twitter quote"},
out: Comment{Text: `<blockquote class="twitter-tweet"><p lang="es" dir="ltr">Silicon iMac Concept<a href="https://t.co/7ga95QxVXn" rel="nofollow">https://t.co/7ga95QxVXn</a> by <a href="https://twitter.com/marcsheep?ref_src=twsrc%5Etfw" rel="nofollow">@marcsheep</a> <a href="https://t.co/ULnVpG8w55" rel="nofollow">pic.twitter.com/ULnVpG8w55</a></p>— Andreas Storm (@avstorm) <a href="https://twitter.com/avstorm/status/1325693387798933504?ref_src=twsrc%5Etfw" rel="nofollow">November 9, 2020</a></blockquote> `, PostTitle: "Twitter quote"},
},
}
for n, tt := range tbl {
tt.inp.Sanitize()
assert.Equal(t, tt.out, tt.inp, "check #%d", n)
t.Run(strconv.Itoa(n), func(t *testing.T) {
tt.inp.Sanitize()
assert.Equal(t, tt.out, tt.inp, "check #%d", n)
})
}
}
@@ -114,7 +131,6 @@ func TestComment_PrepareUntrusted(t *testing.T) {
assert.Equal(t, User{ID: "username"}, comment.User)
assert.Equal(t, 0., comment.Controversy)
assert.Equal(t, false, comment.Imported)
}
func TestComment_SetDeleted(t *testing.T) {
@@ -180,8 +196,8 @@ func TestComment_Snippet(t *testing.T) {
{0, "", ""},
{-1, "test\nblah", "test blah"},
{5, "test\nblah", "test ..."},
{5, "xyz12345 xxx", "xyz12345 ..."},
{10, "xyz12345 xxx\ntest 123456", "xyz12345 xxx test ..."},
{5, "xyz12345 xxx", "xyz12..."},
{10, "xyz12345 xxx\ntest 123456", "xyz12345 ..."},
}
for i, tt := range tbl {
@@ -195,7 +211,6 @@ func TestComment_Snippet(t *testing.T) {
}
func TestComment_sanitizeAsURL(t *testing.T) {
tbl := []struct {
inp, out string
}{
@@ -231,7 +246,6 @@ func TestComment_sanitizeAsURL(t *testing.T) {
}
func TestComment_sanitizeText(t *testing.T) {
tbl := []struct {
inp, out string
}{
@@ -249,7 +263,7 @@ func TestComment_sanitizeText(t *testing.T) {
},
{
"<a href=javascript:alert(document.domain)//>xxx</a>",
"&lt;a/&gt;xxx&lt;/a&gt;",
"xxx&lt;/a&gt;",
},
}
@@ -260,5 +274,4 @@ func TestComment_sanitizeText(t *testing.T) {
assert.Equal(t, tt.out, c.SanitizeText(tt.inp))
})
}
}
+89 -90
View File
@@ -9,7 +9,6 @@ import (
log "github.com/go-pkgz/lgr"
"github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
bolt "go.etcd.io/bbolt"
"github.com/umputun/remark42/backend/app/store"
@@ -17,15 +16,15 @@ import (
// BoltDB implements store.Interface, represents multiple sites with multiplexing to different bolt dbs. Thread safe.
// there are 6 types of top-level buckets:
// - comments for post in "posts" top-level bucket. Each url (post) makes its own bucket and each k:v pair is commentID:comment
// - history of all comments. They all in a single "last" bucket (per site) and key is defined by ref struct as ts+commentID
// value is not full comment but a reference combined from post-url+commentID
// - user to comment references in "users" bucket. It used to get comments for user. Key is userID and value
// is a nested bucket named userID with kv as ts:reference
// - users details in "user_details" bucket. Key is userID, value - UserDetailEntry
// - blocking info sits in "block" bucket. Key is userID, value - ts
// - counts per post to keep number of comments. Key is post url, value - count
// - readonly per post to keep status of manually set RO posts. Key is post url, value - ts
// - comments for post in "posts" top-level bucket. Each url (post) makes its own bucket and each k:v pair is commentID:comment
// - history of all comments. They all in a single "last" bucket (per site) and key is defined by ref struct as ts+commentID
// value is not full comment but a reference combined from post-url+commentID
// - user to comment references in "users" bucket. It used to get comments for user. Key is userID and value
// is a nested bucket named userID with kv as ts:reference
// - users details in "user_details" bucket. Key is userID, value - UserDetailEntry
// - blocking info sits in "block" bucket. Key is userID, value - ts
// - counts per post to keep number of comments. Key is post url, value - count
// - readonly per post to keep status of manually set RO posts. Key is post url, value - ts
type BoltDB struct {
dbs map[string]*bolt.DB
}
@@ -57,7 +56,7 @@ func NewBoltDB(options bolt.Options, sites ...BoltSite) (*BoltDB, error) {
for _, site := range sites {
db, err := bolt.Open(site.FileName, 0o600, &options) //nolint:gocritic //octalLiteral is OK as FileMode
if err != nil {
return nil, errors.Wrapf(err, "failed to make boltdb for %s", site.FileName)
return nil, fmt.Errorf("failed to make boltdb for %s: %w", site.FileName, err)
}
// make top-level buckets
@@ -66,14 +65,14 @@ func NewBoltDB(options bolt.Options, sites ...BoltSite) (*BoltDB, error) {
err = db.Update(func(tx *bolt.Tx) error {
for _, bktName := range topBuckets {
if _, e := tx.CreateBucketIfNotExists([]byte(bktName)); e != nil {
return errors.Wrapf(e, "failed to create top level bucket %s", bktName)
return fmt.Errorf("failed to create top level bucket %s: %w", bktName, e)
}
}
return nil
})
if err != nil {
return nil, errors.Wrap(err, "failed to create top level bucket)")
return nil, fmt.Errorf("failed to create top level bucket): %w", err)
}
result.dbs[site.SiteID] = db
@@ -90,7 +89,7 @@ func (b *BoltDB) Create(comment store.Comment) (commentID string, err error) {
}
if b.checkFlag(FlagRequest{Locator: comment.Locator, Flag: ReadOnly}) {
return "", errors.Errorf("post %s is read-only", comment.Locator.URL)
return "", fmt.Errorf("post %s is read-only", comment.Locator.URL)
}
err = bdb.Update(func(tx *bolt.Tx) (err error) {
@@ -101,12 +100,12 @@ func (b *BoltDB) Create(comment store.Comment) (commentID string, err error) {
}
// check if key already in store, reject doubles
if postBkt.Get([]byte(comment.ID)) != nil {
return errors.Errorf("key %s already in store", comment.ID)
return fmt.Errorf("key %s already in store", comment.ID)
}
// serialize comment to json []byte for bolt and save
if err = b.save(postBkt, comment.ID, comment); err != nil {
return errors.Wrapf(err, "failed to put key %s to bucket %s", comment.ID, comment.Locator.URL)
return fmt.Errorf("failed to put key %s to bucket %s: %w", comment.ID, comment.Locator.URL, err)
}
ref := b.makeRef(comment) // reference combines url and comment id
@@ -115,21 +114,21 @@ func (b *BoltDB) Create(comment store.Comment) (commentID string, err error) {
lastBkt = tx.Bucket([]byte(lastBucketName))
commentTS := []byte(comment.Timestamp.Format(tsNano))
if err = lastBkt.Put(commentTS, ref); err != nil {
return errors.Wrapf(err, "can't put reference %s to %s", ref, lastBucketName)
return fmt.Errorf("can't put reference %s to %s: %w", ref, lastBucketName, err)
}
// add reference to commentID to "users" bucket
if userBkt, err = b.getUserBucket(tx, comment.User.ID); err != nil {
return errors.Wrapf(err, "can't get bucket %s", comment.User.ID)
return fmt.Errorf("can't get bucket %s: %w", comment.User.ID, err)
}
// put into individual user's bucket with ts as a key
if err = userBkt.Put(commentTS, ref); err != nil {
return errors.Wrapf(err, "failed to put user comment %s for %s", comment.ID, comment.User.ID)
return fmt.Errorf("failed to put user comment %s for %s: %w", comment.ID, comment.User.ID, err)
}
// set info with the count for post url
if _, err = b.setInfo(tx, comment); err != nil {
return errors.Wrapf(err, "failed to set info for %s", comment.Locator)
return fmt.Errorf("failed to set info for %s: %w", comment.Locator, err)
}
return nil
})
@@ -139,7 +138,6 @@ func (b *BoltDB) Create(comment store.Comment) (commentID string, err error) {
// Get returns comment for locator.URL and commentID string
func (b *BoltDB) Get(req GetRequest) (comment store.Comment, err error) {
bdb, err := b.db(req.Locator.SiteID)
if err != nil {
return comment, err
@@ -167,7 +165,6 @@ func (b *BoltDB) Find(req FindRequest) (comments []store.Comment, err error) {
switch {
case req.Locator.SiteID != "" && req.Locator.URL != "": // find post comments, i.e. for site and url
err = bdb.View(func(tx *bolt.Tx) error {
bucket, e := b.getPostBucket(tx, req.Locator.URL)
if e != nil {
return e
@@ -176,7 +173,7 @@ func (b *BoltDB) Find(req FindRequest) (comments []store.Comment, err error) {
return bucket.ForEach(func(k, v []byte) error {
comment := store.Comment{}
if e = json.Unmarshal(v, &comment); e != nil {
return errors.Wrap(e, "failed to unmarshal")
return fmt.Errorf("failed to unmarshal: %w", e)
}
if req.Since.IsZero() || comment.Timestamp.After(req.Since) {
comments = append(comments, comment)
@@ -213,7 +210,7 @@ func (b *BoltDB) UserDetail(req UserDetailRequest) ([]UserDetailEntry, error) {
switch req.Detail {
case UserEmail, UserTelegram:
if req.UserID == "" {
return nil, errors.New("userid cannot be empty in request for single detail")
return nil, fmt.Errorf("userid cannot be empty in request for single detail")
}
if req.Update == "" { // read detail value, no update requested
@@ -227,15 +224,14 @@ func (b *BoltDB) UserDetail(req UserDetailRequest) ([]UserDetailEntry, error) {
if req.Update == "" && req.UserID == "" { // read list of all details
return b.listDetails(req.Locator)
}
return nil, errors.New("unsupported request with userdetail all")
return nil, fmt.Errorf("unsupported request with userdetail all")
default:
return nil, errors.Errorf("unsupported detail %q", req.Detail)
return nil, fmt.Errorf("unsupported detail %q", req.Detail)
}
}
// Update for locator.URL with mutable part of comment
func (b *BoltDB) Update(comment store.Comment) error {
getReq := GetRequest{Locator: comment.Locator, CommentID: comment.ID}
if curComment, err := b.Get(getReq); err == nil {
// preserve immutable fields
@@ -261,7 +257,6 @@ func (b *BoltDB) Update(comment store.Comment) error {
// Count returns number of comments for post or user
func (b *BoltDB) Count(req FindRequest) (count int, err error) {
bdb, err := b.db(req.Locator.SiteID)
if err != nil {
return 0, err
@@ -281,7 +276,7 @@ func (b *BoltDB) Count(req FindRequest) (count int, err error) {
usersBkt := tx.Bucket([]byte(userBucketName))
userIDBkt := usersBkt.Bucket([]byte(req.UserID))
if userIDBkt == nil {
return errors.Errorf("no comments for user %s in store for %s site", req.UserID, req.Locator.SiteID)
return fmt.Errorf("no comments for user %s in store for %s site", req.UserID, req.Locator.SiteID)
}
stats := userIDBkt.Stats()
count = stats.KeyN
@@ -290,12 +285,11 @@ func (b *BoltDB) Count(req FindRequest) (count int, err error) {
return count, err
}
return 0, errors.Errorf("invalid count request %+v", req)
return 0, fmt.Errorf("invalid count request %+v", req)
}
// Info get post(s) meta info
func (b *BoltDB) Info(req InfoRequest) ([]store.PostInfo, error) {
bdb, err := b.db(req.Locator.SiteID)
if err != nil {
return []store.PostInfo{}, err
@@ -306,7 +300,7 @@ func (b *BoltDB) Info(req InfoRequest) ([]store.PostInfo, error) {
err = bdb.View(func(tx *bolt.Tx) error {
infoBkt := tx.Bucket([]byte(infoBucketName))
if e := b.load(infoBkt, req.Locator.URL, &info); e != nil {
return errors.Wrapf(e, "can't load info for %s", req.Locator.URL)
return fmt.Errorf("can't load info for %s: %w", req.Locator.URL, e)
}
return nil
})
@@ -336,7 +330,7 @@ func (b *BoltDB) Info(req InfoRequest) ([]store.PostInfo, error) {
infoBkt := tx.Bucket([]byte(infoBucketName))
info := store.PostInfo{}
if e := b.load(infoBkt, postURL, &info); e != nil {
return errors.Wrapf(e, "can't load info for %s", postURL)
return fmt.Errorf("can't load info for %s: %w", postURL, e)
}
list = append(list, info)
if req.Limit > 0 && len(list) >= req.Limit {
@@ -348,13 +342,12 @@ func (b *BoltDB) Info(req InfoRequest) ([]store.PostInfo, error) {
return list, err
}
return nil, errors.Errorf("invalid info request %+v", req)
return nil, fmt.Errorf("invalid info request %+v", req)
}
// ListFlags get list of flagged keys, like blocked & verified user
// works for full locator (post flags) or with userID
func (b *BoltDB) ListFlags(req FlagRequest) (res []interface{}, err error) {
bdb, e := b.db(req.Locator.SiteID)
if e != nil {
return nil, e
@@ -378,7 +371,7 @@ func (b *BoltDB) ListFlags(req FlagRequest) (res []interface{}, err error) {
return bucket.ForEach(func(k []byte, v []byte) error {
ts, errParse := time.ParseInLocation(tsNano, string(v), time.Local)
if errParse != nil {
return errors.Wrap(errParse, "can't parse block ts")
return fmt.Errorf("can't parse block ts: %w", errParse)
}
if time.Now().Before(ts) {
// get user name from comment user section
@@ -395,12 +388,11 @@ func (b *BoltDB) ListFlags(req FlagRequest) (res []interface{}, err error) {
})
return res, err
}
return nil, errors.Errorf("flag %s not listable", req.Flag)
return nil, fmt.Errorf("flag %s not listable", req.Flag)
}
// Delete post(s), user, comment, user details, or everything
func (b *BoltDB) Delete(req DeleteRequest) error {
bdb, e := b.db(req.Locator.SiteID)
if e != nil {
return e
@@ -417,22 +409,23 @@ func (b *BoltDB) Delete(req DeleteRequest) error {
return b.deleteAll(bdb, req.Locator.SiteID)
}
return errors.Errorf("invalid delete request %+v", req)
return fmt.Errorf("invalid delete request %+v", req)
}
// Close boltdb store
func (b *BoltDB) Close() error {
errs := new(multierror.Error)
for site, db := range b.dbs {
err := errors.Wrapf(db.Close(), "can't close site %s", site)
errs = multierror.Append(errs, err)
err := db.Close()
if err != nil {
errs = multierror.Append(errs, fmt.Errorf("can't close site %s: %w", site, err))
}
}
return errs.ErrorOrNil()
}
// Last returns up to max last comments for given siteID
func (b *BoltDB) lastComments(siteID string, max int, since time.Time) (comments []store.Comment, err error) {
comments = []store.Comment{}
if max > lastLimit || max == 0 {
@@ -449,7 +442,6 @@ func (b *BoltDB) lastComments(siteID string, max int, since time.Time) (comments
c := lastBkt.Cursor()
for k, v := c.Last(); k != nil; k, v = c.Prev() {
if !since.IsZero() {
// stop if reached "since" ts
tsSince := []byte(since.Format(tsNano))
@@ -488,7 +480,6 @@ func (b *BoltDB) lastComments(siteID string, max int, since time.Time) (comments
// userComments extracts all comments for given site and given userID
// "users" bucket has sub-bucket for each userID, and keeps it as ts:ref
func (b *BoltDB) userComments(siteID, userID string, limit, skip int) (comments []store.Comment, err error) {
comments = []store.Comment{}
commentRefs := []string{}
@@ -506,7 +497,7 @@ func (b *BoltDB) userComments(siteID, userID string, limit, skip int) (comments
usersBkt := tx.Bucket([]byte(userBucketName))
userIDBkt := usersBkt.Bucket([]byte(userID))
if userIDBkt == nil {
return errors.Errorf("no comments for user %s in store", userID)
return fmt.Errorf("no comments for user %s in store", userID)
}
c := userIDBkt.Cursor()
@@ -532,7 +523,7 @@ func (b *BoltDB) userComments(siteID, userID string, limit, skip int) (comments
for _, v := range commentRefs {
url, commentID, errParse := b.parseRef([]byte(v))
if errParse != nil {
return comments, errors.Wrapf(errParse, "can't parse reference %s", v)
return comments, fmt.Errorf("can't parse reference %s: %w", v, errParse)
}
getReq := GetRequest{Locator: store.Locator{SiteID: siteID, URL: url}, CommentID: commentID}
if c, errRef := b.Get(getReq); errRef == nil {
@@ -544,7 +535,6 @@ func (b *BoltDB) userComments(siteID, userID string, limit, skip int) (comments
}
func (b *BoltDB) checkFlag(req FlagRequest) (val bool) {
bdb, err := b.db(req.Locator.SiteID)
if err != nil {
return false
@@ -611,20 +601,20 @@ func (b *BoltDB) setFlag(req FlagRequest) (res bool, err error) {
val = time.Now().Add(req.TTL).Format(tsNano)
}
if e = bucket.Put([]byte(key), []byte(val)); e != nil {
return errors.Wrapf(e, "failed to put blocked to %s", key)
return fmt.Errorf("failed to put blocked to %s: %w", key, e)
}
res = true
return nil
}
if e = bucket.Put([]byte(key), []byte(time.Now().Format(tsNano))); e != nil {
return errors.Wrapf(e, "failed to set flag %s for %s", req.Flag, req.Locator.URL)
return fmt.Errorf("failed to set flag %s for %s: %w", req.Flag, req.Locator.URL, e)
}
res = true
return nil
case FlagFalse:
if e = bucket.Delete([]byte(key)); e != nil {
return errors.Wrapf(e, "failed to clean flag %s for %s", req.Flag, req.Locator.URL)
return fmt.Errorf("failed to clean flag %s for %s: %w", req.Flag, req.Locator.URL, e)
}
res = false
}
@@ -643,7 +633,7 @@ func (b *BoltDB) flagBucket(tx *bolt.Tx, flag Flag) (bkt *bolt.Bucket, err error
case Verified:
bkt = tx.Bucket([]byte(verifiedBucketName))
default:
return nil, errors.Errorf("unsupported flag %v", flag)
return nil, fmt.Errorf("unsupported flag %v", flag)
}
return bkt, nil
}
@@ -663,7 +653,7 @@ func (b *BoltDB) getUserDetail(req UserDetailRequest) (result []UserDetailEntry,
// return no error in case of absent entry
if value != nil {
if err = json.Unmarshal(value, &entry); err != nil {
return errors.Wrap(e, "failed to unmarshal entry")
return fmt.Errorf("failed to unmarshal entry: %w", e)
}
switch req.Detail {
case UserEmail:
@@ -693,7 +683,7 @@ func (b *BoltDB) setUserDetail(req UserDetailRequest) (result []UserDetailEntry,
// return no error in case of absent entry
if value != nil {
if err = json.Unmarshal(value, &entry); err != nil {
return errors.Wrap(e, "failed to unmarshal entry")
return fmt.Errorf("failed to unmarshal entry: %w", e)
}
}
return nil
@@ -716,7 +706,10 @@ func (b *BoltDB) setUserDetail(req UserDetailRequest) (result []UserDetailEntry,
err = bdb.Update(func(tx *bolt.Tx) error {
err = b.save(tx.Bucket([]byte(userDetailsBucketName)), req.UserID, entry)
return errors.Wrapf(err, "failed to update detail %s for %s in %s", req.Detail, req.UserID, req.Locator.SiteID)
if err != nil {
return fmt.Errorf("failed to update detail %s for %s in %s: %w", req.Detail, req.UserID, req.Locator.SiteID, err)
}
return nil
})
return []UserDetailEntry{entry}, err
@@ -734,7 +727,7 @@ func (b *BoltDB) listDetails(loc store.Locator) (result []UserDetailEntry, err e
bucket := tx.Bucket([]byte(userDetailsBucketName))
return bucket.ForEach(func(userID, value []byte) error {
if err = json.Unmarshal(value, &entry); err != nil {
return errors.Wrap(e, "failed to unmarshal entry")
return fmt.Errorf("failed to unmarshal entry: %w", e)
}
result = append(result, entry)
return nil
@@ -752,7 +745,7 @@ func (b *BoltDB) deleteUserDetail(bdb *bolt.DB, userID string, userDetail UserDe
// return no error in case of absent entry
if value != nil {
if err := json.Unmarshal(value, &entry); err != nil {
return errors.Wrap(err, "failed to unmarshal entry")
return fmt.Errorf("failed to unmarshal entry: %w", err)
}
}
return nil
@@ -779,21 +772,25 @@ func (b *BoltDB) deleteUserDetail(bdb *bolt.DB, userID string, userDetail UserDe
// if entry doesn't have non-empty details, we should delete it
return bdb.Update(func(tx *bolt.Tx) error {
err := tx.Bucket([]byte(userDetailsBucketName)).Delete([]byte(userID))
return errors.Wrapf(err, "failed to delete user detail %s for %s", userDetail, userID)
if err != nil {
return fmt.Errorf("failed to delete user detail %s for %s: %w", userDetail, userID, err)
}
return nil
})
}
return bdb.Update(func(tx *bolt.Tx) error {
// updated entry is not empty and we need to store it's updated copy
// updated entry is not empty and we need to store its updated copy
err := b.save(tx.Bucket([]byte(userDetailsBucketName)), userID, entry)
return errors.Wrapf(err, "failed to update detail %s for %s", userDetail, userID)
if err != nil {
return fmt.Errorf("failed to update detail %s for %s: %w", userDetail, userID, err)
}
return nil
})
}
func (b *BoltDB) deleteComment(bdb *bolt.DB, locator store.Locator, commentID string, mode store.DeleteMode) error {
return bdb.Update(func(tx *bolt.Tx) error {
postBkt, e := b.getPostBucket(tx, locator.URL)
if e != nil {
return e
@@ -801,13 +798,13 @@ func (b *BoltDB) deleteComment(bdb *bolt.DB, locator store.Locator, commentID st
comment := store.Comment{}
if e = b.load(postBkt, commentID, &comment); e != nil {
return errors.Wrapf(e, "can't load key %s from bucket %s", commentID, locator.URL)
return fmt.Errorf("can't load key %s from bucket %s: %w", commentID, locator.URL, e)
}
if !comment.Deleted {
// decrement comments count for post url
if _, e = b.count(tx, comment.Locator.URL, -1); e != nil {
return errors.Wrapf(e, "failed to decrement count for %s", comment.Locator)
return fmt.Errorf("failed to decrement count for %s: %w", comment.Locator, e)
}
}
@@ -815,13 +812,13 @@ func (b *BoltDB) deleteComment(bdb *bolt.DB, locator store.Locator, commentID st
comment.SetDeleted(mode)
if e = b.save(postBkt, commentID, comment); e != nil {
return errors.Wrapf(e, "can't save deleted comment for key %s from bucket %s", commentID, locator.URL)
return fmt.Errorf("can't save deleted comment for key %s from bucket %s: %w", commentID, locator.URL, e)
}
// delete from "last" bucket
lastBkt := tx.Bucket([]byte(lastBucketName))
if e = lastBkt.Delete([]byte(commentID)); e != nil {
return errors.Wrapf(e, "can't delete key %s from bucket %s", commentID, lastBucketName)
return fmt.Errorf("can't delete key %s from bucket %s: %w", commentID, lastBucketName, e)
}
return nil
@@ -830,31 +827,31 @@ func (b *BoltDB) deleteComment(bdb *bolt.DB, locator store.Locator, commentID st
// deleteAll removes all top-level buckets for given siteID
func (b *BoltDB) deleteAll(bdb *bolt.DB, siteID string) error {
// delete all buckets except blocked users
toDelete := []string{postsBucketName, lastBucketName, userBucketName, userDetailsBucketName, infoBucketName}
// delete top-level buckets
err := bdb.Update(func(tx *bolt.Tx) error {
for _, bktName := range toDelete {
if e := tx.DeleteBucket([]byte(bktName)); e != nil {
return errors.Wrapf(e, "failed to delete top level bucket %s", bktName)
return fmt.Errorf("failed to delete top level bucket %s: %w", bktName, e)
}
if _, e := tx.CreateBucketIfNotExists([]byte(bktName)); e != nil {
return errors.Wrapf(e, "failed to create top level bucket %s", bktName)
return fmt.Errorf("failed to create top level bucket %s: %w", bktName, e)
}
}
return nil
})
return errors.Wrapf(err, "failed to delete top level buckets from site %s", siteID)
if err != nil {
return fmt.Errorf("failed to delete top level buckets from site %s: %w", siteID, err)
}
return nil
}
// deleteUser removes all comments and details for given user. Everything will be market as deleted
// and user name and userID will be changed to "deleted". Also removes from last and from user buckets.
func (b *BoltDB) deleteUser(bdb *bolt.DB, siteID, userID string, mode store.DeleteMode) error {
// get list of all comments outside of transaction loop
posts, err := b.Info(InfoRequest{Locator: store.Locator{SiteID: siteID}})
if err != nil {
@@ -876,14 +873,17 @@ func (b *BoltDB) deleteUser(bdb *bolt.DB, siteID, userID string, mode store.Dele
err = postBkt.ForEach(func(postURL []byte, commentVal []byte) error {
comment := store.Comment{}
if err = json.Unmarshal(commentVal, &comment); err != nil {
return errors.Wrap(err, "failed to unmarshal")
return fmt.Errorf("failed to unmarshal: %w", err)
}
if comment.User.ID == userID {
comments = append(comments, commentInfo{locator: comment.Locator, commentID: comment.ID})
}
return nil
})
return errors.Wrapf(err, "failed to collect list of comments for deletion from %s", postInfo.URL)
if err != nil {
return fmt.Errorf("failed to collect list of comments for deletion from %s: %w", postInfo.URL, err)
}
return nil
})
if err != nil {
return err
@@ -895,7 +895,7 @@ func (b *BoltDB) deleteUser(bdb *bolt.DB, siteID, userID string, mode store.Dele
// delete collected comments
for _, ci := range comments {
if e := b.deleteComment(bdb, ci.locator, ci.commentID, mode); e != nil {
return errors.Wrapf(err, "failed to delete comment %+v", ci)
return fmt.Errorf("failed to delete comment %+v: %w", ci, err)
}
}
@@ -905,19 +905,19 @@ func (b *BoltDB) deleteUser(bdb *bolt.DB, siteID, userID string, mode store.Dele
usersBkt := tx.Bucket([]byte(userBucketName))
if usersBkt != nil {
if e := usersBkt.DeleteBucket([]byte(userID)); e != nil {
return errors.Wrapf(err, "failed to delete user bucket for %s", userID)
return fmt.Errorf("failed to delete user bucket for %s: %w", userID, err)
}
}
return nil
})
if err != nil {
return errors.Wrap(err, "can't delete user meta")
return fmt.Errorf("can't delete user meta: %w", err)
}
}
if len(comments) == 0 {
return errors.Errorf("unknown user %s", userID)
return fmt.Errorf("unknown user %s", userID)
}
return b.deleteUserDetail(bdb, userID, AllUserDetails)
@@ -927,11 +927,11 @@ func (b *BoltDB) deleteUser(bdb *bolt.DB, siteID, userID string, mode store.Dele
func (b *BoltDB) getPostBucket(tx *bolt.Tx, postURL string) (*bolt.Bucket, error) {
postsBkt := tx.Bucket([]byte(postsBucketName))
if postsBkt == nil {
return nil, errors.Errorf("no bucket %s", postsBucketName)
return nil, fmt.Errorf("no bucket %s", postsBucketName)
}
res := postsBkt.Bucket([]byte(postURL))
if res == nil {
return nil, errors.Errorf("no bucket %s in store", postURL)
return nil, fmt.Errorf("no bucket %s in store", postURL)
}
return res, nil
}
@@ -940,11 +940,11 @@ func (b *BoltDB) getPostBucket(tx *bolt.Tx, postURL string) (*bolt.Bucket, error
func (b *BoltDB) makePostBucket(tx *bolt.Tx, postURL string) (*bolt.Bucket, error) {
postsBkt := tx.Bucket([]byte(postsBucketName))
if postsBkt == nil {
return nil, errors.Errorf("no bucket %s", postsBucketName)
return nil, fmt.Errorf("no bucket %s", postsBucketName)
}
res, err := postsBkt.CreateBucketIfNotExists([]byte(postURL))
if err != nil {
return nil, errors.Wrapf(err, "no bucket %s in store", postURL)
return nil, fmt.Errorf("no bucket %s in store: %w", postURL, err)
}
return res, nil
}
@@ -953,7 +953,7 @@ func (b *BoltDB) getUserBucket(tx *bolt.Tx, userID string) (*bolt.Bucket, error)
usersBkt := tx.Bucket([]byte(userBucketName))
userIDBkt, e := usersBkt.CreateBucketIfNotExists([]byte(userID)) // get bucket for userID
if e != nil {
return nil, errors.Wrapf(e, "can't get bucket %s", userID)
return nil, fmt.Errorf("can't get bucket %s: %w", userID, e)
}
return userIDBkt, nil
}
@@ -961,14 +961,14 @@ func (b *BoltDB) getUserBucket(tx *bolt.Tx, userID string) (*bolt.Bucket, error)
// save marshaled value to key for bucket. Should run in update tx
func (b *BoltDB) save(bkt *bolt.Bucket, key string, value interface{}) (err error) {
if value == nil {
return errors.Errorf("can't save nil value for %s", key)
return fmt.Errorf("can't save nil value for %s", key)
}
jdata, jerr := json.Marshal(value)
if jerr != nil {
return errors.Wrap(jerr, "can't marshal comment")
return fmt.Errorf("can't marshal comment: %w", jerr)
}
if err = bkt.Put([]byte(key), jdata); err != nil {
return errors.Wrapf(err, "failed to save key %s", key)
return fmt.Errorf("failed to save key %s: %w", key, err)
}
return nil
}
@@ -977,11 +977,11 @@ func (b *BoltDB) save(bkt *bolt.Bucket, key string, value interface{}) (err erro
func (b *BoltDB) load(bkt *bolt.Bucket, key string, res interface{}) error {
value := bkt.Get([]byte(key))
if value == nil {
return errors.Errorf("no value for %s", key)
return fmt.Errorf("no value for %s", key)
}
if err := json.Unmarshal(value, &res); err != nil {
return errors.Wrap(err, "failed to unmarshal")
return fmt.Errorf("failed to unmarshal: %w", err)
}
return nil
}
@@ -989,7 +989,6 @@ func (b *BoltDB) load(bkt *bolt.Bucket, key string, res interface{}) error {
// count adds val to counts key postURL. val can be negative to subtract. if val 0 can be used as accessor
// it uses separate counts bucket because boltdb Stat call is very slow
func (b *BoltDB) count(tx *bolt.Tx, postURL string, val int) (int, error) {
infoBkt := tx.Bucket([]byte(infoBucketName))
info := store.PostInfo{}
@@ -1025,7 +1024,7 @@ func (b *BoltDB) db(siteID string) (*bolt.DB, error) {
if res, ok := b.dbs[siteID]; ok {
return res, nil
}
return nil, errors.Errorf("site %q not found", siteID)
return nil, fmt.Errorf("site %q not found", siteID)
}
// makeRef creates reference combining url and comment id
@@ -1037,7 +1036,7 @@ func (b *BoltDB) makeRef(comment store.Comment) []byte {
func (b *BoltDB) parseRef(val []byte) (url, id string, err error) {
elems := strings.Split(string(val), "!!")
if len(elems) != 2 {
return "", "", errors.Errorf("invalid reference value %s", string(val))
return "", "", fmt.Errorf("invalid reference value %s", string(val))
}
return elems[0], elems[1], nil
}
+7 -12
View File
@@ -416,7 +416,6 @@ func TestBoltDB_InfoList(t *testing.T) {
}
func TestBolt_FlagBlockedUser(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -461,7 +460,6 @@ func TestBolt_FlagBlockedUser(t *testing.T) {
}
func TestBolt_FlagReadOnlyPost(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -503,7 +501,6 @@ func TestBolt_FlagReadOnlyPost(t *testing.T) {
}
func TestBolt_FlagVerified(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -540,7 +537,6 @@ func TestBolt_FlagVerified(t *testing.T) {
}
func TestBolt_FlagListVerified(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -575,7 +571,6 @@ func TestBolt_FlagListVerified(t *testing.T) {
}
func TestBolt_FlagListBlocked(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -617,11 +612,9 @@ func TestBolt_FlagListBlocked(t *testing.T) {
_, err = b.ListFlags(FlagRequest{Flag: Blocked, Locator: store.Locator{SiteID: "bad"}})
assert.EqualError(t, err, `site "bad" not found`)
}
func TestBoltDB_UserDetail(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -672,7 +665,6 @@ func TestBoltDB_UserDetail(t *testing.T) {
}
func TestBolt_DeleteComment(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -728,7 +720,6 @@ func TestBolt_DeleteComment(t *testing.T) {
}
func TestBolt_DeleteHard(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -751,7 +742,6 @@ func TestBolt_DeleteHard(t *testing.T) {
}
func TestBolt_DeleteAll(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -811,7 +801,6 @@ func TestBolt_DeleteUserDetail(t *testing.T) {
}
func TestBoltAdmin_DeleteUserHard(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -852,7 +841,6 @@ func TestBoltAdmin_DeleteUserHard(t *testing.T) {
}
func TestBoltAdmin_DeleteUserSoft(t *testing.T) {
b, teardown := prep(t)
defer teardown()
@@ -923,6 +911,13 @@ func TestBoltDB_NewFailed(t *testing.T) {
assert.EqualError(t, err, "failed to make boltdb for /tmp/no-such-place/tmp.db: open /tmp/no-such-place/tmp.db: no such file or directory")
}
func TestBoltDB_DoubleClose(t *testing.T) {
var b, teardown = prep(t)
defer teardown()
assert.NoError(t, b.Close())
assert.NoError(t, b.Close(), "second call should not result in panic or errors")
}
// makes new boltdb, put two records
func prep(t *testing.T) (b *BoltDB, teardown func()) {
_ = os.Remove(testDB)
+2 -2
View File
@@ -11,8 +11,8 @@ import (
"github.com/umputun/remark42/backend/app/store"
)
// NOTE: mockery works from linked to go-path and with GOFLAGS='-mod=vendor' go generate
//go:generate sh -c "mockery -inpkg -name Interface -print > /tmp/engine-mock.tmp && mv /tmp/engine-mock.tmp engine_mock.go"
// NOTE: matryer/moq should be installed globally and works with `go generate ./...`
//go:generate moq --out engine_mock.go . Interface
// Interface defines methods provided by low-level storage engine
type Interface interface {
+464 -187
View File
@@ -1,231 +1,508 @@
// Code generated by mockery v1.1.2. DO NOT EDIT.
// Code generated by moq; DO NOT EDIT.
// github.com/matryer/moq
package engine
import (
mock "github.com/stretchr/testify/mock"
store "github.com/umputun/remark42/backend/app/store"
"sync"
)
// MockInterface is an autogenerated mock type for the Interface type
type MockInterface struct {
mock.Mock
}
// Ensure, that InterfaceMock does implement Interface.
// If this is not the case, regenerate this file with moq.
var _ Interface = &InterfaceMock{}
// Close provides a mock function with given fields:
func (_m *MockInterface) Close() error {
ret := _m.Called()
// InterfaceMock is a mock implementation of Interface.
//
// func TestSomethingThatUsesInterface(t *testing.T) {
//
// // make and configure a mocked Interface
// mockedInterface := &InterfaceMock{
// CloseFunc: func() error {
// panic("mock out the Close method")
// },
// CountFunc: func(req FindRequest) (int, error) {
// panic("mock out the Count method")
// },
// CreateFunc: func(comment store.Comment) (string, error) {
// panic("mock out the Create method")
// },
// DeleteFunc: func(req DeleteRequest) error {
// panic("mock out the Delete method")
// },
// FindFunc: func(req FindRequest) ([]store.Comment, error) {
// panic("mock out the Find method")
// },
// FlagFunc: func(req FlagRequest) (bool, error) {
// panic("mock out the Flag method")
// },
// GetFunc: func(req GetRequest) (store.Comment, error) {
// panic("mock out the Get method")
// },
// InfoFunc: func(req InfoRequest) ([]store.PostInfo, error) {
// panic("mock out the Info method")
// },
// ListFlagsFunc: func(req FlagRequest) ([]interface{}, error) {
// panic("mock out the ListFlags method")
// },
// UpdateFunc: func(comment store.Comment) error {
// panic("mock out the Update method")
// },
// UserDetailFunc: func(req UserDetailRequest) ([]UserDetailEntry, error) {
// panic("mock out the UserDetail method")
// },
// }
//
// // use mockedInterface in code that requires Interface
// // and then make assertions.
//
// }
type InterfaceMock struct {
// CloseFunc mocks the Close method.
CloseFunc func() error
var r0 error
if rf, ok := ret.Get(0).(func() error); ok {
r0 = rf()
} else {
r0 = ret.Error(0)
}
// CountFunc mocks the Count method.
CountFunc func(req FindRequest) (int, error)
return r0
}
// CreateFunc mocks the Create method.
CreateFunc func(comment store.Comment) (string, error)
// Count provides a mock function with given fields: req
func (_m *MockInterface) Count(req FindRequest) (int, error) {
ret := _m.Called(req)
// DeleteFunc mocks the Delete method.
DeleteFunc func(req DeleteRequest) error
var r0 int
if rf, ok := ret.Get(0).(func(FindRequest) int); ok {
r0 = rf(req)
} else {
r0 = ret.Get(0).(int)
}
// FindFunc mocks the Find method.
FindFunc func(req FindRequest) ([]store.Comment, error)
var r1 error
if rf, ok := ret.Get(1).(func(FindRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
}
// FlagFunc mocks the Flag method.
FlagFunc func(req FlagRequest) (bool, error)
return r0, r1
}
// GetFunc mocks the Get method.
GetFunc func(req GetRequest) (store.Comment, error)
// Create provides a mock function with given fields: comment
func (_m *MockInterface) Create(comment store.Comment) (string, error) {
ret := _m.Called(comment)
// InfoFunc mocks the Info method.
InfoFunc func(req InfoRequest) ([]store.PostInfo, error)
var r0 string
if rf, ok := ret.Get(0).(func(store.Comment) string); ok {
r0 = rf(comment)
} else {
r0 = ret.Get(0).(string)
}
// ListFlagsFunc mocks the ListFlags method.
ListFlagsFunc func(req FlagRequest) ([]interface{}, error)
var r1 error
if rf, ok := ret.Get(1).(func(store.Comment) error); ok {
r1 = rf(comment)
} else {
r1 = ret.Error(1)
}
// UpdateFunc mocks the Update method.
UpdateFunc func(comment store.Comment) error
return r0, r1
}
// UserDetailFunc mocks the UserDetail method.
UserDetailFunc func(req UserDetailRequest) ([]UserDetailEntry, error)
// Delete provides a mock function with given fields: req
func (_m *MockInterface) Delete(req DeleteRequest) error {
ret := _m.Called(req)
var r0 error
if rf, ok := ret.Get(0).(func(DeleteRequest) error); ok {
r0 = rf(req)
} else {
r0 = ret.Error(0)
}
return r0
}
// Find provides a mock function with given fields: req
func (_m *MockInterface) Find(req FindRequest) ([]store.Comment, error) {
ret := _m.Called(req)
var r0 []store.Comment
if rf, ok := ret.Get(0).(func(FindRequest) []store.Comment); ok {
r0 = rf(req)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]store.Comment)
// calls tracks calls to the methods.
calls struct {
// Close holds details about calls to the Close method.
Close []struct {
}
// Count holds details about calls to the Count method.
Count []struct {
// Req is the req argument value.
Req FindRequest
}
// Create holds details about calls to the Create method.
Create []struct {
// Comment is the comment argument value.
Comment store.Comment
}
// Delete holds details about calls to the Delete method.
Delete []struct {
// Req is the req argument value.
Req DeleteRequest
}
// Find holds details about calls to the Find method.
Find []struct {
// Req is the req argument value.
Req FindRequest
}
// Flag holds details about calls to the Flag method.
Flag []struct {
// Req is the req argument value.
Req FlagRequest
}
// Get holds details about calls to the Get method.
Get []struct {
// Req is the req argument value.
Req GetRequest
}
// Info holds details about calls to the Info method.
Info []struct {
// Req is the req argument value.
Req InfoRequest
}
// ListFlags holds details about calls to the ListFlags method.
ListFlags []struct {
// Req is the req argument value.
Req FlagRequest
}
// Update holds details about calls to the Update method.
Update []struct {
// Comment is the comment argument value.
Comment store.Comment
}
// UserDetail holds details about calls to the UserDetail method.
UserDetail []struct {
// Req is the req argument value.
Req UserDetailRequest
}
}
var r1 error
if rf, ok := ret.Get(1).(func(FindRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
}
return r0, r1
lockClose sync.RWMutex
lockCount sync.RWMutex
lockCreate sync.RWMutex
lockDelete sync.RWMutex
lockFind sync.RWMutex
lockFlag sync.RWMutex
lockGet sync.RWMutex
lockInfo sync.RWMutex
lockListFlags sync.RWMutex
lockUpdate sync.RWMutex
lockUserDetail sync.RWMutex
}
// Flag provides a mock function with given fields: req
func (_m *MockInterface) Flag(req FlagRequest) (bool, error) {
ret := _m.Called(req)
var r0 bool
if rf, ok := ret.Get(0).(func(FlagRequest) bool); ok {
r0 = rf(req)
} else {
r0 = ret.Get(0).(bool)
// Close calls CloseFunc.
func (mock *InterfaceMock) Close() error {
if mock.CloseFunc == nil {
panic("InterfaceMock.CloseFunc: method is nil but Interface.Close was just called")
}
var r1 error
if rf, ok := ret.Get(1).(func(FlagRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
}
return r0, r1
callInfo := struct {
}{}
mock.lockClose.Lock()
mock.calls.Close = append(mock.calls.Close, callInfo)
mock.lockClose.Unlock()
return mock.CloseFunc()
}
// Get provides a mock function with given fields: req
func (_m *MockInterface) Get(req GetRequest) (store.Comment, error) {
ret := _m.Called(req)
var r0 store.Comment
if rf, ok := ret.Get(0).(func(GetRequest) store.Comment); ok {
r0 = rf(req)
} else {
r0 = ret.Get(0).(store.Comment)
// CloseCalls gets all the calls that were made to Close.
// Check the length with:
//
// len(mockedInterface.CloseCalls())
func (mock *InterfaceMock) CloseCalls() []struct {
} {
var calls []struct {
}
var r1 error
if rf, ok := ret.Get(1).(func(GetRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
}
return r0, r1
mock.lockClose.RLock()
calls = mock.calls.Close
mock.lockClose.RUnlock()
return calls
}
// Info provides a mock function with given fields: req
func (_m *MockInterface) Info(req InfoRequest) ([]store.PostInfo, error) {
ret := _m.Called(req)
var r0 []store.PostInfo
if rf, ok := ret.Get(0).(func(InfoRequest) []store.PostInfo); ok {
r0 = rf(req)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]store.PostInfo)
}
// Count calls CountFunc.
func (mock *InterfaceMock) Count(req FindRequest) (int, error) {
if mock.CountFunc == nil {
panic("InterfaceMock.CountFunc: method is nil but Interface.Count was just called")
}
var r1 error
if rf, ok := ret.Get(1).(func(InfoRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
callInfo := struct {
Req FindRequest
}{
Req: req,
}
return r0, r1
mock.lockCount.Lock()
mock.calls.Count = append(mock.calls.Count, callInfo)
mock.lockCount.Unlock()
return mock.CountFunc(req)
}
// ListFlags provides a mock function with given fields: req
func (_m *MockInterface) ListFlags(req FlagRequest) ([]interface{}, error) {
ret := _m.Called(req)
var r0 []interface{}
if rf, ok := ret.Get(0).(func(FlagRequest) []interface{}); ok {
r0 = rf(req)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]interface{})
}
// CountCalls gets all the calls that were made to Count.
// Check the length with:
//
// len(mockedInterface.CountCalls())
func (mock *InterfaceMock) CountCalls() []struct {
Req FindRequest
} {
var calls []struct {
Req FindRequest
}
var r1 error
if rf, ok := ret.Get(1).(func(FlagRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
}
return r0, r1
mock.lockCount.RLock()
calls = mock.calls.Count
mock.lockCount.RUnlock()
return calls
}
// Update provides a mock function with given fields: comment
func (_m *MockInterface) Update(comment store.Comment) error {
ret := _m.Called(comment)
var r0 error
if rf, ok := ret.Get(0).(func(store.Comment) error); ok {
r0 = rf(comment)
} else {
r0 = ret.Error(0)
// Create calls CreateFunc.
func (mock *InterfaceMock) Create(comment store.Comment) (string, error) {
if mock.CreateFunc == nil {
panic("InterfaceMock.CreateFunc: method is nil but Interface.Create was just called")
}
return r0
callInfo := struct {
Comment store.Comment
}{
Comment: comment,
}
mock.lockCreate.Lock()
mock.calls.Create = append(mock.calls.Create, callInfo)
mock.lockCreate.Unlock()
return mock.CreateFunc(comment)
}
// UserDetail provides a mock function with given fields: req
func (_m *MockInterface) UserDetail(req UserDetailRequest) ([]UserDetailEntry, error) {
ret := _m.Called(req)
var r0 []UserDetailEntry
if rf, ok := ret.Get(0).(func(UserDetailRequest) []UserDetailEntry); ok {
r0 = rf(req)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]UserDetailEntry)
}
// CreateCalls gets all the calls that were made to Create.
// Check the length with:
//
// len(mockedInterface.CreateCalls())
func (mock *InterfaceMock) CreateCalls() []struct {
Comment store.Comment
} {
var calls []struct {
Comment store.Comment
}
mock.lockCreate.RLock()
calls = mock.calls.Create
mock.lockCreate.RUnlock()
return calls
}
var r1 error
if rf, ok := ret.Get(1).(func(UserDetailRequest) error); ok {
r1 = rf(req)
} else {
r1 = ret.Error(1)
// Delete calls DeleteFunc.
func (mock *InterfaceMock) Delete(req DeleteRequest) error {
if mock.DeleteFunc == nil {
panic("InterfaceMock.DeleteFunc: method is nil but Interface.Delete was just called")
}
callInfo := struct {
Req DeleteRequest
}{
Req: req,
}
mock.lockDelete.Lock()
mock.calls.Delete = append(mock.calls.Delete, callInfo)
mock.lockDelete.Unlock()
return mock.DeleteFunc(req)
}
// DeleteCalls gets all the calls that were made to Delete.
// Check the length with:
//
// len(mockedInterface.DeleteCalls())
func (mock *InterfaceMock) DeleteCalls() []struct {
Req DeleteRequest
} {
var calls []struct {
Req DeleteRequest
}
mock.lockDelete.RLock()
calls = mock.calls.Delete
mock.lockDelete.RUnlock()
return calls
}
// Find calls FindFunc.
func (mock *InterfaceMock) Find(req FindRequest) ([]store.Comment, error) {
if mock.FindFunc == nil {
panic("InterfaceMock.FindFunc: method is nil but Interface.Find was just called")
}
callInfo := struct {
Req FindRequest
}{
Req: req,
}
mock.lockFind.Lock()
mock.calls.Find = append(mock.calls.Find, callInfo)
mock.lockFind.Unlock()
return mock.FindFunc(req)
}
// FindCalls gets all the calls that were made to Find.
// Check the length with:
//
// len(mockedInterface.FindCalls())
func (mock *InterfaceMock) FindCalls() []struct {
Req FindRequest
} {
var calls []struct {
Req FindRequest
}
mock.lockFind.RLock()
calls = mock.calls.Find
mock.lockFind.RUnlock()
return calls
}
return r0, r1
// Flag calls FlagFunc.
func (mock *InterfaceMock) Flag(req FlagRequest) (bool, error) {
if mock.FlagFunc == nil {
panic("InterfaceMock.FlagFunc: method is nil but Interface.Flag was just called")
}
callInfo := struct {
Req FlagRequest
}{
Req: req,
}
mock.lockFlag.Lock()
mock.calls.Flag = append(mock.calls.Flag, callInfo)
mock.lockFlag.Unlock()
return mock.FlagFunc(req)
}
// FlagCalls gets all the calls that were made to Flag.
// Check the length with:
//
// len(mockedInterface.FlagCalls())
func (mock *InterfaceMock) FlagCalls() []struct {
Req FlagRequest
} {
var calls []struct {
Req FlagRequest
}
mock.lockFlag.RLock()
calls = mock.calls.Flag
mock.lockFlag.RUnlock()
return calls
}
// Get calls GetFunc.
func (mock *InterfaceMock) Get(req GetRequest) (store.Comment, error) {
if mock.GetFunc == nil {
panic("InterfaceMock.GetFunc: method is nil but Interface.Get was just called")
}
callInfo := struct {
Req GetRequest
}{
Req: req,
}
mock.lockGet.Lock()
mock.calls.Get = append(mock.calls.Get, callInfo)
mock.lockGet.Unlock()
return mock.GetFunc(req)
}
// GetCalls gets all the calls that were made to Get.
// Check the length with:
//
// len(mockedInterface.GetCalls())
func (mock *InterfaceMock) GetCalls() []struct {
Req GetRequest
} {
var calls []struct {
Req GetRequest
}
mock.lockGet.RLock()
calls = mock.calls.Get
mock.lockGet.RUnlock()
return calls
}
// Info calls InfoFunc.
func (mock *InterfaceMock) Info(req InfoRequest) ([]store.PostInfo, error) {
if mock.InfoFunc == nil {
panic("InterfaceMock.InfoFunc: method is nil but Interface.Info was just called")
}
callInfo := struct {
Req InfoRequest
}{
Req: req,
}
mock.lockInfo.Lock()
mock.calls.Info = append(mock.calls.Info, callInfo)
mock.lockInfo.Unlock()
return mock.InfoFunc(req)
}
// InfoCalls gets all the calls that were made to Info.
// Check the length with:
//
// len(mockedInterface.InfoCalls())
func (mock *InterfaceMock) InfoCalls() []struct {
Req InfoRequest
} {
var calls []struct {
Req InfoRequest
}
mock.lockInfo.RLock()
calls = mock.calls.Info
mock.lockInfo.RUnlock()
return calls
}
// ListFlags calls ListFlagsFunc.
func (mock *InterfaceMock) ListFlags(req FlagRequest) ([]interface{}, error) {
if mock.ListFlagsFunc == nil {
panic("InterfaceMock.ListFlagsFunc: method is nil but Interface.ListFlags was just called")
}
callInfo := struct {
Req FlagRequest
}{
Req: req,
}
mock.lockListFlags.Lock()
mock.calls.ListFlags = append(mock.calls.ListFlags, callInfo)
mock.lockListFlags.Unlock()
return mock.ListFlagsFunc(req)
}
// ListFlagsCalls gets all the calls that were made to ListFlags.
// Check the length with:
//
// len(mockedInterface.ListFlagsCalls())
func (mock *InterfaceMock) ListFlagsCalls() []struct {
Req FlagRequest
} {
var calls []struct {
Req FlagRequest
}
mock.lockListFlags.RLock()
calls = mock.calls.ListFlags
mock.lockListFlags.RUnlock()
return calls
}
// Update calls UpdateFunc.
func (mock *InterfaceMock) Update(comment store.Comment) error {
if mock.UpdateFunc == nil {
panic("InterfaceMock.UpdateFunc: method is nil but Interface.Update was just called")
}
callInfo := struct {
Comment store.Comment
}{
Comment: comment,
}
mock.lockUpdate.Lock()
mock.calls.Update = append(mock.calls.Update, callInfo)
mock.lockUpdate.Unlock()
return mock.UpdateFunc(comment)
}
// UpdateCalls gets all the calls that were made to Update.
// Check the length with:
//
// len(mockedInterface.UpdateCalls())
func (mock *InterfaceMock) UpdateCalls() []struct {
Comment store.Comment
} {
var calls []struct {
Comment store.Comment
}
mock.lockUpdate.RLock()
calls = mock.calls.Update
mock.lockUpdate.RUnlock()
return calls
}
// UserDetail calls UserDetailFunc.
func (mock *InterfaceMock) UserDetail(req UserDetailRequest) ([]UserDetailEntry, error) {
if mock.UserDetailFunc == nil {
panic("InterfaceMock.UserDetailFunc: method is nil but Interface.UserDetail was just called")
}
callInfo := struct {
Req UserDetailRequest
}{
Req: req,
}
mock.lockUserDetail.Lock()
mock.calls.UserDetail = append(mock.calls.UserDetail, callInfo)
mock.lockUserDetail.Unlock()
return mock.UserDetailFunc(req)
}
// UserDetailCalls gets all the calls that were made to UserDetail.
// Check the length with:
//
// len(mockedInterface.UserDetailCalls())
func (mock *InterfaceMock) UserDetailCalls() []struct {
Req UserDetailRequest
} {
var calls []struct {
Req UserDetailRequest
}
mock.lockUserDetail.RLock()
calls = mock.calls.UserDetail
mock.lockUserDetail.RUnlock()
return calls
}
+34 -4
View File
@@ -15,7 +15,6 @@ type RPC struct {
// Create comment and return ID
func (r *RPC) Create(comment store.Comment) (commentID string, err error) {
resp, err := r.Call("store.create", comment)
if err != nil {
return "", err
@@ -72,14 +71,44 @@ func (r *RPC) Flag(req FlagRequest) (status bool, err error) {
return status, err
}
func unmarshalString(data []byte) ([]interface{}, error) {
var strings []string
if err := json.Unmarshal(data, &strings); err != nil {
return nil, err
}
list := make([]interface{}, 0, len(strings))
for _, w := range strings {
list = append(list, w)
}
return list, nil
}
func unmarshalBlockedUser(data []byte) ([]interface{}, error) {
var blockedUsers []store.BlockedUser
if err := json.Unmarshal(data, &blockedUsers); err != nil {
return nil, err
}
list := make([]interface{}, 0, len(blockedUsers))
for _, w := range blockedUsers {
list = append(list, w)
}
return list, nil
}
// ListFlags get list of flagged keys, like blocked & verified user
func (r *RPC) ListFlags(req FlagRequest) (list []interface{}, err error) {
func (r *RPC) ListFlags(req FlagRequest) ([]interface{}, error) {
resp, err := r.Call("store.list_flags", req)
if err != nil {
return nil, err
}
err = json.Unmarshal(*resp.Result, &list)
return list, err
// try []store.BlockedUser
list, err := unmarshalBlockedUser(*resp.Result)
if err == nil {
return list, nil
}
// try []strings
return unmarshalString(*resp.Result)
}
// UserDetail sets or gets single detail value, or gets all details for requested site.
@@ -113,5 +142,6 @@ func (r *RPC) Delete(req DeleteRequest) error {
// Close storage engine
func (r *RPC) Close() error {
_, err := r.Call("store.close")
r.Client.Client.CloseIdleConnections()
return err
}
+16 -5
View File
@@ -95,7 +95,6 @@ func TestRemote_Update(t *testing.T) {
err := c.Update(store.Comment{ID: "123", Locator: store.Locator{URL: "http://example.com/url", SiteID: "site123"},
Text: "msg"})
assert.NoError(t, err)
}
func TestRemote_Find(t *testing.T) {
@@ -135,7 +134,13 @@ func TestRemote_ListFlag(t *testing.T) {
c := RPC{Client: jrpc.Client{API: ts.URL, Client: http.Client{}}}
res, err := c.ListFlags(FlagRequest{Locator: store.Locator{SiteID: "site_id"}, Flag: Blocked})
assert.NoError(t, err)
assert.Equal(t, []interface{}{map[string]interface{}{"ID": "id1"}, map[string]interface{}{"ID": "id2"}}, res)
require.Equal(t, 2, len(res))
blockedUsers := make([]store.BlockedUser, 0, len(res))
for _, v := range res {
blockedUsers = append(blockedUsers, v.(store.BlockedUser))
}
assert.Equal(t, 2, len(blockedUsers))
assert.Equal(t, []store.BlockedUser{{ID: "id1"}, {ID: "id2"}}, blockedUsers)
}
func TestRemote_UserDetail(t *testing.T) {
@@ -181,11 +186,17 @@ func TestRemote_Delete(t *testing.T) {
}
func TestRemote_Close(t *testing.T) {
ts := testServer(t, `{"method":"store.close","id":1}`, `{}`)
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
require.NoError(t, err)
assert.Contains(t, string(body), "{\"method\":\"store.close\",\"id\":")
t.Logf("req: %s", string(body))
_, _ = fmt.Fprint(w, "{}")
}))
defer ts.Close()
c := RPC{Client: jrpc.Client{API: ts.URL, Client: http.Client{}}}
err := c.Close()
assert.NoError(t, err)
assert.NoError(t, c.Close())
assert.NoError(t, c.Close(), "second call should not result in panic or errors")
}
func testServer(t *testing.T, req, resp string) *httptest.Server {
+21 -14
View File
@@ -4,9 +4,9 @@ import (
"net/url"
"strings"
"github.com/Depado/bfchroma"
"github.com/Depado/bfchroma/v2"
"github.com/PuerkitoBio/goquery"
"github.com/alecthomas/chroma/formatters/html"
"github.com/alecthomas/chroma/v2/formatters/html"
bf "github.com/russross/blackfriday/v2"
)
@@ -42,17 +42,8 @@ func (f *CommentFormatter) Format(c Comment) Comment {
// FormatText converts text with markdown processor, applies external converters and shortens links
func (f *CommentFormatter) FormatText(txt string) (res string) {
mdExt := bf.NoIntraEmphasis | bf.Tables | bf.FencedCode |
bf.Strikethrough | bf.SpaceHeadings | bf.HardLineBreak |
bf.BackslashLineBreak | bf.Autolink
rend := bf.NewHTMLRenderer(bf.HTMLRendererParameters{
Flags: bf.Smartypants | bf.SmartypantsFractions | bf.SmartypantsDashes | bf.SmartypantsAngledQuotes,
})
extRend := bfchroma.NewRenderer(bfchroma.Extend(rend), bfchroma.ChromaOptions(html.WithClasses(true)))
res = string(bf.Run([]byte(txt), bf.WithExtensions(mdExt), bf.WithRenderer(extRend)))
mdExt, rend := GetMdExtensionsAndRenderer()
res = string(bf.Run([]byte(txt), bf.WithExtensions(mdExt), bf.WithRenderer(rend)))
res = f.unEscape(res)
for _, conv := range f.converters {
@@ -83,7 +74,8 @@ func (f *CommentFormatter) shortenAutoLinks(commentHTML string, max int) (resHTM
if host == "" {
return
}
short := href[:max-3]
short := string([]rune(href)[:max-3])
if len(short) < len(host) {
short = host
}
@@ -125,3 +117,18 @@ func (f *CommentFormatter) lazyImage(commentHTML string) (resHTML string) {
}
return resHTML
}
// GetMdExtensionsAndRenderer returns blackfriday extensions and renderer used for rendering markdown
// within store module.
func GetMdExtensionsAndRenderer() (bf.Extensions, *bfchroma.Renderer) {
mdExt := bf.NoIntraEmphasis | bf.Tables | bf.FencedCode |
bf.Strikethrough | bf.SpaceHeadings | bf.HardLineBreak |
bf.BackslashLineBreak | bf.Autolink
rend := bf.NewHTMLRenderer(bf.HTMLRendererParameters{
Flags: bf.Smartypants | bf.SmartypantsFractions | bf.SmartypantsDashes | bf.SmartypantsAngledQuotes,
})
extRend := bfchroma.NewRenderer(bfchroma.Extend(rend), bfchroma.ChromaOptions(html.WithClasses(true)))
return mdExt, extRend
}
+8 -6
View File
@@ -25,6 +25,10 @@ func TestFormatter_FormatText(t *testing.T) {
"<p><a href=\"http://127.0.0.1/some-long-link/12345/678901234567890\">http://127.0.0." +
"1/some-long-link/12345/6789012...</a></p>\n!converted", "links",
},
{
"something https://amp.dw.com/ru/илон-маск-купил-twitter/a-61590911",
"<p>something <a href=\"https://amp.dw.com/ru/илон-маск-купил-twitter/a-61590911\">https://amp.dw.com/ru/илон-маск-купил-twitter...</a></p>\n!converted", "links",
},
{
"something <img src=\"some.png\"/> _aaa_",
"<p>something <img src=\"some.png\" loading=\"lazy\"/> <em>aaa</em></p>\n!converted",
@@ -32,10 +36,10 @@ func TestFormatter_FormatText(t *testing.T) {
},
{"&mdash; not translated #354", "<p>— not translated #354</p>\n!converted", "mdash"},
{"smth\n```go\nfunc main(aa string) int {return 0}\n```", `<p>smth</p>
<pre tabindex="0" class="chroma"><span class="kd">func</span> <span class="nf">main</span><span class="p">(</span><span class="nx">aa</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">int</span> <span class="p">{</span><span class="k">return</span> <span class="mi">0</span><span class="p">}</span>
</pre>!converted`, "code with language"},
{"```\ntest_code\n```", `<pre tabindex="0" class="chroma">test_code
</pre>!converted`, "code without language"},
<pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">(</span><span class="nx">aa</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">int</span> <span class="p">{</span><span class="k">return</span> <span class="mi">0</span><span class="p">}</span>
</span></span></code></pre>!converted`, "code with language"},
{"```\ntest_code\n```", `<pre class="chroma"><code><span class="line"><span class="cl">test_code
</span></span></code></pre>!converted`, "code without language"},
}
f := NewCommentFormatter(mockConverter{})
for _, tt := range tbl {
@@ -122,7 +126,6 @@ func TestFormatter_ShortenAutoLinks(t *testing.T) {
}
func TestCommentFormatter_lazyImage(t *testing.T) {
tbl := []struct {
inp, out string
}{
@@ -139,5 +142,4 @@ func TestCommentFormatter_lazyImage(t *testing.T) {
assert.Equal(t, tt.out, f.lazyImage(tt.inp))
})
}
}

Some files were not shown because too many files have changed in this diff Show More