fix typo in file name

* site: fetch latest version client-side instead of embedding at build time

The header version badge was filled in by site/src/data/github.js calling
the GitHub releases API at Eleventy build time and baking data[0].tag_name
into every page. This had three failure modes:

1. Layered cache: Buildx caches the yarn build layer; on a release-triggered
   workflow nothing under ./site changes, so the cached HTML (with the
   previous tag baked in) gets shipped. v1.16.0 went out and remark42.com
   kept showing v1.15.0 until a separate site/ commit landed and naturally
   invalidated the COPY layer.

2. Tag mismatch: the deploy pulls ghcr.io/umputun/remark42-site:master,
   but release events build :v1.16.0 and :latest only. A cache-skip
   workflow tweak wouldn't even reach the served image.

3. API propagation race: the workflow fires ~2s after release publish,
   so even with cache disabled the API might still return the previous
   tag from a stale read replica.

All three vanish if the version is fetched in the browser. GitHub serves
the /releases/latest response with Cache-Control: public, max-age=60 so
per-visitor cost is bounded; failures fall through silently and the
badge stays empty rather than wrong.

Changes:
- header.njk: replace {{ github.latestVersion }} with a
  <span data-remark42-version></span> placeholder.
- inline.js: add a fetch of /releases/latest that fills any
  [data-remark42-version] element on the page. fallback is no-op on any
  network/parse failure.
- delete site/src/data/github.js (Eleventy data file is no longer used).
- drop node-fetch from devDependencies (was used only by github.js).

* site: address PR review on version badge fetch

- gate DOM update on DOMContentLoaded — inline.js is loaded sync in <head>,
  so a cache-hit fetch can resolve before the placeholder span is parsed.
- hide placeholder span by default (`hidden`) so a failed/blocked fetch
  doesn't leave a 0.5rem stray gap before the github icon.
- log fetch failures (rate limit, offline, blocked) instead of silently
  swallowing — matches the prior behaviour of build-time github.js.

* site: cache latest version in sessionStorage with 1h TTL

avoids hitting the GitHub API on every page load — repeated navigations
within a tab read from sessionStorage instead. TTL caps stale display at
1h for very long-lived tabs. cleared on tab close, so each new session
fetches once and reuses the result throughout.

* site: address PR review on header & version fetch

Copilot review on the cache commit raised three points:

1. inline.js had a hard-coded `https://api.github.com/repos/umputun/remark42`
   while the templates use `site.githubUrl`. Rename inline.js → inline.njk
   so nunjucks evaluates it, add `githubApiUrl` to site.json, and template
   the fetch URL from it. One place to update if the repo ever moves.

2. header.njk aria-label said "Remark42's GitHub Repository" but the link
   target is `/releases`. Change to "{{ site.name }} releases on GitHub"
   so screen readers describe the actual destination.

3. console.warn on fetch failure (kept after umputun's prior review noted
   the trade-off): addressed in the PR description, no code change.

* site: actually template fetch URL via site.githubApiUrl

Copilot's second pass caught that 7697dcf3 added site.githubApiUrl,
renamed inline.js → inline.njk, and pointed head.njk at the .njk file
— but the fetch() call itself was never changed to use the template
variable. Build output looked correct because the literal hard-coded
URL happened to match what {{ site.githubApiUrl }} would expand to.

* site: normalise Nunjucks spacing in header.njk

{{ site.githubUrl}}/releases → {{ site.githubUrl }}/releases. Cosmetic
only; matches the spacing used everywhere else in the templates.

.dockerignore

@@ -1,15 +1,30 @@
 /logs/
 /target/
 /var/
-/frontend/node_modules/
-/frontend/public/
 /.vscode/
 /.idea/
 /bin/
+/.git/
+
+# frontend files not needed in docker image
+/frontend/node_modules/
+/frontend/apps/remark42/node_modules/
+/frontend/apps/remark42/public/
+# e2e tests arficats
+/frontend/e2e/playwright-report/
+/frontend/e2e/playwright/.cache/
+/frontend/e2e/test-results/
 
 # source files
 docker-compose.yml
+compose-dev-backend.yml
+compose-dev-frontend.yml
+compose-private-backend.yml
+compose-private-frontend.yml
+compose-e2e-test.yml
+compose-private.yml
 rest-client.env.json
+Makefile
 
 # generated files
 *.cov
@@ -21,4 +36,4 @@ debug.test
 *.test
 remark42
 /backend/var/
-compose-private-backend.yml
+/playwright-report/

.editorconfig

@@ -1,3 +1,5 @@
+root = true
+
 [*]
 indent_style = tab
 insert_final_newline = true
@@ -6,6 +8,6 @@ insert_final_newline = true
 indent_style = space
 trim_trailing_whitespace = false
 
-[*.yml]
+[*.{yml,json}]
 indent_size = 2
 indent_style = space

.github/CODEOWNERS

@@ -2,4 +2,5 @@
 # Unless a later match takes precedence, @umputun will be requested for
 # review when someone opens a pull request.
 
-*       @umputun
+*             @umputun
+frontend/*    @akellbl4 @Mavrin

.github/dependabot.yml

@@ -0,0 +1,72 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
+
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+    groups:
+      "GitHub Actions updates":
+        patterns:
+          - "*"
+  - package-ecosystem: "gomod"
+    directory: "/backend"
+    schedule:
+      interval: "monthly"
+    groups:
+      "Go modules updates":
+        dependency-type: "production"
+  - package-ecosystem: "npm"
+    directory: "/frontend"
+    open-pull-requests-limit: 0
+    schedule:
+      interval: "monthly"
+    groups:
+      "NPM modules updates":
+        dependency-type: "production"
+      "NPM modules updates for tests":
+        dependency-type: "development"
+  - package-ecosystem: "npm"
+    directory: "/frontend/packages/api"
+    open-pull-requests-limit: 0
+    schedule:
+      interval: "monthly"
+    groups:
+      "NPM modules updates":
+        dependency-type: "production"
+      "NPM modules updates for tests":
+        dependency-type: "development"
+  - package-ecosystem: "npm"
+    directory: "/frontend/e2e"
+    open-pull-requests-limit: 0
+    schedule:
+      interval: "monthly"
+    groups:
+      "NPM modules updates":
+        dependency-type: "production"
+      "NPM modules updates for tests":
+        dependency-type: "development"
+  - package-ecosystem: "npm"
+    directory: "/frontend/apps/remark42"
+    open-pull-requests-limit: 0
+    schedule:
+      interval: "monthly"
+    groups:
+      "NPM modules updates":
+        dependency-type: "production"
+      "NPM modules updates for tests":
+        dependency-type: "development"
+  - package-ecosystem: "npm"
+    directory: "/site"
+    open-pull-requests-limit: 0
+    schedule:
+      interval: "monthly"
+    groups:
+      "NPM modules updates":
+        dependency-type: "production"
+      "NPM modules updates for tests":
+        dependency-type: "development"

.github/workflows/ci-backend.yml

@@ -0,0 +1,79 @@
+name: backend
+
+on:
+  push:
+    branches:
+    tags:
+    paths:
+      - ".github/workflows/ci-backend.yml"
+      - "backend/**"
+      - "!backend/scripts/**"
+      - "!**.md"
+  pull_request:
+    types: [opened, reopened]
+    paths:
+      - ".github/workflows/ci-backend.yml"
+      - "backend/**"
+      - "!backend/scripts/**"
+      - "!**.md"
+
+jobs:
+  test:
+    name: Test & Coverage
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: debug if needed
+        run: if [[ "$DEBUG" == "true" ]]; then env; fi
+        env:
+          DEBUG: ${{secrets.DEBUG}}
+
+      - name: install go
+        uses: actions/setup-go@v6
+        with:
+          go-version: "1.25"
+          cache-dependency-path: backend
+
+      - name: test and build backend
+        run: |
+          go test -race -timeout=60s -covermode=atomic -coverprofile=$GITHUB_WORKSPACE/profile.cov_tmp ./...
+          cat $GITHUB_WORKSPACE/profile.cov_tmp | grep -v "_mock.go" > $GITHUB_WORKSPACE/profile.cov
+          go build -race ./...
+        working-directory: backend/app
+        env:
+          TZ: "America/Chicago"
+
+      - name: test examples
+        run: |
+          go test -race ./...
+          go build -race ./...
+        working-directory: backend/_example/memory_store
+        env:
+          TZ: "America/Chicago"
+
+      - name: golangci-lint
+        uses: golangci/golangci-lint-action@v9
+        with:
+          version: "v2.10.1"
+          working-directory: backend/app
+
+      - name: golangci-lint on example directory
+        uses: golangci/golangci-lint-action@v9
+        with:
+          version: "v2.10.1"
+          args: --config ../../.golangci.yml
+          working-directory: backend/_example/memory_store
+
+      - name: submit coverage
+        run: |
+          go install github.com/mattn/goveralls@latest
+          goveralls -service="github" -coverprofile=$GITHUB_WORKSPACE/profile.cov
+        working-directory: backend
+        env:
+          COVERALLS_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/ci-build.yml

@@ -1,85 +1,66 @@
 name: build
 
 on:
-  push:
-    branches:
-    tags:
-    paths:
-      - ".github/workflows/ci-build.yml"
-      - "backend/**"
-      - "frontend/**"
-      - ".dockerignore"
-      - "docker-init.sh"
-      - "Dockerfile"
-      - "!**.md"
   pull_request:
     paths:
       - ".github/workflows/ci-build.yml"
       - "backend/**"
-      - "frontend/**"
+      - "frontend/apps/**"
       - ".dockerignore"
       - "docker-init.sh"
       - "Dockerfile"
       - "!**.md"
 
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   build-images:
+    name: Validate Docker build
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
 
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v6
-
+        with:
-      - name: set up QEMU
+          persist-credentials: false
-        uses: docker/setup-qemu-action@v1
 
       - name: set up Docker Buildx
-        id: buildx
+        uses: docker/setup-buildx-action@v4
-        uses: docker/setup-buildx-action@v1
 
-      - name: available platforms
+      - name: expose GitHub Actions cache
-        run: echo ${{ steps.buildx.outputs.platforms }}
+        uses: actions/cache@v5
+        with:
+          path: /tmp/.buildx-cache
+          key: ${{ runner.os }}-buildx-${{ github.sha }}
+          restore-keys: |
+            ${{ runner.os }}-buildx-
 
-      - name: build and deploy master image to ghcr.io and dockerhub
+      - name: free disk space
-        if: ${{ github.ref == 'refs/heads/master' }}
-        env:
-          GITHUB_PACKAGE_TOKEN: ${{ secrets.PKG_TOKEN }}
-          DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}
-          USERNAME: ${{ github.actor }}
-          GITHUB_SHA: ${{ github.sha}}
-          GITHUB_REF: ${{ github.ref}}
         run: |
-          ref="$(echo ${GITHUB_REF} | cut -d'/' -f3)"
+          sudo rm -rf /usr/share/dotnet
-          echo GITHUB_REF - $ref
+          sudo rm -rf /opt/ghc
-          echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
+          sudo rm -rf /usr/local/share/boost
-          echo ${DOCKER_HUB_TOKEN} | docker login -u ${USERNAME} --password-stdin
+          docker system prune -af
-          docker buildx build --push \
-              --build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true --build-arg CI=github \
-              --build-arg GITHUB_SHA=${GITHUB_SHA} --build-arg GIT_BRANCH=${ref} \
-              --platform linux/amd64,linux/arm/v7,linux/arm64 \
-              -t ghcr.io/${USERNAME}/remark42:${ref} -t ${USERNAME}/remark42:${ref} .
 
-      - name: deploy tagged (latest) to ghcr.io and dockerhub
+      - name: build docker image without pushing
-        if: ${{ startsWith(github.ref, 'refs/tags/') }}
-        env:
-          GITHUB_PACKAGE_TOKEN: ${{ secrets.PKG_TOKEN }}
-          DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}
-          USERNAME: ${{ github.actor }}
-          GITHUB_SHA: ${{ github.sha}}
-          GITHUB_REF: ${{ github.ref}}
         run: |
-          ref="$(echo ${GITHUB_REF} | cut -d'/' -f3)"
+          docker buildx build --load \
-          echo "GITHUB_REF=$ref, GITHUB_SHA=${GITHUB_SHA}"
+              --cache-from type=local,src=/tmp/.buildx-cache \
-          echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
+              --cache-to type=local,dest=/tmp/.buildx-cache-new,mode=max \
-          echo ${DOCKER_HUB_TOKEN} | docker login -u ${USERNAME} --password-stdin
+              --build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true \
-          docker buildx build --push \
+              --platform linux/amd64 .
-              --build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true --build-arg CI=github \
-              --build-arg GITHUB_SHA=${GITHUB_SHA} --build-arg GIT_BRANCH=${ref} \
-              --platform linux/amd64,linux/arm/v7,linux/arm64 \
-              -t ghcr.io/umputun/remark42:${ref} -t ghcr.io/umputun/remark42:latest \
-              -t umputun/remark42:${ref} -t umputun/remark42:latest .
 
-      - name: remote deployment to remark42.com from master
+      - name: build example docker image without pushing
-        if: ${{ github.ref == 'refs/heads/master' }}
+        run: |
-        env:
+          docker buildx build --load \
-          UPDATER_KEY: ${{ secrets.UPDATER_KEY }}
+              --cache-from type=local,src=/tmp/.buildx-cache \
-        run: curl https://jess.umputun.com/update/remark42-core/${UPDATER_KEY}
+              --build-arg SKIP_BACKEND_TEST=true --build-arg SKIP_FRONTEND_TEST=true \
+              --platform linux/amd64 -f backend/_example/memory_store/Dockerfile .
+
+      - name: rotate cache
+        run: |
+          rm -rf /tmp/.buildx-cache
+          mv /tmp/.buildx-cache-new /tmp/.buildx-cache || true

.github/workflows/ci-frontend-api.yml

@@ -0,0 +1,131 @@
+name: "@remark42/api"
+
+on:
+  push:
+    branches:
+      - master
+    paths:
+      - ".github/workflows/ci-frontend-api.yml"
+      - "frontend/packages/**"
+      - "!**.md"
+  pull_request:
+    paths:
+      - ".github/workflows/ci-frontend-api.yml"
+      - "frontend/packages/**"
+      - "!**.md"
+
+jobs:
+  type-check:
+    name: Type check
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    strategy:
+      matrix:
+        node: [ 16 ]
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
+        with:
+          node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
+
+      - name: Install dependencies
+        run: pnpm i
+        working-directory: ./frontend
+
+      - name: Run type check
+        run: pnpm type-check:api
+        working-directory: ./frontend
+
+  lint:
+    name: Lint
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    strategy:
+      matrix:
+        node: [ 16 ]
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
+        with:
+          node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
+
+      - name: Install dependencies
+        run: pnpm i
+        working-directory: ./frontend
+
+      - name: Run linters
+        run: pnpm lint:api
+        working-directory: ./frontend/
+
+  test:
+    name: Tests & Coverage
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    strategy:
+      matrix:
+        node: [ 16 ]
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
+        with:
+          node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
+
+      - name: Install dependencies
+        run: pnpm i
+        working-directory: ./frontend
+
+      - name: Test & Coverage
+        run: pnpm coverage:api
+        working-directory: ./frontend
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v6
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          working-directory: ./frontend
+          codecov_yml_path: ./frontend/apps/remark42/codecov.yml

.github/workflows/ci-frontend-size-limit.yml

@@ -1,20 +0,0 @@
-name: frontend
-
-on:
-  pull_request:
-    paths:
-      - ".github/workflows/ci-frontend-size-limit.yml"
-      - "frontend/**"
-      - "!**.md"
-
-jobs:
-  size:
-    runs-on: ubuntu-latest
-    env:
-      CI_JOB_NUMBER: 1
-    steps:
-      - uses: actions/checkout@v1
-      - uses: andresz1/size-limit-action@v1
-        with:
-          github_token: ${{ secrets.GITHUB_TOKEN }}
-          directory: frontend

.github/workflows/ci-frontend.yml

@@ -3,105 +3,193 @@ name: frontend
 on:
   push:
     branches:
-    tags:
+      - master
     paths:
       - ".github/workflows/ci-frontend.yml"
-      - "frontend/**"
+      - "frontend/apps/remark42/**"
       - "!**.md"
   pull_request:
     paths:
       - ".github/workflows/ci-frontend.yml"
-      - "frontend/**"
+      - "frontend/apps/remark42/**"
       - "!**.md"
 
 jobs:
-  check-translations:
+  translations-check:
+    name: Translations check
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     strategy:
       matrix:
-        node: [14.15]
+        node: [16]
 
     steps:
-      - uses: actions/checkout@v2
+      - name: Checkout
-      - uses: actions/setup-node@v1
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
         with:
           node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
 
-      - run: npm ci --loglevel warn
+      - name: Install dependencies
+        run: pnpm i
         working-directory: ./frontend
 
-      - uses: actions/cache@v2
+      - name: Translations check
-        with:
+        run: pnpm translation-check
-          path: ${{ github.workspace }}/frontend/node_modules/.cache
+        working-directory: ./frontend/apps/remark42
-          key: ${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
-          restore-keys: |
-            ${{ runner.os }}-node-
 
-      - run: npm run check:translation
+  type-check:
-        working-directory: ./frontend
+    name: Type check
-
-  check-typescript:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     strategy:
       matrix:
-        node: [14.15]
+        node: [16]
 
     steps:
-      - uses: actions/checkout@v2
+      - name: Checkout
-      - uses: actions/setup-node@v1
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
         with:
           node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
 
-      - run: npm ci --loglevel warn
+      - name: Install dependencies
+        run: pnpm i
         working-directory: ./frontend
 
-      - uses: actions/cache@v2
+      - name: Run type check
-        with:
+        run: pnpm type-check
-          path: ${{ github.workspace }}/frontend/node_modules/.cache
+        working-directory: ./frontend/apps/remark42
-          key: ${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
-          restore-keys: |
-            ${{ runner.os }}-node-
-
-      - run: npm run check:types
-        working-directory: ./frontend
 
   lint:
+    name: Eslint & Stylelint
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     strategy:
       matrix:
-        node: [14.15]
+        node: [16]
 
     steps:
-      - uses: actions/checkout@v2
+      - name: Checkout
-      - uses: actions/setup-node@v1
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
         with:
           node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
 
-      - run: npm ci --loglevel warn
+      - name: Install dependencies
+        run: pnpm i
         working-directory: ./frontend
 
-      - run: npx run-p lint
+      - name: Run linters
-        working-directory: ./frontend
+        run: pnpm lint
+        working-directory: ./frontend/apps/remark42
+
+  size-limit:
+    name: Size limit
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request'
+    permissions:
+      contents: read
+      pull-requests: write
+    env:
+      CI_JOB_NUMBER: 1
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Check bundle size
+        uses: andresz1/size-limit-action@94bc357df29c36c8f8d50ea497c3e225c3c95d1d
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          directory: ./frontend/apps/remark42
+          package_manager: pnpm
 
   test:
+    name: Tests & Coverage
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     strategy:
       matrix:
-        node: [14.15]
+        node: [16]
 
     steps:
-      - uses: actions/checkout@v2
+      - name: Checkout
-      - uses: actions/setup-node@v1
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: Install node
+        uses: actions/setup-node@v6
         with:
           node-version: ${{ matrix.node }}
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
 
-      - run: npm ci --loglevel warn
+      - name: Install dependencies
+        run: pnpm i
         working-directory: ./frontend
 
-      - run: npm run test:coverage
+      - name: Test & Coverage
-        working-directory: ./frontend
+        run: pnpm coverage
+        working-directory: ./frontend/apps/remark42
 
-      - name: submit coverage
+      - name: Upload coverage to Codecov
-        run: node ${{ github.workspace }}/frontend/node_modules/.bin/codecov
+        uses: codecov/codecov-action@v6
-        env:
+        with:
-          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+          token: ${{ secrets.CODECOV_TOKEN }}
+          working-directory: ./frontend/apps/remark42
+          codecov_yml_path: ./frontend/apps/remark42/codecov.yml

.github/workflows/ci-site.yml

@@ -1,9 +1,11 @@
 name: site
 
 on:
+  release:
+    types: [published]
   push:
     branches:
-    tags:
+      - master
     paths:
       - ".github/workflows/ci-site.yml"
       - "site/**"
@@ -12,57 +14,131 @@ on:
       - ".github/workflows/ci-site.yml"
       - "site/**"
 
-jobs:
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
 
-  build-site:
+jobs:
-    runs-on: ubuntu-latest
+  build:
+    name: Build site image (${{ matrix.platform }})
+    if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')
+    runs-on: ${{ matrix.runner }}
+    permissions:
+      contents: read
+      packages: write
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - platform: linux/amd64
+            runner: ubuntu-latest
+            artifact: linux-amd64
+          - platform: linux/arm64
+            runner: ubuntu-24.04-arm
+            artifact: linux-arm64
 
     steps:
       - name: checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v6
-
+        with:
-      - name: set up QEMU
+          persist-credentials: false
-        uses: docker/setup-qemu-action@v1
 
       - name: set up Docker Buildx
-        id: buildx
+        uses: docker/setup-buildx-action@v4
-        uses: docker/setup-buildx-action@v1
 
-      - name: available platforms
+      - name: login to ghcr.io
-        run: echo ${{ steps.buildx.outputs.platforms }}
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.PKG_TOKEN }}
 
-      - name: build and deploy master image to ghcr.io and dockerhub
+      - name: build and push by digest
-        if: ${{ github.ref == 'refs/heads/master' }}
+        id: build
+        uses: docker/build-push-action@v7
+        with:
+          context: ./site
+          platforms: ${{ matrix.platform }}
+          cache-from: type=gha,scope=site-${{ matrix.platform }}
+          cache-to: type=gha,scope=site-${{ matrix.platform }},mode=max
+          outputs: type=image,name=ghcr.io/umputun/remark42-site,push-by-digest=true,name-canonical=true,push=true
+
+      - name: export digest
+        run: |
+          mkdir -p /tmp/digests
+          digest="${{ steps.build.outputs.digest }}"
+          touch "/tmp/digests/${digest#sha256:}"
+
+      - name: upload digest
+        uses: actions/upload-artifact@v7
+        with:
+          name: site-digests-${{ matrix.artifact }}
+          path: /tmp/digests/*
+          retention-days: 1
+
+  merge:
+    name: Create site multi-arch manifest
+    runs-on: ubuntu-latest
+    needs: build
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: download digests
+        uses: actions/download-artifact@v8
+        with:
+          path: /tmp/digests
+          pattern: site-digests-*
+          merge-multiple: true
+
+      - name: verify all digests present
+        run: |
+          expected=2
+          actual=$(find /tmp/digests -maxdepth 1 -type f | wc -l)
+          if [ "$actual" -ne "$expected" ]; then
+            echo "Expected $expected digests, found $actual"
+            ls -la /tmp/digests
+            exit 1
+          fi
+          echo "All $expected digests present"
+
+      - name: set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+
+      - name: login to ghcr.io
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.PKG_TOKEN }}
+
+      - name: create manifest and push
+        working-directory: /tmp/digests
         env:
-          GITHUB_PACKAGE_TOKEN: ${{ secrets.PKG_TOKEN }}
+          GITHUB_REF: ${{ github.ref }}
-          USERNAME: ${{ github.actor }}
-          GITHUB_SHA: ${{ github.sha}}
-          GITHUB_REF: ${{ github.ref}}
-        working-directory: ./site
         run: |
           ref="$(echo ${GITHUB_REF} | cut -d'/' -f3)"
-          echo GITHUB_REF - $ref
+          if [[ "$GITHUB_REF" == refs/tags/* ]]; then
-          echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
+            docker buildx imagetools create \
-          docker buildx build --push --no-cache --platform linux/amd64,linux/arm/v7,linux/arm64 \
+              -t ghcr.io/umputun/remark42-site:${ref} \
-            -t ghcr.io/${USERNAME}/remark24-site:${ref} .
+              -t ghcr.io/umputun/remark42-site:latest \
+              $(printf 'ghcr.io/umputun/remark42-site@sha256:%s ' *)
+          else
+            docker buildx imagetools create \
+              -t ghcr.io/umputun/remark42-site:${ref} \
+              $(printf 'ghcr.io/umputun/remark42-site@sha256:%s ' *)
+          fi
 
-      - name: deploy tagged (latest) to ghcr.io and dockerhub
+  deploy:
-        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+    name: Deploy site
-        env:
+    runs-on: ubuntu-latest
-          GITHUB_PACKAGE_TOKEN: ${{ secrets.PKG_TOKEN }}
+    needs: merge
-          USERNAME: ${{ github.actor }}
+    if: github.ref == 'refs/heads/master' || github.event_name == 'release'
-          GITHUB_SHA: ${{ github.sha}}
+    permissions: {} # only calls an external URL via curl, no GitHub API access needed
-          GITHUB_REF: ${{ github.ref}}
-        working-directory: ./site
-        run: |
-          ref="$(echo ${GITHUB_REF} | cut -d'/' -f3)"
-          echo "GITHUB_REF=$ref, GITHUB_SHA=${GITHUB_SHA}"
-          echo ${GITHUB_PACKAGE_TOKEN} | docker login ghcr.io -u ${USERNAME} --password-stdin
-          docker buildx build --push --no-cache --platform linux/amd64,linux/arm/v7,linux/arm64 \
-            -t ghcr.io/${USERNAME}/remark24-site:${ref} -t ghcr.io/${USERNAME}/remark24-site:latest .
 
-      - name: remote site deployment from master
+    steps:
-        if: ${{ github.ref == 'refs/heads/master' }}
+      - name: trigger deployment
         env:
           UPDATER_KEY: ${{ secrets.UPDATER_KEY }}
-        run: curl https://jess.umputun.com/update/remark42-site/${UPDATER_KEY}
+        run: curl -sf https://jess.umputun.com/update/remark42-site/${UPDATER_KEY}

.github/workflows/ci-test-backend.yml

@@ -1,64 +0,0 @@
-name: test_backend
-
-on:
-  push:
-    branches:
-    tags:
-    paths:
-      - ".github/workflows/ci-test-backend.yml"
-      - "backend/**"
-      - "!backend/scripts/**"
-      - "!**.md"
-  pull_request:
-    paths:
-      - ".github/workflows/ci-test-backend.yml"
-      - "backend/**"
-      - "!backend/scripts/**"
-      - "!**.md"
-jobs:
-  test-backend:
-    runs-on: ubuntu-latest
-
-    steps:
-      - uses: actions/checkout@v2
-
-      - name: debug if needed
-        run: if [[ "$DEBUG" == "true" ]]; then env; fi
-        env:
-          DEBUG: ${{secrets.DEBUG}}
-
-      - name: install go
-        uses: actions/setup-go@v2
-        with:
-          go-version: 1.16
-
-      - name: install golangci-lint and goveralls
-        run: |
-          curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh| sh -s -- -b $GITHUB_WORKSPACE v1.43.0
-          go get -u github.com/mattn/goveralls
-
-      - name: test and lint backend
-        run: |
-          go test -race -timeout=60s -covermode=atomic -coverprofile=$GITHUB_WORKSPACE/profile.cov_tmp ./...
-          cat $GITHUB_WORKSPACE/profile.cov_tmp | grep -v "_mock.go" > $GITHUB_WORKSPACE/profile.cov
-          $GITHUB_WORKSPACE/golangci-lint --config ${GITHUB_WORKSPACE}/backend/.golangci.yml run --out-format=github-actions ./...
-        working-directory: backend/app
-        env:
-          GOFLAGS: "-mod=vendor"
-          TZ: "America/Chicago"
-
-      - name: test and lint examples
-        run: |
-          go version
-          $GITHUB_WORKSPACE/golangci-lint version
-          go test -race ./...
-          $GITHUB_WORKSPACE/golangci-lint --config ${GITHUB_WORKSPACE}/backend/.golangci.yml run --out-format=github-actions ./...
-        working-directory: backend/_example/memory_store
-        env:
-          TZ: "America/Chicago"
-
-      - name: submit coverage
-        run: $(go env GOPATH)/bin/goveralls -service="github" -coverprofile=$GITHUB_WORKSPACE/profile.cov
-        working-directory: backend
-        env:
-          COVERALLS_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/docker.yml

@@ -0,0 +1,215 @@
+name: docker
+
+on:
+  workflow_run:
+    workflows: [backend]
+    types: [completed]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    name: Build Docker image (${{ matrix.platform }})
+    if: >-
+      github.event.workflow_run.conclusion == 'success' &&
+      github.event.workflow_run.event != 'pull_request' &&
+      (github.event.workflow_run.head_branch == 'master' ||
+       startsWith(github.event.workflow_run.head_branch, 'v'))
+    permissions:
+      contents: read
+      packages: write
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - platform: linux/amd64
+            runner: ubuntu-latest
+            artifact: linux-amd64
+          - platform: linux/arm64
+            runner: ubuntu-24.04-arm
+            artifact: linux-arm64
+    runs-on: ${{ matrix.runner }}
+
+    steps:
+      - name: checkout
+        uses: actions/checkout@v6
+        with:
+          ref: ${{ github.event.workflow_run.head_sha }}
+          persist-credentials: false
+
+      - name: set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+
+      - name: login to ghcr.io
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.PKG_TOKEN }}
+
+      - name: login to DockerHub
+        uses: docker/login-action@v4
+        with:
+          username: umputun
+          password: ${{ secrets.DOCKER_HUB_TOKEN }}
+
+      - name: free disk space
+        run: |
+          sudo rm -rf /usr/share/dotnet
+          sudo rm -rf /opt/ghc
+          sudo rm -rf /usr/local/share/boost
+          docker system prune -af
+
+      - name: build and push to ghcr.io by digest
+        id: build-ghcr
+        uses: docker/build-push-action@v7
+        with:
+          context: .
+          platforms: ${{ matrix.platform }}
+          cache-from: type=gha,scope=${{ matrix.platform }}
+          cache-to: type=gha,scope=${{ matrix.platform }},mode=max
+          build-args: |
+            SKIP_BACKEND_TEST=true
+            SKIP_FRONTEND_TEST=true
+            CI=github
+            GITHUB_SHA=${{ github.event.workflow_run.head_sha }}
+            GIT_BRANCH=${{ github.event.workflow_run.head_branch }}
+            GITHUB_REF=refs/heads/${{ github.event.workflow_run.head_branch }}
+          outputs: type=image,name=ghcr.io/umputun/remark42,push-by-digest=true,name-canonical=true,push=true
+
+      - name: build and push to DockerHub by digest
+        id: build-dockerhub
+        uses: docker/build-push-action@v7
+        with:
+          context: .
+          platforms: ${{ matrix.platform }}
+          cache-from: type=gha,scope=${{ matrix.platform }}
+          build-args: |
+            SKIP_BACKEND_TEST=true
+            SKIP_FRONTEND_TEST=true
+            CI=github
+            GITHUB_SHA=${{ github.event.workflow_run.head_sha }}
+            GIT_BRANCH=${{ github.event.workflow_run.head_branch }}
+            GITHUB_REF=refs/heads/${{ github.event.workflow_run.head_branch }}
+          outputs: type=image,name=umputun/remark42,push-by-digest=true,name-canonical=true,push=true
+
+      - name: export digests
+        run: |
+          mkdir -p /tmp/digests/ghcr /tmp/digests/dockerhub
+          digest_ghcr="${{ steps.build-ghcr.outputs.digest }}"
+          digest_dockerhub="${{ steps.build-dockerhub.outputs.digest }}"
+          touch "/tmp/digests/ghcr/${digest_ghcr#sha256:}"
+          touch "/tmp/digests/dockerhub/${digest_dockerhub#sha256:}"
+
+      - name: upload ghcr digest
+        uses: actions/upload-artifact@v7
+        with:
+          name: digests-ghcr-${{ matrix.artifact }}
+          path: /tmp/digests/ghcr/*
+          retention-days: 1
+
+      - name: upload dockerhub digest
+        uses: actions/upload-artifact@v7
+        with:
+          name: digests-dockerhub-${{ matrix.artifact }}
+          path: /tmp/digests/dockerhub/*
+          retention-days: 1
+
+  merge:
+    name: Create multi-arch manifest
+    runs-on: ubuntu-latest
+    needs: build
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: download ghcr digests
+        uses: actions/download-artifact@v8
+        with:
+          path: /tmp/digests/ghcr
+          pattern: digests-ghcr-*
+          merge-multiple: true
+
+      - name: download dockerhub digests
+        uses: actions/download-artifact@v8
+        with:
+          path: /tmp/digests/dockerhub
+          pattern: digests-dockerhub-*
+          merge-multiple: true
+
+      - name: verify all digests present
+        run: |
+          expected=2
+          for registry in ghcr dockerhub; do
+            actual=$(find /tmp/digests/$registry -maxdepth 1 -type f | wc -l)
+            if [ "$actual" -ne "$expected" ]; then
+              echo "Expected $expected digests for $registry, found $actual"
+              ls -la /tmp/digests/$registry
+              exit 1
+            fi
+          done
+          echo "All digests present for both registries"
+
+      - name: set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+
+      - name: login to ghcr.io
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.PKG_TOKEN }}
+
+      - name: login to DockerHub
+        uses: docker/login-action@v4
+        with:
+          username: umputun
+          password: ${{ secrets.DOCKER_HUB_TOKEN }}
+
+      - name: create ghcr.io manifest and push
+        working-directory: /tmp/digests/ghcr
+        env:
+          HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
+        run: |
+          if [[ "$HEAD_BRANCH" == v* ]]; then
+            docker buildx imagetools create \
+              -t ghcr.io/umputun/remark42:${HEAD_BRANCH} \
+              -t ghcr.io/umputun/remark42:latest \
+              $(printf 'ghcr.io/umputun/remark42@sha256:%s ' *)
+          else
+            docker buildx imagetools create \
+              -t ghcr.io/umputun/remark42:${HEAD_BRANCH} \
+              $(printf 'ghcr.io/umputun/remark42@sha256:%s ' *)
+          fi
+
+      - name: create DockerHub manifest and push
+        working-directory: /tmp/digests/dockerhub
+        env:
+          HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
+        run: |
+          if [[ "$HEAD_BRANCH" == v* ]]; then
+            docker buildx imagetools create \
+              -t umputun/remark42:${HEAD_BRANCH} \
+              -t umputun/remark42:latest \
+              $(printf 'umputun/remark42@sha256:%s ' *)
+          else
+            docker buildx imagetools create \
+              -t umputun/remark42:${HEAD_BRANCH} \
+              $(printf 'umputun/remark42@sha256:%s ' *)
+          fi
+
+  deploy:
+    name: Deploy to remark42.com
+    runs-on: ubuntu-latest
+    needs: merge
+    if: github.event.workflow_run.head_branch == 'master'
+    permissions: {} # only calls an external URL via curl, no GitHub API access needed
+
+    steps:
+      - name: trigger deployment
+        env:
+          UPDATER_KEY: ${{ secrets.UPDATER_KEY }}
+        run: curl -sf https://jess.umputun.com/update/remark42-core/${UPDATER_KEY}

.github/workflows/e2e-tests.yml

@@ -0,0 +1,42 @@
+name: e2e
+
+on:
+  push:
+    branches: [master]
+    paths:
+      - ".github/workflows/e2e-tests.yml"
+      - "frontend/apps/remark42/**"
+      - "frontend/e2e/**"
+      - "frontend/Dockerfile.e2e"
+
+  pull_request:
+    branches: [master]
+    paths:
+      - ".github/workflows/e2e-tests.yml"
+      - "frontend/apps/remark42/**"
+      - "frontend/e2e/**"
+      - "frontend/Dockerfile.e2e"
+
+jobs:
+  tests:
+    name: Tests
+    timeout-minutes: 60
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Build & run containers
+        id: tests
+        run: COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker compose -f compose-e2e-test.yml up --build --quiet-pull --exit-code-from tests
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: playwright-report
+          path: ./playwright-report/
+          retention-days: 30

.github/workflows/release.yml

@@ -0,0 +1,143 @@
+name: release
+
+on:
+  push:
+    tags:
+      - "v*"
+  pull_request:
+    paths:
+      - ".github/workflows/release.yml"
+      - ".goreleaser.yml"
+      - "Makefile"
+      - "scripts/**"
+      - "backend/**"
+      - "frontend/**"
+      - "README.md"
+      - "LICENSE"
+      - "CLAUDE.md"
+      - "site/src/docs/getting-started/installation/index.md"
+
+permissions:
+  contents: read
+
+jobs:
+  validate:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: install go
+        uses: actions/setup-go@v6
+        with:
+          go-version: "1.25"
+          cache-dependency-path: backend/go.sum
+
+      - name: install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: install node
+        uses: actions/setup-node@v6
+        with:
+          node-version: 16
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
+
+      - name: test and build backend
+        run: |
+          go test -race -timeout=120s ./...
+          go build -race ./...
+        working-directory: backend/app
+        env:
+          TZ: "America/Chicago"
+
+      - name: test examples
+        run: |
+          go test -race ./...
+          go build -race ./...
+        working-directory: backend/_example/memory_store
+        env:
+          TZ: "America/Chicago"
+
+      - name: install frontend dependencies
+        run: pnpm install --frozen-lockfile
+        working-directory: frontend
+        env:
+          CI: "true"
+
+      - name: check frontend
+        run: |
+          pnpm lint
+          pnpm type-check
+          pnpm test -- --runInBand
+        working-directory: frontend/apps/remark42
+        env:
+          CI: "true"
+
+      - name: check goreleaser snapshot
+        if: github.event_name == 'pull_request'
+        uses: goreleaser/goreleaser-action@v7
+        with:
+          version: latest
+          args: release --snapshot --clean --skip=publish
+        env:
+          SKIP_PNPM_INSTALL: "true"
+
+      - name: clean generated release assets
+        if: always()
+        run: ./scripts/cleanup-release-assets.sh
+
+  release:
+    if: github.event_name == 'push'
+    needs: validate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: install go
+        uses: actions/setup-go@v6
+        with:
+          go-version: "1.25"
+          cache-dependency-path: backend/go.sum
+
+      - name: install pnpm
+        uses: pnpm/action-setup@v6.0.4
+        with:
+          version: 8
+          run_install: false
+
+      - name: install node
+        uses: actions/setup-node@v6
+        with:
+          node-version: 16
+          cache: "pnpm"
+          cache-dependency-path: frontend/pnpm-lock.yaml
+
+      - name: install frontend dependencies
+        run: pnpm install --frozen-lockfile
+        working-directory: frontend
+        env:
+          CI: "true"
+
+      - name: run goreleaser
+        uses: goreleaser/goreleaser-action@v7
+        with:
+          version: latest
+          args: release --clean
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SKIP_PNPM_INSTALL: "true"
+
+      - name: clean generated release assets
+        if: always()
+        run: ./scripts/cleanup-release-assets.sh

.gitignore

@@ -15,10 +15,18 @@ debug.test
 .mongo
 remark42
 /bin/
+/dist/
 /backend/var/
 /backend/app/var/
+/backend/app/cmd/web/
+/backend/*.html.tmpl
 compose-private-backend.yml
 compose-private-frontend.yml
 compose-private.yml
 /backend/_example/*/vendor
 http-client.env.json
+/playwright-report/
+/backend/app/cmd/var
+
+# ralphex progress logs
+.ralphex/progress/

.goreleaser.yml

@@ -0,0 +1,69 @@
+version: 2
+
+project_name: remark42
+
+git:
+  ignore_tags:
+    - backend/*
+
+before:
+  hooks:
+    - ./scripts/prepare-release-assets.sh
+
+builds:
+  - id: remark42
+    dir: backend
+    main: ./app
+    binary: "remark42.{{ .Os }}-{{ .Arch }}"
+    env:
+      - CGO_ENABLED=0
+    goos:
+      - linux
+      - darwin
+      - freebsd
+      - windows
+    goarch:
+      - amd64
+      - arm64
+      - "386"
+      - arm
+    goarm:
+      - "7"
+    ignore:
+      - goos: darwin
+        goarch: "386"
+      - goos: darwin
+        goarch: arm
+      - goos: freebsd
+        goarch: arm64
+      - goos: freebsd
+        goarch: "386"
+      - goos: freebsd
+        goarch: arm
+      - goos: windows
+        goarch: arm64
+      - goos: windows
+        goarch: "386"
+      - goos: windows
+        goarch: arm
+    ldflags:
+      - -s -w -X main.revision={{ .Tag }}-{{ .ShortCommit }}-{{ trimsuffix (replace (replace .CommitDate "-" "") ":" "") "Z" }}
+
+archives:
+  - id: remark42
+    ids:
+      - remark42
+    name_template: "{{ .ProjectName }}.{{ .Os }}-{{ .Arch }}"
+    formats:
+      - tar.gz
+    format_overrides:
+      - goos: windows
+        formats:
+          - zip
+    files:
+      - LICENSE
+      - README.md
+
+release:
+  name_template: "Version {{ .Version }}"
+  mode: keep-existing

CLAUDE.md

@@ -0,0 +1,59 @@
+# Remark42 Development Guidelines
+
+## Build/Test/Lint Commands
+- **Backend**:
+  - Run server: `make rundev`
+  - Build: `make backend`
+  - Race test: `make race_test`
+- **Backend Testing**:
+  - Run all tests: `cd backend/app && go test -timeout=60s -count 1 ./...`
+  - Run single test: `cd backend/app && go test -run TestName ./path/to/package`
+  - **IMPORTANT**: Run example tests: `cd backend/_example/memory_store && go test -race ./... && go build -race ./...`
+- **Frontend**:
+  - Development: `cd frontend && pnpm dev:app`
+  - Tests: `cd frontend && pnpm test`
+- **Lint**:
+  - Backend: `cd backend && golangci-lint run`
+  - **IMPORTANT**: Example lint: `cd backend/_example/memory_store && golangci-lint run --config ../../.golangci.yml`
+  - Frontend: `cd frontend && pnpm lint`
+  - **Before committing**: Always run tests and linter on both main backend AND examples
+- **Dependency Updates**:
+  - When updating Go modules in `backend/`, also run `go mod tidy` (and `go mod vendor`) in `backend/_example/memory_store` to keep indirect deps in sync. The example module replaces `github.com/umputun/remark42/backend` with `../../` so stale indirect deps there will break the example build.
+
+## Release Procedure
+
+Remark42 uses two tags for each release:
+- `vX.Y.Z` - product release tag used by GitHub releases, GoReleaser binary artifacts, and Docker image publishing.
+- `backend/vX.Y.Z` - nested Go module tag for `github.com/umputun/remark42/backend`.
+
+Release flow:
+1. Create the GitHub release for `vX.Y.Z` with title `Version X.Y.Z`. The GitHub release must exist before the `vX.Y.Z` tag reaches the remote; `gh release create vX.Y.Z` satisfies this because it creates and pushes the tag.
+2. The `vX.Y.Z` tag triggers GoReleaser, which builds and uploads binary artifacts to the existing release.
+3. Create and push the matching backend module tag pointing at the same commit:
+
+```bash
+git fetch origin --tags
+git tag backend/vX.Y.Z vX.Y.Z
+git push origin backend/vX.Y.Z
+```
+
+GoReleaser must ignore `backend/*` tags in `.goreleaser.yml` so release notes and current-tag detection use only product tags. Docker image publishing stays separate and is handled by the existing Docker workflow.
+
+For local artifact runs, install GoReleaser, Go 1.25, Node 16+, PNPM 8, and Perl, then use `make release`. The target runs a snapshot/no-publish GoReleaser build, leaves local artifacts and metadata in `dist/`, and cleans generated frontend embed files after GoReleaser exits. Do not run raw `goreleaser release` for local artifacts unless you also run `./scripts/cleanup-release-assets.sh` afterward.
+
+## Code Style
+- **Backend**: Formatting with golangci-lint, strict error handling
+- **Frontend**: TypeScript with ESLint, Stylelint and Prettier
+- **Imports**: Group stdlib, external packages, then internal packages
+- **CSS**: All components use CSS Modules (`component.module.css`). Class naming: BEM block = `.root`, elements = camelCase, modifiers = camelCase. Use `clsx` for conditional class composition. `raw-content.css` is the only global CSS file (syntax highlighting utility). Root wrapper keeps bare `.dark`/`.light` theme class — 8+ module CSS files depend on `:global(.dark)` ancestor. `comment_highlighting` uses `:global()` for imperative `classList` usage in root.tsx
+
+## Key Backend Packages
+- **Web/API**: `github.com/go-chi/chi/v5`, `github.com/go-pkgz/rest`
+- **Auth**: `github.com/go-pkgz/auth/v2`
+- **Logging**: `github.com/go-pkgz/lgr`
+- **Testing**: `github.com/stretchr/testify`
+- **Notifications**: `github.com/go-pkgz/notify`
+
+## Repository Structure
+- Backend: Go server using BoltDB for storage
+- Frontend: Preact/Redux-based UI with iframe embedding

Dockerfile

@@ -1,82 +1,113 @@
-FROM umputun/baseimage:buildgo-v1.8.0 as build-backend
+FROM --platform=$BUILDPLATFORM node:16.20-alpine AS frontend-deps
+
+ARG SKIP_FRONTEND_TEST
+ARG SKIP_FRONTEND_BUILD
+
+WORKDIR /srv/frontend/
+
+COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml /srv/frontend/
+COPY ./frontend/apps/remark42/package.json /srv/frontend/apps/remark42/
+
+RUN \
+  if [[ -z "$SKIP_FRONTEND_BUILD" || -z "$SKIP_FRONTEND_TEST" ]]; then \
+    apk add --no-cache --update git && \
+    npm i -g pnpm@8; \
+  fi
+
+RUN --mount=type=cache,id=pnpm,target=/root/.pnpm-store/v3 \
+  if [[ -z "$SKIP_FRONTEND_BUILD" || -z "$SKIP_FRONTEND_TEST" ]]; then \
+    pnpm i; \
+  fi
+
+
+FROM --platform=$BUILDPLATFORM frontend-deps AS build-frontend
+
+ARG SKIP_FRONTEND_TEST
+ARG SKIP_FRONTEND_BUILD
+ENV CI=true
+
+WORKDIR /srv/frontend/apps/remark42/
+
+COPY ./frontend/apps/remark42/ /srv/frontend/apps/remark42/
+
+RUN \
+  if [ -z "$SKIP_FRONTEND_TEST" ]; then \
+    pnpm lint type-check translation-check test; \
+  else \
+    echo 'Skip frontend test'; \
+  fi
+
+RUN \
+  if [ -z "$SKIP_FRONTEND_BUILD" ]; then \
+    pnpm build; \
+  else \
+    mkdir /srv/frontend/apps/remark42/public; \
+    echo 'Skip frontend build'; \
+  fi
+
+FROM umputun/baseimage:buildgo-v1.17.0 AS build-backend
 
 ARG CI
-ARG DRONE
+ARG GITHUB_REF
-ARG DRONE_TAG
+ARG GITHUB_SHA
-ARG DRONE_COMMIT
+ARG GIT_BRANCH
-ARG DRONE_BRANCH
-ARG DRONE_PULL_REQUEST
-
 ARG SKIP_BACKEND_TEST
 ARG BACKEND_TEST_TIMEOUT
 
-ADD backend /build/backend
-ADD .git/ /build/backend/.git/
-WORKDIR /build/backend
-
-ENV GOFLAGS="-mod=vendor"
-
 # install gcc in order to be able to go test package with -race
 RUN apk --no-cache add gcc libc-dev
 
+ADD backend /build/backend
+# to embed the frontend files statically into Remark42 binary
+COPY --from=build-frontend /srv/frontend/apps/remark42/public/ /build/backend/app/cmd/web/
+RUN find /build/backend/app/cmd/web/ -regex '.*\.\(html\|js\|mjs\)$' -print -exec sed -i "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \;
+WORKDIR /build/backend
+
+RUN echo go version: `go version`
+
 # run tests
 RUN \
     cd app && \
     if [ -z "$SKIP_BACKEND_TEST" ] ; then \
         CGO_ENABLED=1 go test -race -p 1 -timeout="${BACKEND_TEST_TIMEOUT:-300s}" -covermode=atomic -coverprofile=/profile.cov_tmp ./... && \
-        cat /profile.cov_tmp | grep -v "_mock.go" > /profile.cov ; \
+        cat /profile.cov_tmp | grep -v "_mock.go" > /profile.cov && \
         golangci-lint run --config ../.golangci.yml ./... ; \
-    else echo "skip backend tests and linter" ; fi
+    else \
+      echo "skip backend tests and linter" \
+    ; fi
 
-# if DRONE presented use DRONE_* git env to make version
 RUN \
-    if [ -z "$DRONE" ] ; then echo "runs outside of drone" && version="$(/script/git-rev.sh)" ; \
+    version="$(/script/version.sh)" && \
-    else version=${DRONE_TAG}${DRONE_BRANCH}${DRONE_PULL_REQUEST}-${DRONE_COMMIT:0:7}-$(date +%Y%m%d-%H:%M:%S) ; fi && \
     echo "version=$version" && \
     go build -o remark42 -ldflags "-X main.revision=${version} -s -w" ./app
 
-FROM node:12.16-alpine as build-frontend-deps
+FROM umputun/baseimage:app-v1.17.0
 
-ARG CI
+ARG GITHUB_SHA
-ENV HUSKY_SKIP_INSTALL=true
 
-RUN apk add --no-cache --update git
+LABEL org.opencontainers.image.authors="Umputun <umputun@gmail.com>" \
-ADD frontend/package.json /srv/frontend/package.json
+      org.opencontainers.image.description="Remark42 comment engine" \
-ADD frontend/package-lock.json /srv/frontend/package-lock.json
+      org.opencontainers.image.documentation="https://remark42.com/docs/getting-started/" \
-RUN cd /srv/frontend && CI=true npm ci --loglevel warn
+      org.opencontainers.image.licenses="MIT" \
-
+      org.opencontainers.image.source="https://github.com/umputun/remark42" \
-FROM node:12.16-alpine as build-frontend
+      org.opencontainers.image.title="Remark42" \
-
+      org.opencontainers.image.url="https://remark42.com/" \
-ARG CI
+      org.opencontainers.image.revision="${GITHUB_SHA}"
-ARG SKIP_FRONTEND_TEST
-ARG NODE_ENV=production
-
-COPY --from=build-frontend-deps /srv/frontend/node_modules /srv/frontend/node_modules
-ADD frontend /srv/frontend
-RUN cd /srv/frontend && \
-    if [ -z "$SKIP_FRONTEND_TEST" ] ; then npx run-p lint test check; \
-    else echo "skip frontend tests and lint" ; npm run build ; fi && \
-    rm -rf ./node_modules
-
-FROM umputun/baseimage:app-v1.8.0
 
 WORKDIR /srv
 
-ADD docker-init.sh /entrypoint.sh
+COPY docker-init.sh /srv/init.sh
 ADD backend/scripts/backup.sh /usr/local/bin/backup
 ADD backend/scripts/restore.sh /usr/local/bin/restore
 ADD backend/scripts/import.sh /usr/local/bin/import
-RUN chmod +x /entrypoint.sh /usr/local/bin/backup /usr/local/bin/restore /usr/local/bin/import
+RUN chmod +x /srv/init.sh /usr/local/bin/backup /usr/local/bin/restore /usr/local/bin/import
 
 COPY --from=build-backend /build/backend/remark42 /srv/remark42
-COPY --from=build-backend /build/backend/templates /srv
+COPY --from=build-frontend /srv/frontend/apps/remark42/public/ /srv/web/
-COPY --from=build-frontend /srv/frontend/public/ /srv/web
-COPY docker-init.sh /srv/init.sh
 RUN chown -R app:app /srv
 RUN ln -s /srv/remark42 /usr/bin/remark42
 
 EXPOSE 8080
 HEALTHCHECK --interval=30s --timeout=3s CMD curl --fail http://localhost:8080/ping || exit 1
 
-RUN chmod +x /srv/init.sh
 CMD ["/srv/remark42", "server"]

Dockerfile.artifacts

@@ -1,87 +0,0 @@
-FROM node:12.16-alpine as build-frontend-deps
-
-ARG CI
-ARG DRONE
-ARG DRONE_TAG
-ARG DRONE_COMMIT
-ARG DRONE_BRANCH
-
-ENV SKIP_FRONTEND_TEST=true
-
-RUN apk add --no-cache --update git
-ADD frontend/package.json /srv/frontend/package.json
-ADD frontend/package-lock.json /srv/frontend/package-lock.json
-RUN cd /srv/frontend && CI=true npm ci
-
-FROM node:12.16-alpine as build-frontend
-
-ARG CI
-ARG NODE_ENV=production
-ENV SKIP_FRONTEND_TEST=true
-ENV HUSKY_SKIP_INSTALL=true
-
-COPY --from=build-frontend-deps /srv/frontend/node_modules /srv/frontend/node_modules
-ADD frontend /srv/frontend
-RUN cd /srv/frontend && \
-    npm run build && \
-    rm -rf ./node_modules
-
-FROM umputun/baseimage:buildgo-latest as build-backend
-
-ARG GITHUB_TOKEN
-ENV SKIP_BACKEND_TEST=true
-
-WORKDIR /build/backend
-ADD backend /build/backend
-ADD README.md /build/
-ADD LICENSE /build/
-
-ADD .git/ /build/backend/.git/
-
-COPY --from=build-frontend /srv/frontend/public/ web
-
-RUN \
-    export WEB_ROOT=/build/backend/web && \
-    find . -regex '.*\.\(html\|js\|mjs\)$' -print -exec sed -i "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \; && \
-    statik --src=${WEB_ROOT} --dest=/build/backend/app/rest -p api -f && \
-    statik --src=/build/backend/templates --dest=/build/backend/app -p templates -ns templates -f && \
-    ls -la /build/backend/app/templates/statik.go && \
-    ls -la /build/backend/app/rest/api/statik.go && \
-    ls -la /build/backend/web/
-
-# if DRONE presented use DRONE_* git env to make version
-RUN \
-    if [ -z "$DRONE" ] ; then \
-    echo "runs outside of drone" && version=$("/script/version.sh"); \
-    else version=${DRONE_TAG}${DRONE_BRANCH}${DRONE_PULL_REQUEST}-${DRONE_COMMIT:0:7}-$(date +%Y%m%d-%H:%M:%S); fi && \
-    echo "version=$version" && \
-    export GOFLAGS="-mod=vendor" && \
-    GOOS=linux GOARCH=amd64 go build -o remark42.linux-amd64 -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=linux GOARCH=386 go build -o remark42.linux-386 -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=linux GOARCH=arm go build -o remark42.linux-arm -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=linux GOARCH=arm64 go build -o remark42.linux-arm64 -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=windows GOARCH=amd64 go build -o remark42.windows-amd64.exe -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=darwin GOARCH=amd64 go build -o remark42.darwin-amd64 -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=darwin GOARCH=arm64 go build -o remark42.darwin-arm64 -ldflags "-X main.revision=${version} -s -w" ./app && \
-    GOOS=freebsd GOARCH=amd64 go build -o remark42.freebsd-amd64 -ldflags "-X main.revision=${version} -s -w" ./app
-
-RUN \
-    if [ -z "$DRONE_TAG" ] ; then \
-    echo "runs outside of drone" && tag=""; \
-    else tag=_${DRONE_TAG}; fi && \
-    apk add --no-cache --update zip && \
-    cp ../LICENSE ./LICENSE && cp ../README.md ./README.md && \
-    tar cvzf remark42${tag}.linux-amd64.tar.gz remark42.linux-amd64 LICENSE README.md && \
-    tar cvzf remark42${tag}.linux-386.tar.gz remark42.linux-386 LICENSE README.md && \
-    tar cvzf remark42${tag}.linux-arm.tar.gz remark42.linux-arm LICENSE README.md && \
-    tar cvzf remark42${tag}.linux-arm64.tar.gz remark42.linux-arm64 LICENSE README.md && \
-    tar cvzf remark42${tag}.darwin-amd64.tar.gz remark42.darwin-amd64 LICENSE README.md && \
-    tar cvzf remark42${tag}.darwin-arm64.tar.gz remark42.darwin-arm64 LICENSE README.md && \
-    tar cvzf remark42${tag}.freebsd-amd64.tar.gz remark42.freebsd-amd64 LICENSE README.md && \
-    zip remark42${tag}.windows-amd64.zip remark42.windows-amd64.exe LICENSE README.md
-
-
-FROM alpine
-COPY --from=build-backend /build/backend/remark42.* /artifacts/
-RUN ls -la /artifacts/*
-CMD ["sleep", "100"]

Makefile

@@ -1,46 +1,45 @@
 OS=linux
 ARCH=amd64
+GITHUB_REF=$(shell git rev-parse --symbolic-full-name HEAD)
+GITHUB_SHA=$(shell git rev-parse --short HEAD)
+CLEANUP_RELEASE_ASSETS=$(CURDIR)/scripts/cleanup-release-assets.sh
 
 bin:
-	docker build -f Dockerfile.artifacts -t remark42.bin .
+	@set -e; \
-	- @docker rm -f remark42.bin 2>/dev/null || exit 0
+		./scripts/prepare-release-assets.sh; \
-	docker run -d --name=remark42.bin remark42.bin
+		trap '$(CLEANUP_RELEASE_ASSETS)' EXIT; \
-	docker cp remark42.bin:/artifacts/remark42.$(OS)-$(ARCH) remark42
+		cd backend && CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) go build -o ../remark42 -ldflags "-X main.revision=$(GITHUB_REF)-$(GITHUB_SHA) -s -w" ./app
-	docker rm -f remark42.bin
 
 docker:
-	docker build -t umputun/remark42 --build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true .
+	DOCKER_BUILDKIT=1 docker build -t umputun/remark42 -t ghcr.io/umputun/remark42 --build-arg GITHUB_REF=$(GITHUB_REF) --build-arg GITHUB_SHA=$(GITHUB_SHA) \
+		--build-arg CI=true --build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true .
 
 dockerx:
-	docker buildx build --build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true \
+	docker buildx build --build-arg GITHUB_REF=$(GITHUB_REF) --build-arg GITHUB_SHA=$(GITHUB_SHA) --build-arg CI=true \
-		  --progress=plain --platform linux/amd64,linux/arm/v7,linux/arm64 \
+		--build-arg SKIP_FRONTEND_TEST=true --build-arg SKIP_BACKEND_TEST=true \
-		  -t ghcr.io/umputun/remark42:master -t umputun/remark42:master .
+		--progress=plain --platform linux/amd64,linux/arm/v7,linux/arm64 \
+		-t ghcr.io/umputun/remark42:master -t umputun/remark42:master .
 
 release:
-	docker build -f Dockerfile.artifacts --no-cache --pull -t remark42.bin .
+	@set -e; \
-	- @docker rm -f remark42.bin 2>/dev/null || exit 0
+		trap '$(CLEANUP_RELEASE_ASSETS)' EXIT; \
-	- @mkdir -p bin
+		goreleaser release --snapshot --clean --skip=publish
-	docker run -d --name=remark42.bin remark42.bin
-	docker cp remark42.bin:/artifacts/remark42.linux-amd64.tar.gz bin/remark42.linux-amd64.tar.gz
-	docker cp remark42.bin:/artifacts/remark42.linux-386.tar.gz bin/remark42.linux-386.tar.gz
-	docker cp remark42.bin:/artifacts/remark42.linux-arm64.tar.gz bin/remark42.linux-arm64.tar.gz
-	docker cp remark42.bin:/artifacts/remark42.darwin-amd64.tar.gz bin/remark42.darwin-amd64.tar.gz
-	docker cp remark42.bin:/artifacts/remark42.freebsd-amd64.tar.gz bin/remark42.freebsd-amd64.tar.gz
-	docker cp remark42.bin:/artifacts/remark42.windows-amd64.zip bin/remark42.windows-amd64.zip
-	docker rm -f remark42.bin
 
 race_test:
-	cd backend/app && go test -race -mod=vendor -timeout=60s -count 1 ./...
+	cd backend/app && go test -race -timeout=60s -count 1 ./...
 
 backend:
-	docker-compose -f compose-dev-backend.yml build
+	docker compose -f compose-dev-backend.yml build
 
 frontend:
-	docker-compose -f compose-dev-frontend.yml build
+	docker compose -f compose-dev-frontend.yml build
 
 rundev:
-	docker pull umputun/baseimage:buildgo-latest
+	SKIP_BACKEND_TEST=true SKIP_FRONTEND_TEST=true GITHUB_REF=$(GITHUB_REF) GITHUB_SHA=$(GITHUB_SHA) CI=true \
-	SKIP_BACKEND_TEST=true SKIP_FRONTEND_TEST=true docker-compose -f compose-private.yml build
+		docker compose -f compose-private.yml build
-	docker-compose -f compose-private.yml up
+	docker compose -f compose-private.yml up
 
-.PHONY: bin backend
+e2e:
+	docker compose -f compose-e2e-test.yml up --build --quiet-pull --exit-code-from tests
+
+.PHONY: bin docker dockerx release race_test backend frontend rundev e2e

README.md

@@ -1,8 +1,8 @@
-# Remark42 [![Build Status](https://github.com/umputun/remark42/workflows/build/badge.svg)](https://github.com/umputun/remark42/actions) [![Go Report Card](https://goreportcard.com/badge/github.com/umputun/remark42)](https://goreportcard.com/report/github.com/umputun/remark42) [![Coverage Status](https://coveralls.io/repos/github/umputun/remark42/badge.svg?branch=master)](https://coveralls.io/github/umputun/remark42?branch=master) [![codecov](https://codecov.io/gh/umputun/remark42/branch/master/graph/badge.svg)](https://app.codecov.io/gh/umputun/remark42)
+# Remark42 [![Build Status](https://github.com/umputun/remark42/workflows/build/badge.svg)](https://github.com/umputun/remark42/actions) [![Image Size](https://img.shields.io/docker/image-size/umputun/remark42/master)](https://hub.docker.com/r/umputun/remark42) [![Go Report Card](https://goreportcard.com/badge/github.com/umputun/remark42)](https://goreportcard.com/report/github.com/umputun/remark42) [![Coverage Status](https://coveralls.io/repos/github/umputun/remark42/badge.svg?branch=master)](https://coveralls.io/github/umputun/remark42?branch=master) [![codecov](https://codecov.io/gh/umputun/remark42/branch/master/graph/badge.svg)](https://app.codecov.io/gh/umputun/remark42)
 
 Remark42 is a self-hosted, lightweight and simple (yet functional) comment engine, which doesn't spy on users. It can be embedded into blogs, articles, or any other place where readers add comments.
 
-* Social login via Google, Twitter, Facebook, Microsoft, GitHub, Yandex, Patreon and Telegram
+* Social login via Google, Facebook, Microsoft, GitHub, Apple, Yandex, Patreon, Discord, Telegram and custom OAuth2 providers
 * Login via email
 * Optional anonymous access
 * Multi-level nested comments with both tree and plain presentations
@@ -23,7 +23,7 @@ Remark42 is a self-hosted, lightweight and simple (yet functional) comment engin
 * Clean, lightweight and customizable UI with white and dark themes
 * Multi-site mode from a single instance
 * Integration with automatic SSL (direct and via [nginx-le](https://github.com/nginx-le/nginx-le))
-* [Privacy focused](#privacy)
+* [Privacy focused](https://remark42.com/#privacy)
 
 [Demo site](https://remark42.com/demo/) available with all authentication methods, including email auth and anonymous access.
 
@@ -32,10 +32,16 @@ Remark42 is a self-hosted, lightweight and simple (yet functional) comment engin
 Comments example:
 ![](screenshots/comments.png)
 
-For admin screenshots see [Admin UI wiki](https://github.com/umputun/remark42/wiki/Admin-UI)
+For admin screenshots see [Admin UI documentation](https://remark42.com/docs/manuals/admin-interface/)
 </details>
 
-All remark42 documentation is available [by the link](https://remark42.com/docs/).
+All remark42 documentation is available [by the link](https://remark42.com/docs/getting-started/installation/).
+
+## Contribution
+
+In order to start and work on the project locally in development mode check our contribution documentation for [backend](https://remark42.com/docs/contributing/backend/) and [frontend](https://remark42.com/docs/contributing/frontend/).
+
+If you are interested in adding a new localization please check [these docs](https://remark42.com/docs/contributing/translations/).
 
 ## Related projects
 

SECURITY.md

@@ -12,4 +12,4 @@ We release patches for security vulnerabilities.
 
 ## Reporting a Vulnerability
 
-Please report (suspected) security vulnerabilities to umputun@gmail.com. You will receive a response from us within 48 hours. If the issue is confirmed, we will release a patch as soon as possible depending on complexity but historically within a few days.
+Please report (suspected) security vulnerabilities either by using GitHub's [private vulnerability reporting](https://github.com/umputun/remark42/security/advisories/new) (click the "Report a vulnerability" button on the [Security tab](https://github.com/umputun/remark42/security)) or by emailing umputun@gmail.com. You will receive a response within 48 hours. If the issue is confirmed, we will release a patch as soon as possible depending on complexity but historically within a few days.

backend/.golangci.yml

@@ -1,74 +1,75 @@
-run:
+version: "2"
-  timeout: 5m
-  output:
-    format: tab
-  skip-dirs:
-    - vendor
-
-linters-settings:
-  govet:
-    check-shadowing: true
-  golint:
-    min-confidence: 0.1
-  maligned:
-    suggest-new: true
-  goconst:
-    min-len: 2
-    min-occurrences: 2
-  misspell:
-    locale: US
-  lll:
-    line-length: 140
-  gocritic:
-    enabled-tags:
-      - performance
-      - style
-      - experimental
-    disabled-checks:
-      - wrapperFunc
-# TODO: feel free to remove these excludes and fix the code
-      - hugeParam
-      - rangeValCopy
-
 linters:
+  default: none
   enable:
     - bodyclose
-    - megacheck
+    - copyloopvar
-    - revive
-    - govet
-    - unconvert
-    - megacheck
-    - structcheck
-    - gas
-    - gocyclo
     - dupl
-    - misspell
-    - unparam
-    - varcheck
-    - deadcode
-    - typecheck
-    - ineffassign
-    - varcheck
-    - stylecheck
     - gochecknoinits
-    - exportloopref
     - gocritic
+    - gocyclo
+    - gosec
+    - govet
+    - ineffassign
+    - misspell
     - nakedret
-    - gosimple
     - prealloc
-  fast: false
+    - revive
-  disable-all: true
+    - staticcheck
-
+    - unconvert
-issues:
+    - unparam
-  exclude-rules:
+    - unused
-    - text: "at least one file in a package should have a package comment"
+  settings:
-      linters:
+    goconst:
-        - stylecheck
+      min-len: 2
-    - text: "should have a package comment, unless it's in another file for this package"
+      min-occurrences: 2
-      linters:
+    gosec:
-        - golint
+      excludes:
-    - path: _test\.go
+        - G117 # false positive: struct field name matches "secret" pattern
-      linters:
+    gocritic:
-        - gosec
+      disabled-checks:
-        - dupl
+        - wrapperFunc
-  exclude-use-default: false
+        - hugeParam
+        - rangeValCopy
+      enabled-tags:
+        - performance
+        - style
+        - experimental
+    govet:
+      enable:
+        - shadow
+    lll:
+      line-length: 140
+    misspell:
+      locale: US
+  exclusions:
+    generated: lax
+    rules:
+      - linters:
+          - staticcheck
+        text: at least one file in a package should have a package comment
+      - linters:
+          - revive
+        text: 'package-comments: should have a package comment'
+      - linters:
+          - revive
+        text: 'var-naming: avoid meaningless package names'
+      - linters:
+          - revive
+        text: 'var-naming: avoid package names that conflict with Go standard library package names'
+      - linters:
+          - dupl
+          - gosec
+        path: _test\.go
+    paths:
+      - vendor
+      - third_party$
+      - builtin$
+      - examples$
+formatters:
+  exclusions:
+    generated: lax
+    paths:
+      - third_party$
+      - builtin$
+      - examples$

backend/README.md

@@ -0,0 +1 @@
+../site/src/docs/contributing/backend/index.md

backend/_example/memory_store/Dockerfile

@@ -1,12 +1,22 @@
-FROM umputun/baseimage:buildgo-latest as build-backend
+FROM umputun/baseimage:buildgo-v1.17.0 AS build-backend
 
 ADD backend /build/backend
 WORKDIR /build/backend/_example/memory_store
 
 RUN go build -o /build/bin/memory_store -ldflags "-X main.revision=0.0.0 -s -w"
 
+FROM umputun/baseimage:app-v1.17.0
 
-FROM umputun/baseimage:app-latest
+ARG GITHUB_SHA
+
+LABEL org.opencontainers.image.authors="Umputun <umputun@gmail.com>" \
+      org.opencontainers.image.description="Remark42 comment engine example JRPC memory store" \
+      org.opencontainers.image.documentation="https://github.com/umputun/remark42/tree/master/backend/_example/memory_store" \
+      org.opencontainers.image.licenses="MIT" \
+      org.opencontainers.image.source="https://github.com/umputun/remark42" \
+      org.opencontainers.image.title="Remark42 JRPC example memory store" \
+      org.opencontainers.image.url="https://remark42.com/" \
+      org.opencontainers.image.revision="${GITHUB_SHA}"
 
 WORKDIR /srv
 COPY --from=build-backend /build/bin/memory_store /srv/memory_store

backend/_example/memory_store/README.md

@@ -4,9 +4,9 @@
 
 In order to run remark42 with memory_store copy provided `compose-dev-memstore.yml` to the root directory and run:
 
-1. `docker-compose -f compose-dev-memstore.yml build`
+1. `docker compose -f compose-dev-memstore.yml build`
-1. `docker-compose -f compose-dev-memstore.yml up`
+1. `docker compose -f compose-dev-memstore.yml up`
 
 As usual, demo site will run on http://127.0.0.1:8080/web/
 
-note: in order to work with the latest (current) version of master `go.mod` uses replacement directive for the backend package. In real-life usage `replace github.com/umputun/remark42/backend => ../../`  should not be used. 
+note: in order to work with the latest (current) version of master `go.mod` uses replacement directive for the backend package. In real-life usage `replace github.com/umputun/remark42/backend => ../../`  should not be used. 

backend/_example/memory_store/accessor/admin.go

@@ -7,8 +7,9 @@
 package accessor
 
 import (
+	"fmt"
+
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store/admin"
 )
@@ -34,7 +35,8 @@ func NewMemAdminStore(key string) *MemAdmin {
 	return &MemAdmin{data: map[string]AdminRec{}, key: key}
 }
 
-// Key executes find by siteID and returns substructure with secret key
+// Key supposed to execute find by siteID and returns substructure with secret key,
+// but in this case the shared secret is used for all sites
 func (m *MemAdmin) Key(_ string) (key string, err error) {
 	return m.key, nil
 }
@@ -43,7 +45,7 @@ func (m *MemAdmin) Key(_ string) (key string, err error) {
 func (m *MemAdmin) Admins(siteID string) (ids []string, err error) {
 	resp, ok := m.data[siteID]
 	if !ok {
-		return nil, errors.Errorf("site %s not found", siteID)
+		return nil, fmt.Errorf("site %s not found", siteID)
 	}
 	log.Printf("[DEBUG] admins for %s, %+v", siteID, resp.IDs)
 	return resp.IDs, nil
@@ -53,7 +55,7 @@ func (m *MemAdmin) Admins(siteID string) (ids []string, err error) {
 func (m *MemAdmin) Email(siteID string) (email string, err error) {
 	resp, ok := m.data[siteID]
 	if !ok {
-		return "", errors.Errorf("site %s not found", siteID)
+		return "", fmt.Errorf("site %s not found", siteID)
 	}
 
 	return resp.Email, nil
@@ -63,7 +65,7 @@ func (m *MemAdmin) Email(siteID string) (email string, err error) {
 func (m *MemAdmin) Enabled(siteID string) (ok bool, err error) {
 	resp, ok := m.data[siteID]
 	if !ok {
-		return false, errors.Errorf("site %s not found", siteID)
+		return false, fmt.Errorf("site %s not found", siteID)
 	}
 	return resp.Enabled, nil
 }
@@ -72,7 +74,7 @@ func (m *MemAdmin) Enabled(siteID string) (ok bool, err error) {
 func (m *MemAdmin) OnEvent(siteID string, ev admin.EventType) error {
 	resp, ok := m.data[siteID]
 	if !ok {
-		return errors.Errorf("site %s not found", siteID)
+		return fmt.Errorf("site %s not found", siteID)
 	}
 	if ev == admin.EvCreate {
 		resp.CountCreated++ // not a good idea, just for demo

backend/_example/memory_store/accessor/data.go

@@ -7,13 +7,12 @@
 package accessor
 
 import (
-	"log"
+	"fmt"
 	"sort"
 	"sync"
 	"time"
 
-	"github.com/pkg/errors"
+	log "github.com/go-pkgz/lgr"
-
 	"github.com/umputun/remark42/backend/app/store"
 	"github.com/umputun/remark42/backend/app/store/engine"
 )
@@ -58,7 +57,7 @@ func NewMemData() *MemData {
 func (m *MemData) Create(comment store.Comment) (commentID string, err error) {
 
 	if ro, e := m.Flag(engine.FlagRequest{Flag: engine.ReadOnly, Locator: comment.Locator}); e == nil && ro {
-		return "", errors.Errorf("post %s is read-only", comment.Locator.URL)
+		return "", fmt.Errorf("post %s is read-only", comment.Locator.URL)
 	}
 
 	m.mu.Lock()
@@ -66,7 +65,7 @@ func (m *MemData) Create(comment store.Comment) (commentID string, err error) {
 	comments := m.posts[comment.Locator.SiteID]
 	for _, c := range comments { // don't allow duplicated IDs
 		if c.ID == comment.ID {
-			return "", errors.New("dup key")
+			return "", fmt.Errorf("dup key")
 		}
 	}
 	comments = append(comments, comment)
@@ -161,7 +160,7 @@ func (m *MemData) Count(req engine.FindRequest) (count int, err error) {
 		})
 		return len(comments), nil
 	default:
-		return 0, errors.Errorf("invalid count request %+v", req)
+		return 0, fmt.Errorf("invalid count request %+v", req)
 	}
 }
 
@@ -176,7 +175,7 @@ func (m *MemData) Info(req engine.InfoRequest) (res []store.PostInfo, err error)
 			return c.Locator == req.Locator
 		})
 		if len(comments) == 0 {
-			return nil, errors.New("not found")
+			return nil, fmt.Errorf("not found")
 		}
 		info := store.PostInfo{
 			URL:      req.Locator.URL,
@@ -235,7 +234,7 @@ func (m *MemData) Info(req engine.InfoRequest) (res []store.PostInfo, err error)
 		return res, nil
 	}
 
-	return nil, errors.Errorf("invalid info request %+v", req)
+	return nil, fmt.Errorf("invalid info request %+v", req)
 }
 
 // Flag sets and gets flag values
@@ -252,11 +251,11 @@ func (m *MemData) Flag(req engine.FlagRequest) (val bool, err error) {
 
 // ListFlags get list of flagged keys, like blocked & verified user
 // works for full locator (post flags) or with userID
-func (m *MemData) ListFlags(req engine.FlagRequest) (res []interface{}, err error) {
+func (m *MemData) ListFlags(req engine.FlagRequest) (res []any, err error) {
 	m.mu.RLock()
 	defer m.mu.RUnlock()
 
-	res = []interface{}{}
+	res = []any{}
 
 	switch req.Flag {
 	case engine.Verified:
@@ -268,7 +267,7 @@ func (m *MemData) ListFlags(req engine.FlagRequest) (res []interface{}, err erro
 		return res, nil
 
 	case engine.Blocked:
-		log.Printf("%+v", m.metaUsers)
+		log.Printf("[INFO] metaUsers: %+v", m.metaUsers)
 		for _, u := range m.metaUsers {
 			if u.SiteID == req.Locator.SiteID && u.Blocked && u.BlockedUntil.After(time.Now()) {
 				res = append(res, store.BlockedUser{ID: u.UserID, Until: u.BlockedUntil})
@@ -277,7 +276,7 @@ func (m *MemData) ListFlags(req engine.FlagRequest) (res []interface{}, err erro
 		return res, nil
 	}
 
-	return nil, errors.Errorf("flag %s not listable", req.Flag)
+	return nil, fmt.Errorf("flag %s not listable", req.Flag)
 }
 
 // UserDetail sets or gets single detail value, or gets all details fo§r requested site.
@@ -287,28 +286,28 @@ func (m *MemData) UserDetail(req engine.UserDetailRequest) ([]engine.UserDetailE
 	switch req.Detail {
 	case engine.UserEmail, engine.UserTelegram:
 		if req.UserID == "" {
-			return nil, errors.New("userid cannot be empty in request for single detail")
+			return nil, fmt.Errorf("userid cannot be empty in request for single detail")
 		}
 
 		m.mu.Lock()
 		defer m.mu.Unlock()
 
 		if req.Update == "" { // read detail value, no update requested
-			return m.getUserDetail(req)
+			return m.getUserDetail(req), nil
 		}
 
-		return m.setUserDetail(req)
+		return m.setUserDetail(req), nil
 	case engine.AllUserDetails:
 		// list of all details returned in case request is a read request
 		// (Update is not set) and does not have UserID or Detail set
 		if req.Update == "" && req.UserID == "" { // read list of all details
 			m.mu.Lock()
 			defer m.mu.Unlock()
-			return m.listDetails(req.Locator)
+			return m.listDetails(req.Locator), nil
 		}
-		return nil, errors.New("unsupported request with userdetail all")
+		return nil, fmt.Errorf("unsupported request with userdetail all")
 	default:
-		return nil, errors.Errorf("unsupported detail %q", req.Detail)
+		return nil, fmt.Errorf("unsupported detail %q", req.Detail)
 	}
 }
 
@@ -320,7 +319,8 @@ func (m *MemData) Delete(req engine.DeleteRequest) error {
 
 	switch {
 	case req.UserDetail != "": // delete user detail
-		return m.deleteUserDetail(req.Locator, req.UserID, req.UserDetail)
+		m.deleteUserDetail(req.Locator, req.UserID, req.UserDetail)
+		return nil
 	case req.Locator.URL != "" && req.CommentID != "" && req.UserDetail == "": // delete comment
 		return m.deleteComment(req.Locator, req.CommentID, req.DeleteMode)
 
@@ -333,17 +333,18 @@ func (m *MemData) Delete(req engine.DeleteRequest) error {
 				return e
 			}
 		}
-		return m.deleteUserDetail(req.Locator, req.UserID, engine.AllUserDetails)
+		m.deleteUserDetail(req.Locator, req.UserID, engine.AllUserDetails)
+		return nil
 
 	case req.Locator.SiteID != "" && req.Locator.URL == "" && req.CommentID == "" && req.UserID == "" && req.UserDetail == "": // delete site
 		if _, ok := m.posts[req.Locator.SiteID]; !ok {
-			return errors.New("not found")
+			return fmt.Errorf("not found")
 		}
 		m.posts[req.Locator.SiteID] = []store.Comment{}
 		return nil
 	}
 
-	return errors.Errorf("invalid delete request %+v", req)
+	return fmt.Errorf("invalid delete request %+v", req)
 }
 
 func (m *MemData) deleteComment(loc store.Locator, id string, mode store.DeleteMode) error {
@@ -352,7 +353,7 @@ func (m *MemData) deleteComment(loc store.Locator, id string, mode store.DeleteM
 		return c.Locator == loc && c.ID == id
 	})
 	if len(comments) == 0 {
-		return errors.New("not found")
+		return fmt.Errorf("not found")
 	}
 
 	comments[0].SetDeleted(mode)
@@ -390,10 +391,7 @@ func (m *MemData) checkFlag(req engine.FlagRequest) (val bool) {
 
 func (m *MemData) setFlag(req engine.FlagRequest) (res bool, err error) {
 
-	status := false
+	status := req.Update == engine.FlagTrue
-	if req.Update == engine.FlagTrue {
-		status = true
-	}
 
 	switch req.Flag {
 
@@ -430,34 +428,37 @@ func (m *MemData) setFlag(req engine.FlagRequest) (res bool, err error) {
 		info.ReadOnly = status
 		m.metaPosts[req.Locator] = info
 	}
-	return status, errors.Wrapf(err, "failed to set flag %+v", req)
+	if err != nil {
+		return false, fmt.Errorf("failed to set flag %+v: %w", req, err)
+	}
+	return status, nil
 }
 
 // getUserDetail returns UserDetailEntry with requested userDetail (omitting other details)
 // as an only element of the slice.
-func (m *MemData) getUserDetail(req engine.UserDetailRequest) ([]engine.UserDetailEntry, error) {
+func (m *MemData) getUserDetail(req engine.UserDetailRequest) []engine.UserDetailEntry {
 	if meta, ok := m.metaUsers[req.UserID]; ok {
 		if meta.SiteID != req.Locator.SiteID {
-			return []engine.UserDetailEntry{}, nil
+			return []engine.UserDetailEntry{}
 		}
 		switch req.Detail {
 		case engine.UserEmail:
-			return []engine.UserDetailEntry{{UserID: req.UserID, Email: meta.Details.Email}}, nil
+			return []engine.UserDetailEntry{{UserID: req.UserID, Email: meta.Details.Email}}
 		case engine.UserTelegram:
-			return []engine.UserDetailEntry{{UserID: req.UserID, Telegram: meta.Details.Telegram}}, nil
+			return []engine.UserDetailEntry{{UserID: req.UserID, Telegram: meta.Details.Telegram}}
 		}
 	}
 
-	return []engine.UserDetailEntry{}, nil
+	return []engine.UserDetailEntry{}
 }
 
 // setUserDetail sets requested userDetail, returning complete updated UserDetailEntry as an onlyIps
 // element of the slice in case of success
-func (m *MemData) setUserDetail(req engine.UserDetailRequest) ([]engine.UserDetailEntry, error) {
+func (m *MemData) setUserDetail(req engine.UserDetailRequest) []engine.UserDetailEntry {
 	var entry metaUser
 	if meta, ok := m.metaUsers[req.UserID]; ok {
 		if meta.SiteID != req.Locator.SiteID {
-			return []engine.UserDetailEntry{}, nil
+			return []engine.UserDetailEntry{}
 		}
 		entry = meta
 	}
@@ -474,42 +475,42 @@ func (m *MemData) setUserDetail(req engine.UserDetailRequest) ([]engine.UserDeta
 	case engine.UserEmail:
 		entry.Details.Email = req.Update
 		m.metaUsers[req.UserID] = entry
-		return []engine.UserDetailEntry{{UserID: req.UserID, Email: req.Update}}, nil
+		return []engine.UserDetailEntry{{UserID: req.UserID, Email: req.Update}}
 	case engine.UserTelegram:
 		entry.Details.Telegram = req.Update
 		m.metaUsers[req.UserID] = entry
-		return []engine.UserDetailEntry{{UserID: req.UserID, Telegram: req.Update}}, nil
+		return []engine.UserDetailEntry{{UserID: req.UserID, Telegram: req.Update}}
 	}
 
-	return []engine.UserDetailEntry{}, nil
+	return []engine.UserDetailEntry{}
 }
 
 // listDetails lists all available users details for given siteID
-func (m *MemData) listDetails(loc store.Locator) ([]engine.UserDetailEntry, error) {
+func (m *MemData) listDetails(loc store.Locator) []engine.UserDetailEntry {
 	var res []engine.UserDetailEntry
 	for _, u := range m.metaUsers {
 		if u.SiteID == loc.SiteID {
 			res = append(res, u.Details)
 		}
 	}
-	return res, nil
+	return res
 }
 
 // deleteUserDetail deletes requested UserDetail or whole UserDetailEntry,
 // deletion of the absent entry doesn't produce error.
 // Trying to delete user with wrong siteID doesn't to anything and doesn't produce error.
-func (m *MemData) deleteUserDetail(locator store.Locator, userID string, userDetail engine.UserDetail) error {
+func (m *MemData) deleteUserDetail(locator store.Locator, userID string, userDetail engine.UserDetail) {
 	var entry metaUser
 	if meta, ok := m.metaUsers[userID]; ok {
 		if meta.SiteID != locator.SiteID {
-			return nil
+			return
 		}
 		entry = meta
 	}
 
 	if entry == (metaUser{}) || entry.Details == (engine.UserDetailEntry{}) {
 		// absent entry means that we should not do anything
-		return nil
+		return
 	}
 
 	switch userDetail {
@@ -527,7 +528,6 @@ func (m *MemData) deleteUserDetail(locator store.Locator, userID string, userDet
 	}
 
 	m.metaUsers[userID] = entry
-	return nil
 }
 
 func (m *MemData) get(loc store.Locator, commentID string) (store.Comment, error) {
@@ -535,7 +535,7 @@ func (m *MemData) get(loc store.Locator, commentID string) (store.Comment, error
 		return c.Locator == loc && c.ID == commentID
 	})
 	if len(comments) == 0 {
-		return store.Comment{}, errors.New("not found")
+		return store.Comment{}, fmt.Errorf("not found")
 	}
 	return comments[0], nil
 }
@@ -557,7 +557,7 @@ func (m *MemData) updateComment(comment store.Comment) error {
 		m.posts[comment.Locator.SiteID] = comments
 		return nil
 	}
-	return errors.New("not found")
+	return fmt.Errorf("not found")
 }
 
 func (m *MemData) match(comments []store.Comment, fn func(c store.Comment) bool) (res []store.Comment) {

backend/_example/memory_store/accessor/data_test.go

@@ -10,6 +10,7 @@ import (
 	"fmt"
 	"sort"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	"github.com/stretchr/testify/assert"
@@ -198,7 +199,7 @@ func TestMemData_FindForUserPagination(t *testing.T) {
 	}
 
 	// write 200 comments
-	for i := 0; i < 200; i++ {
+	for i := range 200 {
 		c.ID = fmt.Sprintf("idd-%d", i)
 		c.Text = fmt.Sprintf("text #%d", i)
 		c.Timestamp = time.Date(2017, 12, 20, 15, 18, i, 0, time.Local)
@@ -286,7 +287,7 @@ func TestMemData_CountUser(t *testing.T) {
 
 func TestMemData_InfoPost(t *testing.T) {
 	b := prepMem(t)
-	ts := func(min int) time.Time { return time.Date(2017, 12, 20, 15, 18, min, 0, time.Local).In(time.UTC) }
+	ts := func(minute int) time.Time { return time.Date(2017, 12, 20, 15, 18, minute, 0, time.Local).In(time.UTC) }
 
 	// add one more for https://radio-t.com/2
 	comment := store.Comment{
@@ -484,7 +485,7 @@ func TestMemData_FlagVerified(t *testing.T) {
 func TestMemData_FlagListVerified(t *testing.T) {
 
 	b := prepMem(t)
-	toIDs := func(inp []interface{}) (res []string) {
+	toIDs := func(inp []any) (res []string) {
 		res = make([]string, len(inp))
 		for i, v := range inp {
 			vv, ok := v.(string)
@@ -521,51 +522,52 @@ func TestMemData_FlagListVerified(t *testing.T) {
 }
 
 func TestMemData_FlagListBlocked(t *testing.T) {
-
+	synctest.Test(t, func(t *testing.T) {
-	b := prepMem(t)
+		b := prepMem(t)
-	setBlocked := func(site, user string, status engine.FlagStatus, ttl time.Duration) error {
+		setBlocked := func(site, user string, status engine.FlagStatus, ttl time.Duration) error {
-		req := engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: site}, UserID: user, Update: status,
+			req := engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: site}, UserID: user, Update: status,
-			TTL: ttl}
+				TTL: ttl}
-		_, err := b.Flag(req)
+			_, err := b.Flag(req)
-		return err
+			return err
-	}
-
-	toBlocked := func(inp []interface{}) (res []store.BlockedUser) {
-		res = make([]store.BlockedUser, len(inp))
-		for i, v := range inp {
-			vv, ok := v.(store.BlockedUser)
-			require.True(t, ok)
-			res[i] = vv
 		}
-		return res
-	}
-	assert.NoError(t, setBlocked("radio-t", "user1", engine.FlagTrue, 0))
-	assert.NoError(t, setBlocked("radio-t", "user2", engine.FlagTrue, 50*time.Millisecond))
-	assert.NoError(t, setBlocked("radio-t", "user3", engine.FlagFalse, 0))
 
-	vv, err := b.ListFlags(engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: "radio-t"}})
+		toBlocked := func(inp []any) (res []store.BlockedUser) {
-	assert.NoError(t, err)
+			res = make([]store.BlockedUser, len(inp))
+			for i, v := range inp {
+				vv, ok := v.(store.BlockedUser)
+				require.True(t, ok)
+				res[i] = vv
+			}
+			return res
+		}
+		assert.NoError(t, setBlocked("radio-t", "user1", engine.FlagTrue, 0))
+		assert.NoError(t, setBlocked("radio-t", "user2", engine.FlagTrue, 50*time.Millisecond))
+		assert.NoError(t, setBlocked("radio-t", "user3", engine.FlagFalse, 0))
 
-	blockedList := toBlocked(vv)
+		vv, err := b.ListFlags(engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: "radio-t"}})
-	var blockedIds = make([]string, len(blockedList))
+		assert.NoError(t, err)
-	for i, x := range blockedList {
-		blockedIds[i] = x.ID
-	}
-	require.Equal(t, 2, len(blockedList), b.metaUsers)
-	assert.ElementsMatch(t, []string{"user1", "user2"}, blockedIds)
-	t.Logf("%+v", blockedList)
 
-	// check block expiration
+		blockedList := toBlocked(vv)
-	time.Sleep(50 * time.Millisecond)
+		var blockedIDs = make([]string, len(blockedList))
-	vv, err = b.ListFlags(engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: "radio-t"}})
+		for i, x := range blockedList {
-	assert.NoError(t, err)
+			blockedIDs[i] = x.ID
-	blockedList = toBlocked(vv)
+		}
-	require.Equal(t, 1, len(blockedList))
+		require.Equal(t, 2, len(blockedList), b.metaUsers)
-	assert.Equal(t, "user1", blockedList[0].ID)
+		assert.ElementsMatch(t, []string{"user1", "user2"}, blockedIDs)
+		t.Logf("%+v", blockedList)
 
-	vv, err = b.ListFlags(engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: "bad"}})
+		// check block expiration
-	assert.NoError(t, err)
+		time.Sleep(50 * time.Millisecond)
-	assert.Equal(t, 0, len(vv))
+		vv, err = b.ListFlags(engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: "radio-t"}})
+		assert.NoError(t, err)
+		blockedList = toBlocked(vv)
+		require.Equal(t, 1, len(blockedList))
+		assert.Equal(t, "user1", blockedList[0].ID)
+
+		vv, err = b.ListFlags(engine.FlagRequest{Flag: engine.Blocked, Locator: store.Locator{SiteID: "bad"}})
+		assert.NoError(t, err)
+		assert.Equal(t, 0, len(vv))
+	})
 }
 
 func TestMemData_DeleteComment(t *testing.T) {
@@ -624,6 +626,7 @@ func TestMemData_DeleteComment(t *testing.T) {
 func TestMemData_Close(t *testing.T) {
 	b := prepMem(t)
 	assert.NoError(t, b.Close())
+	assert.NoError(t, b.Close(), "second call should not result in panic or errors")
 }
 
 func TestMemData_DeleteHard(t *testing.T) {

backend/_example/memory_store/accessor/image.go

@@ -8,11 +8,11 @@ package accessor
 
 import (
 	"context"
+	"fmt"
 	"sync"
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store/image"
 )
@@ -53,7 +53,7 @@ func (m *MemImage) ResetCleanupTimer(id string) error {
 		m.insertTime[id] = time.Now()
 		return nil
 	}
-	return errors.Errorf("image %s not found", id)
+	return fmt.Errorf("image %s not found", id)
 }
 
 // Load image by ID
@@ -65,18 +65,29 @@ func (m *MemImage) Load(id string) ([]byte, error) {
 	}
 	m.mu.RUnlock()
 	if !ok {
-		return nil, errors.Errorf("image %s not found", id)
+		return nil, fmt.Errorf("image %s not found", id)
 	}
 	return img, nil
 }
 
+// Delete image by ID
+func (m *MemImage) Delete(id string) error {
+	m.mu.Lock()
+	// delete key from permanent and staging storage
+	delete(m.images, id)
+	delete(m.insertTime, id)
+	delete(m.imagesStaging, id)
+	m.mu.Unlock()
+	return nil
+}
+
 // Commit moves image from staging to permanent
 func (m *MemImage) Commit(id string) error {
 	m.mu.RLock()
 	img, ok := m.imagesStaging[id]
 	m.mu.RUnlock()
 	if !ok {
-		return errors.Errorf("failed to commit %s, not found in staging", id)
+		return fmt.Errorf("failed to commit %s, not found in staging", id)
 	}
 
 	m.mu.Lock()

backend/_example/memory_store/accessor/image_test.go

@@ -18,7 +18,7 @@ import (
 )
 
 // gopher png for test, from https://golang.org/src/image/png/example_test.go
-const gopher = "iVBORw0KGgoAAAANSUhEUgAAAEsAAAA8CAAAAAALAhhPAAAFfUlEQVRYw62XeWwUVRzHf2" +
+const rawGopher = "iVBORw0KGgoAAAANSUhEUgAAAEsAAAA8CAAAAAALAhhPAAAFfUlEQVRYw62XeWwUVRzHf2" +
 	"+OPbo9d7tsWyiyaZti6eWGAhISoIGKECEKCAiJJkYTiUgTMYSIosYYBBIUIxoSPIINEBDi2VhwkQrVsj1ESgu9doHWdrul7ba" +
 	"73WNm3vOPtsseM9MdwvvrzTs+8/t95ze/33sI5BqiabU6m9En8oNjduLnAEDLUsQXFF8tQ5oxK3vmnNmDSMtrncks9Hhtt" +
 	"/qeWZapHb1ha3UqYSWVl2ZmpWgaXMXGohQAvmeop3bjTRtv6SgaK/Pb9/bFzUrYslbFAmHPp+3WhAYdr+7GN/YnpN46Opv55VDs" +
@@ -38,7 +38,9 @@ const gopher = "iVBORw0KGgoAAAANSUhEUgAAAEsAAAA8CAAAAAALAhhPAAAFfUlEQVRYw62XeWwU
 	"1y98c3D27eppUjsZ6fql3jcd5rUe7+ZIlLNQny3Rd+E5Tct3WVhTM5RBCEdiEK0b6B+/ca2gYU393nFj/n1AygRQxPIUA043M42u85+z2S" +
 	"nssKrPl8Mx76NL3E6eXc3be7OD+H4WHbJkKI8AU8irbITQjZ+0hQcPEgId/Fn/pl9crKH02+5o2b9T/eMx7pKoskYgAAAABJRU5ErkJggg=="
 
-func gopherPNG() io.Reader { return base64.NewDecoder(base64.StdEncoding, strings.NewReader(gopher)) }
+func gopherPNG() io.Reader {
+	return base64.NewDecoder(base64.StdEncoding, strings.NewReader(rawGopher))
+}
 
 func TestMemImage_LoadAfterSave(t *testing.T) {
 	svc := NewMemImageStore()
@@ -57,7 +59,8 @@ func TestMemImage_LoadAfterSave(t *testing.T) {
 	assert.NoError(t, err)
 	assert.Equal(t, gopher, img)
 
-	svc.ResetCleanupTimer(id)
+	err = svc.ResetCleanupTimer(id)
+	assert.NoError(t, err)
 
 	err = svc.Commit(id)
 	assert.NoError(t, err)
@@ -70,6 +73,26 @@ func TestMemImage_LoadAfterSave(t *testing.T) {
 	assert.Equal(t, gopher, img)
 }
 
+func TestMemImage_LoadAfterDelete(t *testing.T) {
+	svc := NewMemImageStore()
+	gopher, err := io.ReadAll(gopherPNG())
+	assert.NoError(t, err)
+
+	id := "test_img"
+	err = svc.Save(id, gopher)
+	assert.NoError(t, err)
+
+	err = svc.Delete(id)
+	assert.NoError(t, err)
+
+	img, err := svc.Load(id)
+	assert.EqualError(t, err, "image test_img not found")
+	assert.Empty(t, img)
+
+	err = svc.ResetCleanupTimer(id)
+	assert.EqualError(t, err, "image test_img not found")
+}
+
 func TestMemImage_CommitFail(t *testing.T) {
 	svc := NewMemImageStore()
 	err := svc.Commit("test_id")

backend/_example/memory_store/compose-dev-memstore.yml

@@ -11,7 +11,7 @@ services:
       args:
         - SKIP_BACKEND_TEST=true
         - SKIP_FRONTEND_TEST=true
-    image: umputun/remark42:dev
+    image: ghcr.io/umputun/remark42:dev
     container_name: "remark42-dev"
     hostname: "remark42-dev"
     restart: always
@@ -29,7 +29,6 @@ services:
     environment:
       - REMARK_URL=http://127.0.0.1:8080
       - SECRET=123456
-      - BACKUP_PATH=/srv/var/backup
       - DEBUG=true
       - EMOJI=true
       - AUTH_ANON=true

backend/_example/memory_store/go.mod

@@ -1,14 +1,40 @@
 module github.com/umputun/remark42/memory_store
 
-go 1.16
+go 1.25.0
 
 require (
-	github.com/go-pkgz/jrpc v0.2.0
+	github.com/go-pkgz/jrpc v0.4.0
-	github.com/go-pkgz/lgr v0.10.4
+	github.com/go-pkgz/lgr v0.12.3
-	github.com/pkg/errors v0.9.1
+	github.com/jessevdk/go-flags v1.6.1
-	github.com/stretchr/testify v1.7.0
+	github.com/stretchr/testify v1.11.1
-	github.com/umputun/go-flags v1.5.1
+	github.com/umputun/remark42/backend v1.1000.0
-	github.com/umputun/remark42/backend v1.8.1
 )
 
-replace github.com/umputun/remark42/backend => ../../
+require (
+	github.com/Depado/bfchroma/v2 v2.0.0 // indirect
+	github.com/PuerkitoBio/goquery v1.12.0 // indirect
+	github.com/alecthomas/chroma/v2 v2.24.1 // indirect
+	github.com/andybalholm/cascadia v1.3.3 // indirect
+	github.com/aymerick/douceur v0.2.0 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/dlclark/regexp2 v1.12.0 // indirect
+	github.com/go-pkgz/rest v1.21.0 // indirect
+	github.com/go-pkgz/routegroup v1.6.0 // indirect
+	github.com/gorilla/css v1.0.1 // indirect
+	github.com/hashicorp/errwrap v1.1.0 // indirect
+	github.com/hashicorp/go-multierror v1.1.1 // indirect
+	github.com/kr/text v0.2.0 // indirect
+	github.com/microcosm-cc/bluemonday v1.0.27 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/rogpeppe/go-internal v1.14.1 // indirect
+	github.com/rs/xid v1.6.0 // indirect
+	github.com/russross/blackfriday/v2 v2.1.0 // indirect
+	go.etcd.io/bbolt v1.4.3 // indirect
+	golang.org/x/crypto v0.51.0 // indirect
+	golang.org/x/image v0.40.0 // indirect
+	golang.org/x/net v0.54.0 // indirect
+	golang.org/x/sys v0.44.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)
+
+replace github.com/umputun/remark42/backend v1.1000.0 => ../../

backend/_example/memory_store/go.sum

@@ -1,786 +1,136 @@
-cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+github.com/Depado/bfchroma/v2 v2.0.0 h1:IRpN9BPkNwEpR6w1ectIcNWOuhDSLx+8f1pn83fzxx8=
-cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+github.com/Depado/bfchroma/v2 v2.0.0/go.mod h1:wFwW/Pw8Tnd0irzgO9Zxtxgzp3aPS8qBWlyadxujxmw=
-cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
+github.com/PuerkitoBio/goquery v1.12.0 h1:pAcL4g3WRXekcB9AU/y1mbKez2dbY2AajVhtkO8RIBo=
-cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
+github.com/PuerkitoBio/goquery v1.12.0/go.mod h1:802ej+gV2y7bbIhOIoPY5sT183ZW0YFofScC4q/hIpQ=
-cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
+github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
-cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
+github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
-cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
+github.com/alecthomas/chroma/v2 v2.24.1 h1:m5ffpfZbIb++k8AqFEKy9uVgY12xIQtBsQlc6DfZJQM=
-cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
+github.com/alecthomas/chroma/v2 v2.24.1/go.mod h1:l+ohZ9xRXIbGe7cIW+YZgOGbvuVLjMps/FYN/CwuabI=
-cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
+github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
-cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
+github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
-cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
+github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
-cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
+github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
-cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
-cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
-cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
-cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
-cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
-cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=
-cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=
-cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=
-cloud.google.com/go v0.83.0/go.mod h1:Z7MJUsANfY0pYPdw0lbnivPx4/vhy/e2FEkSkF7vAVY=
-cloud.google.com/go v0.84.0/go.mod h1:RazrYuxIK6Kb7YrzzhPoLmCVzl7Sup4NrbKPg8KHSUM=
-cloud.google.com/go v0.87.0/go.mod h1:TpDYlFy7vuLzZMMZ+B6iRiELaY7z/gJPaqbMx6mlWcY=
-cloud.google.com/go v0.90.0/go.mod h1:kRX0mNRHe0e2rC6oNakvwQqzyDmg57xJ+SZU1eT2aDQ=
-cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI=
-cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4=
-cloud.google.com/go v0.97.0/go.mod h1:GF7l59pYBVlXQIBLx3a761cZ41F9bBH3JUlihCt2Udc=
-cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
-cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
-cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
-cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
-cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
-cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
-cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
-cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
-cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
-cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
-cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
-cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
-cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
-cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
-cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
-cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
-cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
-dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
-github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
-github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
-github.com/Depado/bfchroma v1.3.0 h1:zz14vpvySU6S0CL6yGPr1vkFevQecIt8dJdCsMS2JpM=
-github.com/Depado/bfchroma v1.3.0/go.mod h1:c0bFk0tFmT+clD3TIGurjWCfD/QV8/EebfM3JGr+98M=
-github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
-github.com/PuerkitoBio/goquery v1.8.0 h1:PJTF7AmFCFKk1N6V6jmKfrNH9tV5pNE6lZMkG0gta/U=
-github.com/PuerkitoBio/goquery v1.8.0/go.mod h1:ypIiRMtY7COPGk+I/YbZLbxsxn9g5ejnI2HSMtkjZvI=
-github.com/ajg/form v1.5.1/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY=
-github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38/go.mod h1:r7bzyVFMNntcxPZXK3/+KdruV1H5KSlyVY0gc+NgInI=
-github.com/alecthomas/chroma v0.7.3/go.mod h1:sko8vR34/90zvl5QdcUdvzL3J8NKjAUx9va9jPuFNoM=
-github.com/alecthomas/chroma v0.9.4 h1:YL7sOAE3p8HS96T9km7RgvmsZIctqbK1qJ0b7hzed44=
-github.com/alecthomas/chroma v0.9.4/go.mod h1:jtJATyUxlIORhUOFNA9NZDWGAQ8wpxQQqNSB4rjA/1s=
-github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721/go.mod h1:QO9JBoKquHd+jz9nshCh40fOfO+JzsoXy8qTHF68zU0=
-github.com/alecthomas/kong v0.2.4/go.mod h1:kQOmtJgV+Lb4aj+I2LEn40cbtawdWJ9Y8QLq+lElKxE=
-github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
-github.com/alecthomas/repr v0.0.0-20200325044227-4184120f674c/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
-github.com/alicebob/gopher-json v0.0.0-20180125190556-5a6b3ba71ee6/go.mod h1:SGnFV6hVsYE877CKEZ6tDNTjaSXYUk6QqoIK6PrAtcc=
-github.com/alicebob/miniredis/v2 v2.11.4/go.mod h1:VL3UDEfAH59bSa7MuHMuFToxkqyHh69s/WUbYlOAuyg=
-github.com/andybalholm/cascadia v1.3.1 h1:nhxRkql1kdYCc8Snf7D5/D3spOX+dBgjA6u8x004T2c=
-github.com/andybalholm/cascadia v1.3.1/go.mod h1:R4bJ1UQfqADjvDa4P6HZHLh/3OxWWEqc0Sk8XGwHqvA=
-github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
 github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
 github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
-github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
-github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
-github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
-github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
-github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
-github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
-github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
-github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
-github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
-github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
-github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964/go.mod h1:Xd9hchkHSWYkEqJwUGisez3G1QY8Ryz0sdWrLPMGjLk=
-github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/dghubble/oauth1 v0.7.0/go.mod h1:8pFdfPkv/jr8mkChVbNVuJ0suiHe278BtWI4Tk1ujxk=
+github.com/dlclark/regexp2 v1.12.0 h1:0j4c5qQmnC6XOWNjP3PIXURXN2gWx76rd3KvgdPkCz8=
-github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
+github.com/dlclark/regexp2 v1.12.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
-github.com/didip/tollbooth/v6 v6.0.1/go.mod h1:j2pKs+JQ5PvU/K4jFnrnwntrmfUbYLJE5oSdxR37FD0=
+github.com/go-pkgz/jrpc v0.4.0 h1:oD7xiGrzDkndkuCjeHGugQXxbggLSV7O1QmHhoc5pYY=
-github.com/didip/tollbooth/v6 v6.1.1 h1:Nt7PvWLa9Y94OrykXsFNBinVRQIu8xdy4avpl99Dc1M=
+github.com/go-pkgz/jrpc v0.4.0/go.mod h1:JFoY3bRjRyx4M3CbEVDFQStMB1m2gmQ7OjqFK7q3kOo=
-github.com/didip/tollbooth/v6 v6.1.1/go.mod h1:xjcse6CTHCLuOkzsWrEgdy9WPJFv+p/x6v+MyfP+O9s=
+github.com/go-pkgz/lgr v0.12.3 h1:QDug7kRkEsuQtruT9fNF5PVT2kZUqCDPc4GmsgS3fP8=
-github.com/didip/tollbooth_chi v0.0.0-20200524181329-8b84cd7183d9/go.mod h1:YWyIfq3y4ArRfWZ9XksmuusP+7Mad+T0iFZ0kv0XG/M=
+github.com/go-pkgz/lgr v0.12.3/go.mod h1:lpCDgVvCIxBHZp8+sGCj9MPctIzKZyZ3QdE19ddqd54=
-github.com/didip/tollbooth_chi v0.0.0-20200828173446-a7173453ea21 h1:x7YpwKSBIBcKe9I3aTNOqgSyJ6QKDdtOxnEkxBTsi9w=
+github.com/go-pkgz/rest v1.21.0 h1:Y/C4d/TpclJJDxqnH1RAcS6Hmox0RIReAlkwMcUWXK4=
-github.com/didip/tollbooth_chi v0.0.0-20200828173446-a7173453ea21/go.mod h1:0ZVa6kSzS011nfTC1rELyxK4tjVf6vqBnOv7oY2KlsA=
+github.com/go-pkgz/rest v1.21.0/go.mod h1:+AHzjHazq7Z3Tk/kRWOhbbAz/YZlUV40feC1Hf4NtbE=
-github.com/dlclark/regexp2 v1.2.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
+github.com/go-pkgz/routegroup v1.6.0 h1:44XHZgF6JIIldRlv+zjg6SygULASmjifnfIQjwCT0e4=
-github.com/dlclark/regexp2 v1.4.0 h1:F1rxgk7p4uKjwIQxBs9oAXe5CqrXlCduYEJvrF4u93E=
+github.com/go-pkgz/routegroup v1.6.0/go.mod h1:Pmu04fhgWhRtBMIJ8HXppnnzOPjnL/IEPBIdO2zmeqg=
-github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
-github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
-github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
-github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
-github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
-github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
-github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
-github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ=
-github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
-github.com/fasthttp-contrib/websocket v0.0.0-20160511215533-1f3b11f56072/go.mod h1:duJ4Jxv5lDcvg4QuQr0oowTf7dz4/CR8NtyCooz9HL8=
-github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
-github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
-github.com/gavv/httpexpect v2.0.0+incompatible/go.mod h1:x+9tiU1YnrOvnB725RkpoLv1M62hOWzwo5OXotisrKc=
-github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
-github.com/go-chi/chi v4.1.1+incompatible h1:MmTgB0R8Bt/jccxp+t6S/1VGIKdJw5J74CK/c9tTfA4=
-github.com/go-chi/chi v4.1.1+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ=
-github.com/go-chi/chi/v5 v5.0.5/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
-github.com/go-chi/cors v1.2.0/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
-github.com/go-chi/render v1.0.1 h1:4/5tis2cKaNdnv9zFLfXzcquC9HbeZgCnxGnKrltBS8=
-github.com/go-chi/render v1.0.1/go.mod h1:pq4Rr7HbnsdaeHagklXub+p6Wd16Af5l9koip1OvJns=
-github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
-github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
-github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
-github.com/go-pkgz/auth v1.18.1-0.20211207184750-78e5d290333d/go.mod h1:N1VM53c22Uw0UQSYgdTOBK1D2vtPjSc/RWGf/Fhc1y8=
-github.com/go-pkgz/expirable-cache v0.0.3 h1:rTh6qNPp78z0bQE6HDhXBHUwqnV9i09Vm6dksJLXQDc=
-github.com/go-pkgz/expirable-cache v0.0.3/go.mod h1:+IauqN00R2FqNRLCLA+X5YljQJrwB179PfiAoMPlTlQ=
-github.com/go-pkgz/jrpc v0.2.0 h1:CLy/eZyekjraVrxZV18N2R1mYLMJ/nWrgdfyIOGPY/E=
-github.com/go-pkgz/jrpc v0.2.0/go.mod h1:wd8vtQ4CgtCnuqua6x2b1SKIgv0VSOh5Dn0uUITbiUE=
-github.com/go-pkgz/lcw v0.8.1/go.mod h1:Xw0/ZfApATgbjVPYRZO4XHdWyxAjErDWDWJ7TLlw1Vc=
-github.com/go-pkgz/lgr v0.10.4 h1:l7qyFjqEZgwRgaQQSEp6tve4A3OU80VrfzpvtEX8ngw=
-github.com/go-pkgz/lgr v0.10.4/go.mod h1:CD0s1z6EFpIUplV067gitF77tn25JItzwHNKAPqeCF0=
-github.com/go-pkgz/repeater v1.1.3/go.mod h1:hVTavuO5x3Gxnu8zW7d6sQBfAneKV8X2FjU48kGfpKw=
-github.com/go-pkgz/rest v1.5.0/go.mod h1:nQaM3RhSTUAmbBZWY4hfe4buyeC9VckvhoCktiQXJxI=
-github.com/go-pkgz/rest v1.11.0 h1:Z//qgmM0NhBYfhXYEP/aJtDVLK5XlJGxqcb4sHFNN0E=
-github.com/go-pkgz/rest v1.11.0/go.mod h1:wZ/dGipZUaF9to0vIQl7PwDHgWQDB0jsrFg1xnAKLDw=
-github.com/go-pkgz/syncs v1.1.1/go.mod h1:bt9lxWRRJ9vOCMGc8Big8ttjYHLKP88ofj1y38UlaHE=
-github.com/go-redis/redis/v7 v7.4.0/go.mod h1:JDNMw23GTyLNC4GZu9njt15ctBQVn7xjRfnwdHj/Dcg=
-github.com/go-session/session v3.1.2+incompatible/go.mod h1:8B3iivBQjrz/JtC68Np2T1yBBLxTan3mn/3OM0CyRt0=
-github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
-github.com/go-stack/stack v1.8.1/go.mod h1:dcoOX6HbPZSZptuspn9bctJ+N/CnF5gGygcUP3XYfe4=
-github.com/go-test/deep v1.0.4/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
-github.com/gobuffalo/attrs v0.0.0-20190224210810-a9411de4debd/go.mod h1:4duuawTqi2wkkpB4ePgWMaai6/Kc6WEz83bhFwpHzj0=
-github.com/gobuffalo/depgen v0.0.0-20190329151759-d478694a28d3/go.mod h1:3STtPUQYuzV0gBVOY3vy6CfMm/ljR4pABfrTeHNLHUY=
-github.com/gobuffalo/depgen v0.1.0/go.mod h1:+ifsuy7fhi15RWncXQQKjWS9JPkdah5sZvtHc2RXGlg=
-github.com/gobuffalo/envy v1.6.15/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI=
-github.com/gobuffalo/envy v1.7.0/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI=
-github.com/gobuffalo/flect v0.1.0/go.mod h1:d2ehjJqGOH/Kjqcoz+F7jHTBbmDb38yXA598Hb50EGs=
-github.com/gobuffalo/flect v0.1.1/go.mod h1:8JCgGVbRjJhVgD6399mQr4fx5rRfGKVzFjbj6RE/9UI=
-github.com/gobuffalo/flect v0.1.3/go.mod h1:8JCgGVbRjJhVgD6399mQr4fx5rRfGKVzFjbj6RE/9UI=
-github.com/gobuffalo/genny v0.0.0-20190329151137-27723ad26ef9/go.mod h1:rWs4Z12d1Zbf19rlsn0nurr75KqhYp52EAGGxTbBhNk=
-github.com/gobuffalo/genny v0.0.0-20190403191548-3ca520ef0d9e/go.mod h1:80lIj3kVJWwOrXWWMRzzdhW3DsrdjILVil/SFKBzF28=
-github.com/gobuffalo/genny v0.1.0/go.mod h1:XidbUqzak3lHdS//TPu2OgiFB+51Ur5f7CSnXZ/JDvo=
-github.com/gobuffalo/genny v0.1.1/go.mod h1:5TExbEyY48pfunL4QSXxlDOmdsD44RRq4mVZ0Ex28Xk=
-github.com/gobuffalo/gitgen v0.0.0-20190315122116-cc086187d211/go.mod h1:vEHJk/E9DmhejeLeNt7UVvlSGv3ziL+djtTr3yyzcOw=
-github.com/gobuffalo/gogen v0.0.0-20190315121717-8f38393713f5/go.mod h1:V9QVDIxsgKNZs6L2IYiGR8datgMhB577vzTDqypH360=
-github.com/gobuffalo/gogen v0.1.0/go.mod h1:8NTelM5qd8RZ15VjQTFkAW6qOMx5wBbW4dSCS3BY8gg=
-github.com/gobuffalo/gogen v0.1.1/go.mod h1:y8iBtmHmGc4qa3urIyo1shvOD8JftTtfcKi+71xfDNE=
-github.com/gobuffalo/logger v0.0.0-20190315122211-86e12af44bc2/go.mod h1:QdxcLw541hSGtBnhUc4gaNIXRjiDppFGaDqzbrBd3v8=
-github.com/gobuffalo/mapi v1.0.1/go.mod h1:4VAGh89y6rVOvm5A8fKFxYG+wIW6LO1FMTG9hnKStFc=
-github.com/gobuffalo/mapi v1.0.2/go.mod h1:4VAGh89y6rVOvm5A8fKFxYG+wIW6LO1FMTG9hnKStFc=
-github.com/gobuffalo/packd v0.0.0-20190315124812-a385830c7fc0/go.mod h1:M2Juc+hhDXf/PnmBANFCqx4DM3wRbgDvnVWeG2RIxq4=
-github.com/gobuffalo/packd v0.1.0/go.mod h1:M2Juc+hhDXf/PnmBANFCqx4DM3wRbgDvnVWeG2RIxq4=
-github.com/gobuffalo/packr/v2 v2.0.9/go.mod h1:emmyGweYTm6Kdper+iywB6YK5YzuKchGtJQZ0Odn4pQ=
-github.com/gobuffalo/packr/v2 v2.2.0/go.mod h1:CaAwI0GPIAv+5wKLtv8Afwl+Cm78K/I/VCm/3ptBN+0=
-github.com/gobuffalo/syncx v0.0.0-20190224160051-33c29581e754/go.mod h1:HhnNqWY95UYwwW3uSASeV7vtgYkT2t16hJgV3AEPUpw=
-github.com/golang-jwt/jwt v3.2.2+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I=
-github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
-github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
-github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
-github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
-github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
-github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
-github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
-github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
-github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=
-github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs=
-github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
-github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
-github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
-github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
-github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
-github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
-github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
-github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
-github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
-github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
-github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
-github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
-github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=
-github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
-github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
-github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
-github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
-github.com/gomodule/redigo v1.7.1-0.20190322064113-39e2c31b7ca3/go.mod h1:B4C85qUVwatsJoIUNIfCRsp7qO0iAmpGFZ4EELWSbC4=
-github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
-github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
-github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
-github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
-github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
-github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
-github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
-github.com/google/martian/v3 v3.2.1/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk=
-github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
-github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
-github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
-github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
-github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
-github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
-github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0=
-github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
-github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
-github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
-github.com/gorilla/feeds v1.1.1/go.mod h1:Nk0jZrvPFZX1OBe5NPiddPw7CfwF6Q9eqzaBbaightA=
-github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
-github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
-github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
-github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA=
 github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
-github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA=
+github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
+github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
 github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
 github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
-github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
-github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
-github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
+github.com/jessevdk/go-flags v1.6.1 h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bBK4=
-github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
+github.com/jessevdk/go-flags v1.6.1/go.mod h1:Mk8T1hIAWpOiJiHa9rJASDK2UGWji0EuPGBnNLMooyc=
-github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
-github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
-github.com/imkira/go-interpol v1.1.0/go.mod h1:z0h2/2T3XF8kyEPpRgJ3kmNv+C43p+I/CoI+jC3w2iA=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
-github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
-github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=
+github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
-github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
+github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
-github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
-github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
-github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88/go.mod h1:3w7q1U84EfirKl04SVQ/s7nPm1ZPhiXd34z40TNz36k=
-github.com/karrick/godirwalk v1.8.0/go.mod h1:H5KPZjojv4lE+QYImBI8xVtrBRgYrIVsaRPx4tDPEn4=
-github.com/karrick/godirwalk v1.10.3/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA=
-github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
-github.com/klauspost/compress v1.8.2/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
-github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
-github.com/klauspost/cpuid v1.2.1/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
-github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
-github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
-github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
-github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
-github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
-github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
-github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
-github.com/kyokomi/emoji/v2 v2.2.8/go.mod h1:JUcn42DTdsXJo1SWanHh4HKDEyPaR5CqkmoirZZP9qE=
-github.com/markbates/oncer v0.0.0-20181203154359-bf2de49a0be2/go.mod h1:Ld9puTsIW75CHf65OeIOkyKbteujpZVXDpWK6YGZbxE=
-github.com/markbates/safe v1.0.1/go.mod h1:nAqgmRi7cY2nqMc92/bSEeQA+R4OheNU2T1kNSCBdG0=
-github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
-github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
-github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
-github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
-github.com/microcosm-cc/bluemonday v1.0.16 h1:kHmAq2t7WPWLjiGvzKa5o3HzSfahUKiOq7fAPUiMNIc=
-github.com/microcosm-cc/bluemonday v1.0.16/go.mod h1:Z0r70sCuXHig8YpBzCc5eGHAap2K7e/u082ZUpDRRqM=
-github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc=
-github.com/moul/http2curl v1.0.0/go.mod h1:8UbvGypXm98wA/IqH45anm5Y2Z6ep6O31QGOAZ3H0fQ=
-github.com/nullrocks/identicon v0.0.0-20180626043057-7875f45b0022/go.mod h1:x4NsS+uc7ecH/Cbm9xKQ6XzmJM57rWTkjywjfB2yQ18=
-github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
-github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
-github.com/onsi/ginkgo v1.10.2/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
-github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
-github.com/pelletier/go-toml v1.7.0/go.mod h1:vwGMzjaWMwyfHwgIBhI2YUM4fB6nL6lVAvS1LBMMhTE=
-github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
-github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
-github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
-github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
-github.com/rakyll/statik v0.1.7/go.mod h1:AlZONWzMtEnMs7W4e/1LURLiI49pIMmp6V9Unghqrcc=
+github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
-github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
+github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
-github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
+github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
-github.com/rogpeppe/go-internal v1.2.2/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
-github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
-github.com/rs/xid v1.3.0 h1:6NjYksEUlhurdVehpc7S7dk6DAmcKv8V9gG0FsVN2U4=
-github.com/rs/xid v1.3.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
-github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
-github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
+github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
-github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
+github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
-github.com/sirupsen/logrus v1.4.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
-github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
+go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
-github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
+go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
-github.com/slack-go/slack v0.9.5/go.mod h1:wWL//kk0ho+FcQXcBTmEafUI5dz4qz5f4mMk8oIkioQ=
-github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
-github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
-github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
-github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
-github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
-github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
-github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
-github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
-github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
-github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
-github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
-github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-github.com/tidwall/btree v0.0.0-20170113224114-9876f1454cf0/go.mod h1:huei1BkDWJ3/sLXmO+bsCNELL+Bp2Kks9OLyQFkzvA8=
-github.com/tidwall/buntdb v1.1.0/go.mod h1:Y39xhcDW10WlyYXeLgGftXVbjtM0QP+/kpz8xl9cbzE=
-github.com/tidwall/gjson v1.3.2/go.mod h1:P256ACg0Mn+j1RXIDXoss50DeIABTYK1PULOJHhxOls=
-github.com/tidwall/gjson v1.11.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
-github.com/tidwall/grect v0.0.0-20161006141115-ba9a043346eb/go.mod h1:lKYYLFIr9OIgdgrtgkZ9zgRxRdvPYsExnYBsEAd8W5M=
-github.com/tidwall/match v1.0.1/go.mod h1:LujAq0jyVjBy028G1WhWfIzbpQfMO8bBZ6Tyb0+pL9E=
-github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
-github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
-github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
-github.com/tidwall/rtree v0.0.0-20180113144539-6cd427091e0e/go.mod h1:/h+UnNGt0IhNNJLkGikcdcJqm66zGD/uJGMRxK/9+Ao=
-github.com/tidwall/tinyqueue v0.0.0-20180302190814-1e39f5511563/go.mod h1:mLqSmt7Dv/CNneF2wfcChfN1rvapyQr01LGKnKex0DQ=
-github.com/umputun/go-flags v1.5.1 h1:vRauoXV3Ultt1HrxivSxowbintgZLJE+EcBy5ta3/mY=
-github.com/umputun/go-flags v1.5.1/go.mod h1:nTbvsO/hKqe7Utri/NoyN18GR3+EWf+9RrmsdwdhrEc=
-github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
-github.com/valyala/fasthttp v1.6.0/go.mod h1:FstJa9V+Pj9vQ7OJie2qMHdwemEDaDiSdBnvPM1Su9w=
-github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
-github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
-github.com/xdg-go/scram v1.0.2/go.mod h1:1WAq6h33pAW+iRreB34OORO2Nf7qel3VV3fjBj+hCSs=
-github.com/xdg-go/stringprep v1.0.2/go.mod h1:8F9zXuvzgwmyT5DUm4GUfZGDdT3W+LCvS6+da4O5kxM=
-github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
-github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
-github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
-github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0/go.mod h1:/LWChgwKmvncFJFHJ7Gvn9wZArjbV5/FppcK2fKk/tI=
-github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA=
-github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a/go.mod h1:ul22v+Nro/R083muKhosV54bj5niojjWZvU8xrevuH4=
-github.com/yudai/gojsondiff v1.0.0/go.mod h1:AY32+k2cwILAkW1fbgxQ5mUmMiZFgLIV+FBNExI05xg=
-github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82/go.mod h1:lgjkn3NuSvDfVJdfcVVdX+jpBxNmX4rDAzaS45IcYoM=
-github.com/yudai/pp v2.0.1+incompatible/go.mod h1:PuxR/8QJ7cyCkFp/aUDS+JY727OFEZkTdatxwunjIkc=
-github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
-github.com/yuin/gopher-lua v0.0.0-20191220021717-ab39c6098bdb/go.mod h1:gqRgreBUhTSL0GeU64rtZ3Uq3wtjOa/TB2YfrtkCbVQ=
-go.etcd.io/bbolt v1.3.6 h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU=
-go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4=
-go.mongodb.org/mongo-driver v1.7.4/go.mod h1:NqaYOwnXWr5Pm7AOpO5QFxKJ503nbMse/R79oO62zWg=
-go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
-go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
-go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
-go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
-go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
-go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
-go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
-go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
-go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ=
-golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/crypto v0.0.0-20190422162423-af44ce270edf/go.mod h1:WFFai1msRO1wXaEeE5yQxYXgSfI8pQAWXbQop6sCtWE=
-golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
-golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
-golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
-golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
-golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
+golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
-golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
+golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
-golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
+golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
-golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
-golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/image v0.40.0 h1:Tw4GyDXMo+daZN1znreBRC3VayR1aLFUyUEOLUdW1a8=
-golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/image v0.40.0/go.mod h1:uIc348UZMSvS5Z65CVZ7iDPaNobNFEPeJ4kbqTOszmA=
-golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
-golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
+golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
-golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
+golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
-golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
+golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
-golang.org/x/image v0.0.0-20211028202545-6944b10bf410 h1:hTftEOvwiOq2+O8k2D5/Q7COC7k5Qcrgc2TFURJYnvQ=
+golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
-golang.org/x/image v0.0.0-20211028202545-6944b10bf410/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM=
-golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
-golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
-golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
-golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
-golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
-golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
-golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
-golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
-golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
-golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
-golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
-golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
-golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
-golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
+golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
-golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
-golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
-golang.org/x/net v0.0.0-20210916014120-12bc252f5db8/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
-golang.org/x/net v0.0.0-20211105192438-b53810dc28af h1:SMeNJG/vclJ5wyBBd4xupMsSJIHTd1coW9g7q6KOjmY=
+golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
-golang.org/x/net v0.0.0-20211105192438-b53810dc28af/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
-golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w=
-golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ=
-golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190412183630-56d357773e84/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
-golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
-golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
-golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
-golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
-golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
-golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190419153524-e8e3143a4f4a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190531175056-4c3a928424d2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200413165638-669c56c373c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20211105183446-c75c47738b0c h1:+8miTPjMCTXwih7BQmvWwd0PjdBZq2MKp/qQaahSzEM=
+golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
-golang.org/x/sys v0.0.0-20211105183446-c75c47738b0c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
+golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
+golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
+golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
+golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
+golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
-golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
-golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
-golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1 h1:NusfzzA6yGQ+ua51ck7E3omNUX/JuqbFSaRGqU8CcLI=
+golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
-golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
+golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
-golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190329151228-23e29df326fe/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190416151739-9c9e1878f421/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190420181800-aa740d480789/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20190531172133-b3315ee88b7d/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
-golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
-golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
-golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
-golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
-golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
-golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
-golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
-golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
-golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
-golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
-golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
-golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
-golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
-golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
-golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
-golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
-google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
-google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
-google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
-google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
-google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
-google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
-google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
-google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
-google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
-google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
-google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
-google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
-google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
-google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=
-google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=
-google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo=
-google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtukyy4=
-google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw=
-google.golang.org/api v0.51.0/go.mod h1:t4HdrdoNgyN5cbEfm7Lum0lcLDLiise1F8qDKX00sOU=
-google.golang.org/api v0.54.0/go.mod h1:7C4bFFOvVDGXjfDTAsgGwDgAxRDeQ4X8NvUedIt6z3k=
-google.golang.org/api v0.55.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE=
-google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdrMgI=
-google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
-google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
-google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
-google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
-google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
-google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
-google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
-google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
-google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
-google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
-google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
-google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=
-google.golang.org/genproto v0.0.0-20210513213006-bf773b8c8384/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A=
-google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=
-google.golang.org/genproto v0.0.0-20210604141403-392c879c8b08/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=
-google.golang.org/genproto v0.0.0-20210608205507-b6d2f5bf0d7d/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=
-google.golang.org/genproto v0.0.0-20210624195500-8bfb893ecb84/go.mod h1:SzzZ/N+nwJDaO1kznhnlzqS8ocJICar6hYhVyhi++24=
-google.golang.org/genproto v0.0.0-20210713002101-d411969a0d9a/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k=
-google.golang.org/genproto v0.0.0-20210716133855-ce7ef5c701ea/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k=
-google.golang.org/genproto v0.0.0-20210728212813-7823e685a01f/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48=
-google.golang.org/genproto v0.0.0-20210805201207-89edb61ffb67/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48=
-google.golang.org/genproto v0.0.0-20210813162853-db860fec028c/go.mod h1:cFeNkxwySK631ADgubI+/XFU/xp8FD5KIVV4rj8UC5w=
-google.golang.org/genproto v0.0.0-20210821163610-241b8fcbd6c8/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
-google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
-google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
-google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY=
-google.golang.org/genproto v0.0.0-20210924002016-3dee208752a0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc=
-google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
-google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
-google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
-google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
-google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
-google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
-google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
-google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
-google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
-google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
-google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
-google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
-google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
-google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
-google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
-google.golang.org/grpc v1.37.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=
-google.golang.org/grpc v1.37.1/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=
-google.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=
-google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE=
-google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE=
-google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34=
-google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw=
-google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
-google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
-google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
-google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
-google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
-google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
-google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
-google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
-google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
-google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
-google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
-google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
-google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
-gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
-gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
-gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
-gopkg.in/oauth2.v3 v3.12.0/go.mod h1:XEYgKqWX095YiPT+Aw5y3tCn+7/FMnlTFKrupgSiJ3I=
-gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
-gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
-gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
-honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
-honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
-honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
-honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
-honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
-rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
-rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
-rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
-rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
-rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
-rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=

backend/_example/memory_store/main.go

@@ -12,7 +12,7 @@ import (
 
 	"github.com/go-pkgz/jrpc"
 	log "github.com/go-pkgz/lgr"
-	"github.com/umputun/go-flags"
+	"github.com/jessevdk/go-flags"
 
 	"github.com/umputun/remark42/memory_store/accessor"
 	"github.com/umputun/remark42/memory_store/server"
@@ -43,16 +43,14 @@ func main() {
 	adminStore := accessor.NewMemAdminStore(opts.Secret)
 	imgStore := accessor.NewMemImageStore()
 
-	rpcServer := jrpc.Server{
+	rpcServer := jrpc.NewServer(
-		API:        opts.API,
+		opts.API,
-		AuthUser:   opts.AuthUser,
+		jrpc.Auth(opts.AuthUser, opts.AuthPasswd),
-		AuthPasswd: opts.AuthPasswd,
+		jrpc.WithSignature("remark42-memory", "umputun", revision),
-		Version:    revision,
+		jrpc.WithLogger(log.Default()),
-		AppName:    "remark42-memory",
+	)
-		Logger:     log.Default(),
-	}
 
-	srv := server.NewRPC(dataStore, adminStore, imgStore, &rpcServer)
+	srv := server.NewRPC(dataStore, adminStore, imgStore, rpcServer)
 
 	admRec := accessor.AdminRec{
 		SiteID:  "remark",

backend/_example/memory_store/server/admin.go

@@ -73,7 +73,7 @@ func (s *RPC) admEnabledHndl(id uint64, params json.RawMessage) (rr jrpc.Respons
 // onEvent returns nothing, callback to OnEvent
 func (s *RPC) admEventHndl(id uint64, params json.RawMessage) (rr jrpc.Response) {
 	var siteID string
-	var ps []interface{}
+	var ps []any
 	if err := json.Unmarshal(params, &ps); err != nil {
 		return jrpc.Response{Error: err.Error()}
 	}

backend/_example/memory_store/server/data_test.go

@@ -198,25 +198,62 @@ func TestRPC_listFlagsHndl(t *testing.T) {
 	assert.NoError(t, err)
 	assert.Equal(t, "123456", id)
 
-	flagReq := engine.FlagRequest{
+	// verify user
+	verifyFlagReq := engine.FlagRequest{
 		Flag:   engine.Verified,
 		UserID: "u1",
 		Locator: store.Locator{
 			SiteID: "test-site",
 		},
 	}
-	flags, err := re.ListFlags(flagReq)
+	flags, err := re.ListFlags(verifyFlagReq)
 	require.NoError(t, err)
-	assert.Equal(t, []interface{}{}, flags)
+	assert.Empty(t, flags)
 
-	flagReq.Update = engine.FlagTrue
+	verifyFlagReq.Update = engine.FlagTrue
-	status, err := re.Flag(flagReq)
+	status, err := re.Flag(verifyFlagReq)
 	require.NoError(t, err)
 	assert.Equal(t, true, status)
 
-	flags, err = re.ListFlags(flagReq)
+	flags, err = re.ListFlags(verifyFlagReq)
 	require.NoError(t, err)
-	assert.Equal(t, []interface{}{"u1"}, flags)
+	assert.Equal(t, []any{"u1"}, flags)
+	verifiedUsers := make([]string, 0, len(flags))
+	for _, v := range flags {
+		verifiedUsers = append(verifiedUsers, v.(string))
+	}
+	assert.Equal(t, []string{"u1"}, verifiedUsers)
+
+	// block user
+	blockFlagReq := engine.FlagRequest{
+		Flag:   engine.Blocked,
+		UserID: "u1",
+		Locator: store.Locator{
+			SiteID: "test-site",
+		},
+		TTL: time.Hour,
+	}
+	flags, err = re.ListFlags(blockFlagReq)
+	require.NoError(t, err)
+	assert.Empty(t, flags)
+
+	blockFlagReq.Update = engine.FlagTrue
+	status, err = re.Flag(blockFlagReq)
+	require.NoError(t, err)
+	assert.Equal(t, true, status)
+
+	flags, err = re.ListFlags(blockFlagReq)
+	require.NoError(t, err)
+	assert.NotEmpty(t, flags)
+	blockedUsers := make([]store.BlockedUser, 0, len(flags))
+	for _, v := range flags {
+		blockedUsers = append(blockedUsers, v.(store.BlockedUser))
+	}
+	require.Equal(t, 1, len(blockedUsers))
+	blockedUserInfo := blockedUsers[0]
+	assert.Equal(t, "u1", blockedUserInfo.ID)
+	assert.True(t, blockedUserInfo.Until.After(time.Now().Add(time.Minute*59)), "blocked duration is more than 59m away")
+	assert.True(t, blockedUserInfo.Until.Before(time.Now().Add(time.Minute*61)), "blocked duration is less than 61m away")
 }
 
 func TestRPC_userDetailHndl(t *testing.T) {
@@ -301,6 +338,6 @@ func TestRPC_closeHndl(t *testing.T) {
 	api := fmt.Sprintf("http://localhost:%d/test", port)
 
 	re := engine.RPC{Client: jrpc.Client{API: api, Client: http.Client{Timeout: 1 * time.Second}}}
-	err := re.Close()
+	assert.NoError(t, re.Close())
-	assert.NoError(t, err)
+	assert.NoError(t, re.Close(), "second call should not result in panic or errors")
 }

backend/_example/memory_store/server/image.go

@@ -35,7 +35,6 @@ func (s *RPC) imgResetClnTimerHndl(id uint64, params json.RawMessage) (rr jrpc.R
 	}
 	err := s.img.ResetCleanupTimer(fileID)
 	return jrpc.EncodeResponse(id, nil, err)
-
 }
 
 func (s *RPC) imgLoadHndl(id uint64, params json.RawMessage) (rr jrpc.Response) {
@@ -47,6 +46,16 @@ func (s *RPC) imgLoadHndl(id uint64, params json.RawMessage) (rr jrpc.Response)
 	return jrpc.EncodeResponse(id, value, err)
 }
 
+func (s *RPC) imgDeleteHndl(id uint64, params json.RawMessage) (rr jrpc.Response) {
+	var fileID string
+	if err := json.Unmarshal(params, &fileID); err != nil {
+		return jrpc.Response{Error: err.Error()}
+	}
+	err := s.img.Delete(fileID)
+	return jrpc.EncodeResponse(id, nil, err)
+
+}
+
 func (s *RPC) imgCommitHndl(id uint64, params json.RawMessage) (rr jrpc.Response) {
 	var fileID string
 	if err := json.Unmarshal(params, &fileID); err != nil {

backend/_example/memory_store/server/image_test.go

@@ -158,4 +158,9 @@ func TestRPC_imgInfoHndl(t *testing.T) {
 	info, err = ri.Info()
 	assert.NoError(t, err)
 	assert.False(t, info.FirstStagingImageTS.IsZero())
+
+	err = ri.Delete("test_img")
+	assert.NoError(t, err)
+	_, err = ri.Load("test_img")
+	assert.EqualError(t, err, "image test_img not found")
 }

backend/_example/memory_store/server/rpc.go

@@ -60,6 +60,7 @@ func (s *RPC) addHandlers() {
 		"save_with_id":        s.imgSaveWithIDHndl,
 		"reset_cleanup_timer": s.imgResetClnTimerHndl,
 		"load":                s.imgLoadHndl,
+		"delete":              s.imgDeleteHndl,
 		"commit":              s.imgCommitHndl,
 		"cleanup":             s.imgCleanupHndl,
 		"info":                s.imgInfoHndl,

backend/_example/memory_store/server/rpc_test.go

@@ -21,7 +21,7 @@ import (
 )
 
 func chooseRandomUnusedPort() (port int) {
-	for i := 0; i < 10; i++ {
+	for range 10 {
 		port = 40000 + int(rand.Int31n(10000))
 		if ln, err := net.Listen("tcp", fmt.Sprintf(":%d", port)); err == nil {
 			_ = ln.Close()
@@ -34,7 +34,7 @@ func chooseRandomUnusedPort() (port int) {
 func waitForHTTPServerStart(port int) {
 	// wait for up to 3 seconds for server to start before returning it
 	client := http.Client{Timeout: time.Second}
-	for i := 0; i < 300; i++ {
+	for range 300 {
 		time.Sleep(time.Millisecond * 10)
 		if resp, err := client.Get(fmt.Sprintf("http://localhost:%d", port)); err == nil {
 			_ = resp.Body.Close()
@@ -47,7 +47,7 @@ func prepTestStore(t *testing.T) (port int, teardown func()) {
 	mg := accessor.NewMemData()
 	adm := accessor.NewMemAdminStore("secret")
 	img := accessor.NewMemImageStore()
-	s := NewRPC(mg, adm, img, &jrpc.Server{API: "/test", Logger: jrpc.NoOpLogger})
+	s := NewRPC(mg, adm, img, jrpc.NewServer("/test"))
 
 	admRec := accessor.AdminRec{
 		SiteID:  "test-site",

backend/app/cmd/avatar.go

@@ -1,13 +1,13 @@
 package cmd
 
 import (
+	"fmt"
 	"path"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 	bolt "go.etcd.io/bbolt"
 
-	"github.com/go-pkgz/auth/avatar"
+	"github.com/go-pkgz/auth/v2/avatar"
 )
 
 // AvatarCommand set of flags and command for avatar migration
@@ -39,12 +39,12 @@ func (ac *AvatarCommand) Execute(_ []string) error {
 
 	src, err := ac.makeAvatarStore(ac.AvatarSrc)
 	if err != nil {
-		return errors.Wrapf(err, "can't make avatart store for %s", ac.AvatarSrc.Type)
+		return fmt.Errorf("can't make avatart store for %s: %w", ac.AvatarSrc.Type, err)
 	}
 
 	dst, err := ac.makeAvatarStore(ac.AvatarDst)
 	if err != nil {
-		return errors.Wrapf(err, "can't make avatart store for %s", ac.AvatarDst.Type)
+		return fmt.Errorf("can't make avatart store for %s: %w", ac.AvatarDst.Type, err)
 	}
 
 	if ac.migrator == nil {
@@ -72,14 +72,14 @@ func (ac *AvatarCommand) makeAvatarStore(gr AvatarGroup) (avatar.Store, error) {
 	switch gr.Type {
 	case "fs":
 		if err := makeDirs(gr.FS.Path); err != nil {
-			return nil, errors.Wrap(err, "failed to create avatar store")
+			return nil, fmt.Errorf("failed to create avatar store: %w", err)
 		}
 		return avatar.NewLocalFS(gr.FS.Path), nil
 	case "bolt":
 		if err := makeDirs(path.Dir(gr.Bolt.File)); err != nil {
-			return nil, errors.Wrap(err, "failed to create avatar store")
+			return nil, fmt.Errorf("failed to create avatar store: %w", err)
 		}
 		return avatar.NewBoltDB(gr.Bolt.File, bolt.Options{})
 	}
-	return nil, errors.Errorf("unsupported avatar store type %s", gr.Type)
+	return nil, fmt.Errorf("unsupported avatar store type %s", gr.Type)
 }

backend/app/cmd/avatar_test.go

@@ -1,18 +1,17 @@
 package cmd
 
 import (
-	"errors"
+	"fmt"
 	"os"
 	"testing"
 
-	"github.com/go-pkgz/auth/avatar"
+	"github.com/go-pkgz/auth/v2/avatar"
+	"github.com/jessevdk/go-flags"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
-	"github.com/umputun/go-flags"
 )
 
 func TestAvatar_Execute(t *testing.T) {
-
 	defer os.RemoveAll("/tmp/ava-test")
 
 	// from fs to bolt
@@ -22,16 +21,18 @@ func TestAvatar_Execute(t *testing.T) {
 	_, err := p.ParseArgs([]string{"--src.type=fs", "--src.fs.path=/tmp/ava-test", "--dst.type=bolt",
 		"--dst.bolt.file=/tmp/ava-test.db"})
 	require.NoError(t, err)
+	defer os.Remove("/tmp/ava-test.db")
 	err = cmd.Execute(nil)
 	assert.NoError(t, err)
 
 	// failed
-	cmd = AvatarCommand{migrator: &avatarMigratorMock{retCount: 0, retError: errors.New("failed blah")}}
+	cmd = AvatarCommand{migrator: &avatarMigratorMock{retCount: 0, retError: fmt.Errorf("failed blah")}}
 	cmd.SetCommon(CommonOpts{RemarkURL: "", SharedSecret: "123456"})
 	p = flags.NewParser(&cmd, flags.Default)
 	_, err = p.ParseArgs([]string{"--src.type=fs", "--src.fs.path=/tmp/ava-test", "--dst.type=bolt",
 		"--dst.bolt.file=/tmp/ava-test2.db"})
 	require.NoError(t, err)
+	defer os.Remove("/tmp/ava-test2.db")
 	err = cmd.Execute(nil)
 	assert.Error(t, err, "failed blah")
 }

backend/app/cmd/backup.go

@@ -9,17 +9,15 @@ import (
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 )
 
 // BackupCommand set of flags and command for export
 // ExportPath used as a separate element to leverage BACKUP_PATH. If ExportFile has a path (i.e. with /) BACKUP_PATH ignored.
 type BackupCommand struct {
-	ExportPath  string        `short:"p" long:"path" env:"BACKUP_PATH" default:"./var/backup" description:"export path"`
+	ExportPath string `short:"p" long:"path" env:"BACKUP_PATH" default:"./var/backup" description:"export path"`
-	ExportFile  string        `short:"f" long:"file" default:"userbackup-{{.SITE}}-{{.TS}}.gz" description:"file name"`
+	ExportFile string `short:"f" long:"file" default:"userbackup-{{.SITE}}-{{.TS}}.gz" description:"file name"`
-	Site        string        `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
+
-	Timeout     time.Duration `long:"timeout" default:"15m" description:"export (backup) timeout"`
+	SupportCmdOpts
-	AdminPasswd string        `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
 	CommonOpts
 }
 
@@ -38,19 +36,20 @@ func (ec *BackupCommand) Execute(_ []string) error {
 
 	// prepare http client and request
 	client := http.Client{}
+	defer client.CloseIdleConnections()
 	ctx, cancel := context.WithTimeout(context.Background(), ec.Timeout)
 	defer cancel()
 	exportURL := fmt.Sprintf("%s/api/v1/admin/export?mode=file&site=%s", ec.RemarkURL, ec.Site)
 	req, err := http.NewRequest(http.MethodGet, exportURL, http.NoBody)
 	if err != nil {
-		return errors.Wrapf(err, "can't make export request for %s", exportURL)
+		return fmt.Errorf("can't make export request for %s: %w", exportURL, err)
 	}
 	req.SetBasicAuth("admin", ec.AdminPasswd)
 
 	// get with timeout
-	resp, err := client.Do(req.WithContext(ctx))
+	resp, err := client.Do(req.WithContext(ctx)) //nolint:gosec // exportURL is built from operator-supplied CLI flags, not user input
 	if err != nil {
-		return errors.Wrapf(err, "request failed for %s", exportURL)
+		return fmt.Errorf("request failed for %s: %w", exportURL, err)
 	}
 	defer func() {
 		if err = resp.Body.Close(); err != nil {
@@ -62,9 +61,9 @@ func (ec *BackupCommand) Execute(_ []string) error {
 		return responseError(resp)
 	}
 
-	fh, err := os.Create(fname)
+	fh, err := os.Create(fname) //nolint:gosec // harmless
 	if err != nil {
-		return errors.Wrapf(err, "can't create backup file %s", fname)
+		return fmt.Errorf("can't create backup file %s: %w", fname, err)
 	}
 	defer func() { //nolint:gosec // false positive on defer without error check when it's checked here
 		if err = fh.Close(); err != nil {
@@ -73,7 +72,7 @@ func (ec *BackupCommand) Execute(_ []string) error {
 	}()
 
 	if _, err = io.Copy(fh, resp.Body); err != nil {
-		return errors.Wrapf(err, "failed to write backup file %s", fname)
+		return fmt.Errorf("failed to write backup file %s: %w", fname, err)
 	}
 
 	log.Printf("[INFO] export completed, file %s", fname)

backend/app/cmd/backup_test.go

@@ -1,14 +1,15 @@
 package cmd
 
 import (
+	"encoding/base64"
 	"fmt"
 	"net/http"
 	"net/http/httptest"
 	"os"
+	"strings"
 	"testing"
 
-	"github.com/umputun/go-flags"
+	"github.com/jessevdk/go-flags"
-
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 )
@@ -17,6 +18,10 @@ func TestBackup_Execute(t *testing.T) {
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		assert.Equal(t, r.URL.Path, "/api/v1/admin/export")
 		assert.Equal(t, "GET", r.Method)
+		t.Logf("Authorization header: %+v", r.Header.Get("Authorization"))
+		auth, err := base64.StdEncoding.DecodeString(strings.Split(r.Header.Get("Authorization"), " ")[1])
+		require.NoError(t, err)
+		assert.Equal(t, "admin:secret", string(auth))
 		fmt.Fprint(w, "blah\nblah2\n12345678\n")
 	}))
 	defer ts.Close()
@@ -35,6 +40,28 @@ func TestBackup_Execute(t *testing.T) {
 	assert.Equal(t, "blah\nblah2\n12345678\n", string(data))
 }
 
+func TestBackup_ExecuteNoPassword(t *testing.T) {
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		assert.Equal(t, r.URL.Path, "/api/v1/admin/export")
+		assert.Equal(t, "GET", r.Method)
+		t.Logf("Authorization: %+v", r.Header.Get("Authorization"))
+		auth, err := base64.StdEncoding.DecodeString(strings.Split(r.Header.Get("Authorization"), " ")[1])
+		require.NoError(t, err)
+		require.Equal(t, "admin:", string(auth))
+		w.WriteHeader(http.StatusUnauthorized)
+		fmt.Fprint(w, "Unauthorized")
+	}))
+	defer ts.Close()
+
+	cmd := BackupCommand{}
+	cmd.SetCommon(CommonOpts{RemarkURL: ts.URL})
+	p := flags.NewParser(&cmd, flags.Default)
+	_, err := p.ParseArgs([]string{"--site=remark", "--path=/tmp", "--file={{.SITE}}-test.export"})
+	require.NoError(t, err)
+	err = cmd.Execute(nil)
+	assert.EqualError(t, err, "error response \"401 Unauthorized\", ensure you have set ADMIN_PASSWD and provided it to the command you're running: Unauthorized")
+}
+
 func TestBackup_ExecuteFailedStatus(t *testing.T) {
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		assert.Equal(t, r.URL.Path, "/api/v1/admin/export")

backend/app/cmd/cleanup.go

@@ -9,21 +9,20 @@ import (
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store"
 )
 
 // CleanupCommand set of flags and command for cleanup
 type CleanupCommand struct {
-	Site        string   `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
+	Dry      bool     `long:"dry" description:"dry mode, will not remove comments"`
-	Dry         bool     `long:"dry" description:"dry mode, will not remove comments"`
+	From     string   `long:"from" description:"from yyyymmdd"`
-	From        string   `long:"from" description:"from yyyymmdd"`
+	To       string   `long:"to" description:"from yyyymmdd"`
-	To          string   `long:"to" description:"from yyyymmdd"`
+	BadWords []string `short:"w" long:"bword" description:"bad word(s)"`
-	BadWords    []string `short:"w" long:"bword" description:"bad word(s)"`
+	BadUsers []string `short:"u" long:"buser" description:"bad user(s)"`
-	BadUsers    []string `short:"u" long:"buser" description:"bad user(s)"`
+	SetTitle bool     `long:"title" description:"title mode, will not remove comments, but reset titles to page's title'"`
-	AdminPasswd string   `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
+
-	SetTitle    bool     `long:"title" description:"title mode, will not remove comments, but reset titles to page's title'"`
+	SupportCmdOpts
 	CommonOpts
 }
 
@@ -39,7 +38,7 @@ func (cc *CleanupCommand) Execute(_ []string) error {
 
 	posts, err := cc.postsInRange(cc.From, cc.To)
 	if err != nil {
-		return errors.Wrap(err, "can't get posts")
+		return fmt.Errorf("can't get posts: %w", err)
 	}
 	log.Printf("[DEBUG] got %d posts", len(posts))
 
@@ -55,7 +54,6 @@ func (cc *CleanupCommand) Execute(_ []string) error {
 			cc.procTitles(comments)
 		} else {
 			spamComments += cc.procSpam(comments)
-
 		}
 	}
 
@@ -79,7 +77,7 @@ func (cc *CleanupCommand) procSpam(comments []store.Comment) int {
 					log.Printf("[WARN] can't remove comment, %v", err)
 				}
 			}
-			comment.Text = strings.Replace(comment.Text, "\n", " ", -1)
+			comment.Text = strings.ReplaceAll(comment.Text, "\n", " ")
 			log.Printf("[SPAM] %+v [%.0f%%]", comment, score)
 		}
 	}
@@ -100,7 +98,7 @@ func (cc *CleanupCommand) procTitles(comments []store.Comment) {
 func (cc *CleanupCommand) postsInRange(fromS, toS string) ([]store.PostInfo, error) {
 	posts, err := cc.listPosts()
 	if err != nil {
-		return nil, errors.Wrapf(err, "can't list posts for %s", cc.Site)
+		return nil, fmt.Errorf("can't list posts for %s: %w", cc.Site, err)
 	}
 
 	from, to := defaultFrom, defaultTo
@@ -108,14 +106,14 @@ func (cc *CleanupCommand) postsInRange(fromS, toS string) ([]store.PostInfo, err
 	if fromS != "" {
 		from, err = time.ParseInLocation("20060102", fromS, time.Local)
 		if err != nil {
-			return nil, errors.Wrap(err, "can't parse --from")
+			return nil, fmt.Errorf("can't parse --from: %w", err)
 		}
 	}
 
 	if toS != "" {
 		to, err = time.ParseInLocation("20060102", toS, time.Local)
 		if err != nil {
-			return nil, errors.Wrap(err, "can't parse --to")
+			return nil, fmt.Errorf("can't parse --to: %w", err)
 		}
 	}
 
@@ -132,37 +130,38 @@ func (cc *CleanupCommand) postsInRange(fromS, toS string) ([]store.PostInfo, err
 func (cc *CleanupCommand) listPosts() ([]store.PostInfo, error) {
 	listURL := fmt.Sprintf("%s/api/v1/list?site=%s&limit=10000", cc.RemarkURL, cc.Site)
 	client := http.Client{Timeout: 30 * time.Second}
+	defer client.CloseIdleConnections()
 	r, err := client.Get(listURL)
 	if err != nil {
-		return nil, errors.Wrapf(err, "get request failed for list of posts, site %s", cc.Site)
+		return nil, fmt.Errorf("get request failed for list of posts, site %s: %w", cc.Site, err)
 	}
 	defer func() { _ = r.Body.Close() }()
 
 	if r.StatusCode != 200 {
-		return nil, errors.Errorf("request %s failed with status %d", listURL, r.StatusCode)
+		return nil, fmt.Errorf("request %s failed with status %d", listURL, r.StatusCode)
 	}
 
 	list := []store.PostInfo{}
 	if err = json.NewDecoder(r.Body).Decode(&list); err != nil {
-		return nil, errors.Wrapf(err, "can't decode list of posts for site %s", cc.Site)
+		return nil, fmt.Errorf("can't decode list of posts for site %s: %w", cc.Site, err)
 	}
 	return list, nil
 }
 
 // get all comments for post url via /find?site=siteID&url=post-url&format=[tree|plain]
 func (cc *CleanupCommand) listComments(postURL string) ([]store.Comment, error) {
-
 	commentsURL := fmt.Sprintf("%s/api/v1/find?site=%s&url=%s&format=plain", cc.RemarkURL, cc.Site, postURL)
 
 	var r *http.Response
 	var err error
 
 	// handle 429 error from limiter
+	client := http.Client{Timeout: 30 * time.Second}
+	defer client.CloseIdleConnections()
 	for {
-		client := http.Client{Timeout: 30 * time.Second}
 		r, err = client.Get(commentsURL)
 		if err != nil {
-			return nil, errors.Wrapf(err, "get request failed for comments, %s", postURL)
+			return nil, fmt.Errorf("get request failed for comments, %s: %w", postURL, err)
 		}
 		if r.StatusCode == http.StatusTooManyRequests {
 			_ = r.Body.Close()
@@ -175,67 +174,66 @@ func (cc *CleanupCommand) listComments(postURL string) ([]store.Comment, error)
 	defer func() { _ = r.Body.Close() }()
 
 	if r.StatusCode != http.StatusOK {
-		return nil, errors.Errorf("request %s failed with status %d", commentsURL, r.StatusCode)
+		return nil, fmt.Errorf("request %s failed with status %d", commentsURL, r.StatusCode)
 	}
 
 	commentsWithInfo := struct {
 		Comments []store.Comment `json:"comments"`
-		Info     store.PostInfo  `json:"info,omitempty"`
+		Info     store.PostInfo  `json:"info"`
 	}{}
 
 	if err = json.NewDecoder(r.Body).Decode(&commentsWithInfo); err != nil {
-		return nil, errors.Wrapf(err, "can't decode list of comments for %s", postURL)
+		return nil, fmt.Errorf("can't decode list of comments for %s: %w", postURL, err)
 	}
 	return commentsWithInfo.Comments, nil
 }
 
 // deleteComment with DELETE /admin/comment/{id}?site=siteID&url=post-url
-func (cc *CleanupCommand) deleteComment(c store.Comment) error {
+func (cc *CleanupCommand) deleteComment(c store.Comment) error { //nolint:dupl // not worth combining
-
 	deleteURL := fmt.Sprintf("%s/api/v1/admin/comment/%s?site=%s&url=%s&format=plain", cc.RemarkURL, c.ID, cc.Site, c.Locator.URL)
 	req, err := http.NewRequest("DELETE", deleteURL, http.NoBody)
 	if err != nil {
-		return errors.Wrapf(err, "failed to make delete request for comment %s, %s", c.ID, c.Locator.URL)
+		return fmt.Errorf("failed to make delete request for comment %s, %s: %w", c.ID, c.Locator.URL, err)
 	}
 	req.SetBasicAuth("admin", cc.AdminPasswd)
 
 	client := http.Client{}
-	r, err := client.Do(req)
+	defer client.CloseIdleConnections()
+	r, err := client.Do(req) //nolint:gosec // RemarkURL comes from operator CLI flag, not user input
 	if err != nil {
-		return errors.Wrapf(err, "delete request failed for comment %s, %s", c.ID, c.Locator.URL)
+		return fmt.Errorf("delete request failed for comment %s, %s: %w", c.ID, c.Locator.URL, err)
 	}
 	defer func() { _ = r.Body.Close() }()
 	if r.StatusCode != http.StatusOK {
-		return errors.Errorf("delete request failed with status %s", r.Status)
+		return fmt.Errorf("delete request failed with status %s", r.Status)
 	}
 	return nil
 }
 
 // setTitle with PUT /admin/title/{id}?site=siteID&url=post-url
-func (cc *CleanupCommand) setTitle(c store.Comment) error {
+func (cc *CleanupCommand) setTitle(c store.Comment) error { //nolint:dupl // not worth combining
-
 	titleURL := fmt.Sprintf("%s/api/v1/admin/title/%s?site=%s&url=%s&format=plain", cc.RemarkURL, c.ID, cc.Site, c.Locator.URL)
 	req, err := http.NewRequest("PUT", titleURL, http.NoBody)
 	if err != nil {
-		return errors.Wrapf(err, "failed to make title request for comment %s, %s", c.ID, c.Locator.URL)
+		return fmt.Errorf("failed to make title request for comment %s, %s: %w", c.ID, c.Locator.URL, err)
 	}
 	req.SetBasicAuth("admin", cc.AdminPasswd)
 
 	client := http.Client{}
-	r, err := client.Do(req)
+	defer client.CloseIdleConnections()
+	r, err := client.Do(req) //nolint:gosec // RemarkURL comes from operator CLI flag, not user input
 	if err != nil {
-		return errors.Wrapf(err, "title request failed for comment %s, %s", c.ID, c.Locator.URL)
+		return fmt.Errorf("title request failed for comment %s, %s: %w", c.ID, c.Locator.URL, err)
 	}
 	defer func() { _ = r.Body.Close() }()
 	if r.StatusCode != http.StatusOK {
-		return errors.Errorf("title request failed with status %s", r.Status)
+		return fmt.Errorf("title request failed with status %s", r.Status)
 	}
 	return nil
 }
 
 // isSpam calculates spam's probability as a score
 func (cc *CleanupCommand) isSpam(comment store.Comment) (isSpam bool, spamScore float64) {
-
 	badWord := func(txt string) float64 {
 		res := 0.0
 		for _, w := range cc.BadWords {

backend/app/cmd/cleanup_test.go

@@ -10,9 +10,9 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/jessevdk/go-flags"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
-	"github.com/umputun/go-flags"
 
 	"github.com/umputun/remark42/backend/app/store"
 )
@@ -46,7 +46,6 @@ func TestCleanup_IsSpam(t *testing.T) {
 	}
 
 	for n, tt := range tbl {
-		tt := tt
 		checkName := fmt.Sprintf("check-%d-%s", n, tt.name)
 		t.Run(checkName, func(t *testing.T) {
 			c := store.Comment{ID: checkName, Text: tt.text, Score: tt.score}
@@ -59,7 +58,6 @@ func TestCleanup_IsSpam(t *testing.T) {
 }
 
 func TestCleanup_postsInRange(t *testing.T) {
-
 	r := chi.NewRouter()
 	cleanupRoutes(t, r, nil)
 	ts := httptest.NewServer(r)
@@ -175,7 +173,7 @@ func cleanupRoutes(t *testing.T, r *chi.Mux, c *cleanedComments) {
 
 		commentsWithInfo := struct {
 			Comments []store.Comment `json:"comments"`
-			Info     store.PostInfo  `json:"info,omitempty"`
+			Info     store.PostInfo  `json:"info"`
 		}{}
 
 		switch r.URL.Query().Get("url") {
@@ -196,7 +194,7 @@ func cleanupRoutes(t *testing.T, r *chi.Mux, c *cleanedComments) {
 		require.NoError(t, json.NewEncoder(w).Encode(commentsWithInfo))
 	})
 
-	r.HandleFunc("/api/v1/admin/comment/{id}", func(w http.ResponseWriter, r *http.Request) {
+	r.HandleFunc("/api/v1/admin/comment/{id}", func(_ http.ResponseWriter, r *http.Request) {
 		require.Equal(t, "DELETE", r.Method)
 		t.Log("delete ", r.URL.Path)
 		c.lock.Lock()
@@ -204,12 +202,11 @@ func cleanupRoutes(t *testing.T, r *chi.Mux, c *cleanedComments) {
 		c.lock.Unlock()
 	})
 
-	r.HandleFunc("/api/v1/admin/title/{id}", func(w http.ResponseWriter, r *http.Request) {
+	r.HandleFunc("/api/v1/admin/title/{id}", func(_ http.ResponseWriter, r *http.Request) {
 		require.Equal(t, "PUT", r.Method)
 		t.Log("title for ", r.URL.Path)
 		c.lock.Lock()
 		c.ids = append(c.ids, r.URL.Path)
 		c.lock.Unlock()
 	})
-
 }

backend/app/cmd/cmd.go

@@ -4,6 +4,7 @@ package cmd
 
 import (
 	"bytes"
+	"fmt"
 	"io"
 	"net/http"
 	"os"
@@ -13,7 +14,6 @@ import (
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 )
 
 // CommonOptionsCommander extends flags.Commander with SetCommon
@@ -31,11 +31,20 @@ type CommonOpts struct {
 	Revision     string
 }
 
+// SupportCmdOpts is set of commands shared among similar commands like backup/restore and such.
+// Order of fields defines the help command output order.
+type SupportCmdOpts struct {
+	Site        string        `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
+	AdminPasswd string        `long:"admin-passwd" env:"ADMIN_PASSWD" default:"" description:"admin basic auth password"`
+	Timeout     time.Duration `long:"timeout" default:"60m" description:"timeout for the command run"`
+}
+
 // DeprecatedFlag contains information about deprecated option
 type DeprecatedFlag struct {
-	Old     string
+	Old       string
-	New     string
+	New       string
-	Version string
+	Version   string
+	Collision bool
 }
 
 // SetCommon satisfies CommonOptionsCommander interface and sets common option fields
@@ -58,7 +67,6 @@ type fileParser struct {
 
 // parse apply template and also concat path and file. In case if file contains path separator path will be ignored
 func (p *fileParser) parse(now time.Time) (string, error) {
-
 	// file/location parameters my have template masks
 	fileTemplate := struct {
 		YYYYMMDD string
@@ -87,7 +95,7 @@ func (p *fileParser) parse(now time.Time) (string, error) {
 	}
 
 	if err := template.Must(template.New("bb").Parse(fname)).Execute(&bb, fileTemplate); err != nil {
-		return "", errors.Wrapf(err, "failed to parse %q", fname)
+		return "", fmt.Errorf("failed to parse %q: %w", fname, err)
 	}
 	return bb.String(), nil
 }
@@ -107,14 +115,17 @@ func responseError(resp *http.Response) error {
 	if e != nil {
 		body = []byte("")
 	}
-	return errors.Errorf("error response %q, %s", resp.Status, body)
+	if resp.StatusCode == http.StatusUnauthorized {
+		return fmt.Errorf("error response %q, ensure you have set ADMIN_PASSWD and provided it to the command you're running: %s", resp.Status, body)
+	}
+	return fmt.Errorf("error response %q, %s", resp.Status, body)
 }
 
 // mkdir -p for all dirs
 func makeDirs(dirs ...string) error {
 	for _, dir := range dirs {
-		if err := os.MkdirAll(dir, 0o700); err != nil { // If path is already a directory, MkdirAll does nothing
+		if err := os.MkdirAll(dir, 0o700); err != nil { // if path is already a directory, MkdirAll does nothing
-			return errors.Wrapf(err, "can't make directory %s", dir)
+			return fmt.Errorf("can't make directory %s: %w", dir, err)
 		}
 	}
 	return nil

backend/app/cmd/import.go

@@ -8,19 +8,16 @@ import (
 	"net/http"
 	"os"
 	"strings"
-	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 )
 
 // ImportCommand set of flags and command for import
 type ImportCommand struct {
-	InputFile   string        `short:"f" long:"file" description:"input file name" required:"true"`
+	InputFile string `short:"f" long:"file" description:"input file name" required:"true"`
-	Provider    string        `short:"p" long:"provider" default:"disqus" choice:"disqus" choice:"wordpress" choice:"commento" description:"import format"` //nolint
+	Provider  string `short:"p" long:"provider" default:"disqus" choice:"disqus" choice:"wordpress" choice:"commento" description:"import format"` //nolint
-	Site        string        `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
+
-	Timeout     time.Duration `long:"timeout" default:"15m" description:"import timeout"`
+	SupportCmdOpts
-	AdminPasswd string        `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
 	CommonOpts
 }
 
@@ -31,22 +28,23 @@ func (ic *ImportCommand) Execute(_ []string) error {
 
 	reader, err := ic.reader(ic.InputFile)
 	if err != nil {
-		return errors.Wrapf(err, "can't open import file %s", ic.InputFile)
+		return fmt.Errorf("can't open import file %s: %w", ic.InputFile, err)
 	}
 
 	client := http.Client{}
+	defer client.CloseIdleConnections()
 	ctx, cancel := context.WithTimeout(context.Background(), ic.Timeout)
 	defer cancel()
 	importURL := fmt.Sprintf("%s/api/v1/admin/import?site=%s&provider=%s", ic.RemarkURL, ic.Site, ic.Provider)
 	req, err := http.NewRequest(http.MethodPost, importURL, reader)
 	if err != nil {
-		return errors.Wrapf(err, "can't make import request for %s", importURL)
+		return fmt.Errorf("can't make import request for %s: %w", importURL, err)
 	}
 	req.SetBasicAuth("admin", ic.AdminPasswd)
 
-	resp, err := client.Do(req.WithContext(ctx)) // closes request's reader
+	resp, err := client.Do(req.WithContext(ctx)) //nolint:gosec // importURL built from operator CLI flags, not user input; closes request's reader
 	if err != nil {
-		return errors.Wrapf(err, "request failed for %s", importURL)
+		return fmt.Errorf("request failed for %s: %w", importURL, err)
 	}
 	defer func() {
 		if err = resp.Body.Close(); err != nil {
@@ -59,7 +57,7 @@ func (ic *ImportCommand) Execute(_ []string) error {
 
 	body, err := io.ReadAll(resp.Body)
 	if err != nil {
-		return errors.Wrap(err, "can't get response from importer")
+		return fmt.Errorf("can't get response from importer: %w", err)
 	}
 
 	log.Printf("[INFO] completed, status=%d, %s", resp.StatusCode, string(body))
@@ -70,13 +68,13 @@ func (ic *ImportCommand) Execute(_ []string) error {
 func (ic *ImportCommand) reader(inp string) (reader io.Reader, err error) {
 	inpFile, err := os.Open(inp) // nolint
 	if err != nil {
-		return nil, errors.Wrapf(err, "import failed, can't open %s", inp)
+		return nil, fmt.Errorf("import failed, can't open %s: %w", inp, err)
 	}
 
 	reader = inpFile
 	if strings.HasSuffix(ic.InputFile, ".gz") {
 		if reader, err = gzip.NewReader(inpFile); err != nil {
-			return nil, errors.Wrap(err, "can't make gz reader")
+			return nil, fmt.Errorf("can't make gz reader: %w", err)
 		}
 	}
 	return reader, nil

backend/app/cmd/import_test.go

@@ -1,25 +1,29 @@
 package cmd
 
 import (
+	"encoding/base64"
 	"fmt"
 	"io"
 	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/umputun/go-flags"
+	"github.com/jessevdk/go-flags"
-
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 )
 
 func TestImport_Execute(t *testing.T) {
-
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
 		assert.Equal(t, "POST", r.Method)
+		t.Logf("Authorization header: %+v", r.Header.Get("Authorization"))
+		auth, err := base64.StdEncoding.DecodeString(strings.Split(r.Header.Get("Authorization"), " ")[1])
+		require.NoError(t, err)
+		assert.Equal(t, "admin:secret", string(auth))
 		body, err := io.ReadAll(r.Body)
 		assert.NoError(t, err)
 		assert.Equal(t, "blah\nblah2\n12345678\n", string(body))
@@ -48,8 +52,43 @@ func TestImport_Execute(t *testing.T) {
 	assert.NoError(t, err)
 }
 
-func TestImport_ExecuteFailed(t *testing.T) {
+func TestImport_ExecuteNoPassword(t *testing.T) {
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
+		assert.Equal(t, "POST", r.Method)
+		t.Logf("Authorization header: %+v", r.Header.Get("Authorization"))
+		auth, err := base64.StdEncoding.DecodeString(strings.Split(r.Header.Get("Authorization"), " ")[1])
+		require.NoError(t, err)
+		assert.Equal(t, "admin:", string(auth))
+		body, err := io.ReadAll(r.Body)
+		assert.NoError(t, err)
+		assert.Equal(t, "blah\nblah2\n12345678\n", string(body))
 
+		w.WriteHeader(401)
+		fmt.Fprint(w, "Unauthorized")
+	}))
+	defer ts.Close()
+
+	cmd := ImportCommand{}
+	cmd.SetCommon(CommonOpts{RemarkURL: ts.URL})
+
+	p := flags.NewParser(&cmd, flags.Default)
+	_, err := p.ParseArgs([]string{"--site=remark", "--file=testdata/import.txt"})
+	require.NoError(t, err)
+	err = cmd.Execute(nil)
+	assert.EqualError(t, err, "error response \"401 Unauthorized\", ensure you have set ADMIN_PASSWD and provided it to the command you're running: Unauthorized")
+
+	cmd = ImportCommand{}
+	cmd.SetCommon(CommonOpts{RemarkURL: ts.URL})
+
+	p = flags.NewParser(&cmd, flags.Default)
+	_, err = p.ParseArgs([]string{"--site=remark", "--file=testdata/import.txt.gz"})
+	require.NoError(t, err)
+	err = cmd.Execute(nil)
+	assert.EqualError(t, err, "error response \"401 Unauthorized\", ensure you have set ADMIN_PASSWD and provided it to the command you're running: Unauthorized")
+}
+
+func TestImport_ExecuteFailed(t *testing.T) {
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
 		assert.Equal(t, "POST", r.Method)
@@ -103,7 +142,6 @@ func TestImport_ExecuteTimeout(t *testing.T) {
 		time.Sleep(500 * time.Millisecond)
 		fmt.Fprintln(w, "some response")
 		fmt.Fprintln(w, string(body))
-
 	}))
 	defer ts.Close()
 

backend/app/cmd/remap.go

@@ -6,19 +6,16 @@ import (
 	"io"
 	"net/http"
 	"os"
-	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 )
 
 // RemapCommand set of flags and command for change linkage between comments to
 // different urls based on given rules (input file)
 type RemapCommand struct {
-	Site        string        `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
+	InputFile string `short:"f" long:"file" description:"input file name" required:"true"`
-	InputFile   string        `short:"f" long:"file" description:"input file name" required:"true"`
+
-	AdminPasswd string        `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
+	SupportCmdOpts
-	Timeout     time.Duration `long:"timeout" default:"15m" description:"remap timeout"`
 	CommonOpts
 }
 
@@ -29,22 +26,23 @@ func (rc *RemapCommand) Execute(_ []string) error {
 
 	rulesReader, err := os.Open(rc.InputFile)
 	if err != nil {
-		return errors.Wrapf(err, "cant open file %s", rc.InputFile)
+		return fmt.Errorf("cant open file %s: %w", rc.InputFile, err)
 	}
 
 	client := http.Client{}
+	defer client.CloseIdleConnections()
 	ctx, cancel := context.WithTimeout(context.Background(), rc.Timeout)
 	defer cancel()
 	remapURL := fmt.Sprintf("%s/api/v1/admin/remap?site=%s", rc.RemarkURL, rc.Site)
-	req, err := http.NewRequest(http.MethodPost, remapURL, rulesReader)
+	req, err := http.NewRequest(http.MethodPost, remapURL, rulesReader) //nolint:gosec // RemarkURL is operator CLI flag, not user input
 	if err != nil {
-		return errors.Wrapf(err, "can't make remap request for %s", remapURL)
+		return fmt.Errorf("can't make remap request for %s: %w", remapURL, err)
 	}
 	req.SetBasicAuth("admin", rc.AdminPasswd)
 
-	resp, err := client.Do(req.WithContext(ctx))
+	resp, err := client.Do(req.WithContext(ctx)) //nolint:gosec // see above
 	if err != nil {
-		return errors.Wrapf(err, "request failed for %s", remapURL)
+		return fmt.Errorf("request failed for %s: %w", remapURL, err)
 	}
 	defer func() {
 		if err = resp.Body.Close(); err != nil {
@@ -57,7 +55,7 @@ func (rc *RemapCommand) Execute(_ []string) error {
 
 	body, err := io.ReadAll(resp.Body)
 	if err != nil {
-		return errors.Wrap(err, "can't get response")
+		return fmt.Errorf("can't get response: %w", err)
 	}
 
 	log.Printf("[INFO] completed, status=%d, %s", resp.StatusCode, string(body))

backend/app/cmd/remap_test.go

@@ -1,23 +1,28 @@
 package cmd
 
 import (
+	"encoding/base64"
+	"fmt"
 	"io"
 	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
 
-	"github.com/umputun/go-flags"
+	"github.com/jessevdk/go-flags"
-
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 )
 
 func TestRemap_Execute(t *testing.T) {
-
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		assert.Equal(t, r.URL.Path, "/api/v1/admin/remap")
 		assert.Equal(t, "POST", r.Method)
 		assert.Equal(t, "remark", r.URL.Query().Get("site"))
+		t.Logf("Authorization header: %+v", r.Header.Get("Authorization"))
+		auth, err := base64.StdEncoding.DecodeString(strings.Split(r.Header.Get("Authorization"), " ")[1])
+		require.NoError(t, err)
+		assert.Equal(t, "admin:secret", string(auth))
 		body, err := io.ReadAll(r.Body)
 		assert.NoError(t, err)
 		assert.Equal(t, "http://oldsite.com* https://newsite.com*\nhttp://oldsite.com/from-old-page/1 https://newsite.com/to-new-page/1", string(body))
@@ -35,3 +40,31 @@ func TestRemap_Execute(t *testing.T) {
 	err = cmd.Execute(nil)
 	assert.NoError(t, err)
 }
+
+func TestRemap_ExecuteNoPassword(t *testing.T) {
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		assert.Equal(t, r.URL.Path, "/api/v1/admin/remap")
+		assert.Equal(t, "POST", r.Method)
+		assert.Equal(t, "remark", r.URL.Query().Get("site"))
+		t.Logf("Authorization header: %+v", r.Header.Get("Authorization"))
+		auth, err := base64.StdEncoding.DecodeString(strings.Split(r.Header.Get("Authorization"), " ")[1])
+		require.NoError(t, err)
+		assert.Equal(t, "admin:", string(auth))
+		body, err := io.ReadAll(r.Body)
+		assert.NoError(t, err)
+		assert.Equal(t, "http://oldsite.com* https://newsite.com*\nhttp://oldsite.com/from-old-page/1 https://newsite.com/to-new-page/1", string(body))
+
+		w.WriteHeader(401)
+		fmt.Fprint(w, "Unauthorized")
+	}))
+	defer ts.Close()
+
+	cmd := RemapCommand{}
+	cmd.SetCommon(CommonOpts{RemarkURL: ts.URL})
+
+	p := flags.NewParser(&cmd, flags.Default)
+	_, err := p.ParseArgs([]string{"--site=remark", "--file=testdata/remap_urls.txt"})
+	require.NoError(t, err)
+	err = cmd.Execute(nil)
+	assert.EqualError(t, err, "error response \"401 Unauthorized\", ensure you have set ADMIN_PASSWD and provided it to the command you're running: Unauthorized")
+}

backend/app/cmd/restore.go

@@ -11,9 +11,7 @@ type RestoreCommand struct {
 	ImportPath string `short:"p" long:"path" env:"BACKUP_PATH" default:"./var/backup" description:"export path"`
 	ImportFile string `short:"f" long:"file" default:"userbackup-{{.SITE}}-{{.YYYYMMDD}}.gz" description:"file name" required:"true"`
 
-	Site        string        `short:"s" long:"site" env:"SITE" default:"remark" description:"site name"`
+	SupportCmdOpts
-	Timeout     time.Duration `long:"timeout" default:"15m" description:"import timeout"`
-	AdminPasswd string        `long:"admin-passwd" env:"ADMIN_PASSWD" required:"true" description:"admin basic auth password"`
 	CommonOpts
 }
 
@@ -29,12 +27,10 @@ func (rc *RestoreCommand) Execute(args []string) error {
 		return err
 	}
 	importer := ImportCommand{
-		InputFile:   fname,
+		InputFile:      fname,
-		Site:        rc.Site,
+		Provider:       "native",
-		Provider:    "native",
+		SupportCmdOpts: rc.SupportCmdOpts,
-		Timeout:     rc.Timeout,
+		CommonOpts:     rc.CommonOpts,
-		AdminPasswd: rc.AdminPasswd,
-		CommonOpts:  rc.CommonOpts,
 	}
 	return importer.Execute(args)
 }

backend/app/cmd/restore_test.go

@@ -7,14 +7,12 @@ import (
 	"net/http/httptest"
 	"testing"
 
-	"github.com/umputun/go-flags"
+	"github.com/jessevdk/go-flags"
-
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 )
 
 func TestRestore_Execute(t *testing.T) {
-
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		assert.Equal(t, r.URL.Path, "/api/v1/admin/import")
 		assert.Equal(t, "POST", r.Method)

backend/app/cmd/server_test.go

@@ -15,9 +15,10 @@ import (
 	"testing"
 	"time"
 
-	"github.com/go-pkgz/auth/token"
+	"github.com/go-pkgz/auth/v2/provider"
-	"github.com/golang-jwt/jwt"
+	"github.com/go-pkgz/auth/v2/token"
-	"github.com/umputun/go-flags"
+	"github.com/golang-jwt/jwt/v5"
+	"github.com/jessevdk/go-flags"
 	"go.uber.org/goleak"
 
 	"github.com/stretchr/testify/assert"
@@ -36,9 +37,10 @@ func TestServerApp(t *testing.T) {
 
 	// send ping
 	resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
+	defer http.DefaultClient.CloseIdleConnections()
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	body, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.Equal(t, "pong", string(body))
@@ -46,7 +48,7 @@ func TestServerApp(t *testing.T) {
 	// add comment
 	client := http.Client{Timeout: 10 * time.Second}
 	defer client.CloseIdleConnections()
-	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
+	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
 		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/blah1", "site": "remark"}}`))
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -78,12 +80,13 @@ func TestServerApp_DevMode(t *testing.T) {
 	waitForHTTPServerStart(port)
 
 	providers := app.restSrv.Authenticator.Providers()
-	require.Equal(t, 9+1, len(providers), "extra auth provider")
+	require.Equal(t, 11+1, len(providers), "extra auth provider")
 	assert.Equal(t, "dev", providers[len(providers)-2].Name(), "dev auth provider")
 	// send ping
 	resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
+	defer http.DefaultClient.CloseIdleConnections()
 	require.NoError(t, err)
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	body, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
@@ -93,6 +96,30 @@ func TestServerApp_DevMode(t *testing.T) {
 	app.Wait()
 }
 
+func TestServerApp_CustomOAuthProvider(t *testing.T) {
+	port := chooseRandomUnusedPort()
+	app, ctx, cancel := prepServerApp(t, func(o ServerCommand) ServerCommand {
+		o.Port = port
+		o.Auth.Custom.Name = "oidc"
+		o.Auth.Custom.CID = "cid"
+		o.Auth.Custom.CSEC = "csec"
+		o.Auth.Custom.AuthURL = "https://example.com/oauth2/authorize"
+		o.Auth.Custom.TokenURL = "https://example.com/oauth2/token"
+		o.Auth.Custom.InfoURL = "https://example.com/oauth2/userinfo"
+		return o
+	})
+
+	go func() { _ = app.run(ctx) }()
+	waitForHTTPServerStart(port)
+
+	providers := app.restSrv.Authenticator.Providers()
+	require.Equal(t, 11+1, len(providers), "extra auth provider")
+	assert.Equal(t, "oidc", providers[len(providers)-2].Name(), "custom auth provider")
+
+	cancel()
+	app.Wait()
+}
+
 func TestServerApp_AnonMode(t *testing.T) {
 	port := chooseRandomUnusedPort()
 	app, ctx, cancel := prepServerApp(t, func(o ServerCommand) ServerCommand {
@@ -105,11 +132,14 @@ func TestServerApp_AnonMode(t *testing.T) {
 	waitForHTTPServerStart(port)
 
 	providers := app.restSrv.Authenticator.Providers()
-	require.Equal(t, 9+1, len(providers), "extra auth provider for anon")
+	require.Equal(t, 11+1, len(providers), "extra auth provider for anon")
 	assert.Equal(t, "anonymous", providers[len(providers)-1].Name(), "anon auth provider")
 
+	client := http.Client{Timeout: 10 * time.Second}
+	defer client.CloseIdleConnections()
+
 	// send ping
-	resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
+	resp, err := client.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusOK, resp.StatusCode)
@@ -118,15 +148,13 @@ func TestServerApp_AnonMode(t *testing.T) {
 	assert.Equal(t, "pong", string(body))
 
 	// try to login with good name
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=blah123&aud=remark", port))
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=blah123&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	// try to add a comment as good anonymous
-	client := http.Client{Timeout: 10 * time.Second}
+	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
-	defer client.CloseIdleConnections()
-	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
 		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/blah1", "site": "remark"}}`))
 	require.NoError(t, err)
 
@@ -140,51 +168,58 @@ func TestServerApp_AnonMode(t *testing.T) {
 	assert.Equal(t, http.StatusCreated, resp.StatusCode)
 
 	// try to login with non-latin name
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=Раз_Два%20%20Три_34567&aud=remark", port))
+	time.Sleep(time.Second)
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=Раз_Два%20%20Три_34567&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	// try to login with bad name
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=**blah123&aud=remark", port))
+	time.Sleep(time.Second)
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=**blah123&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 
 	// try to login with short name
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=bl%%20%%20&aud=remark", port))
+	time.Sleep(time.Second)
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=bl%%20%%20&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 
 	// try to login with name what have space in prefix
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%%20somebody&aud=remark", port))
+	time.Sleep(time.Second)
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%%20somebody&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 
 	// try to login with name what have space in suffix
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=somebody%%20&aud=remark", port))
+	time.Sleep(time.Second)
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=somebody%%20&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 
 	// try to login with long name
+	time.Sleep(time.Second)
 	ln := strings.Repeat("x", 65)
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%s&aud=remark", port, ln))
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=%s&aud=remark", port, ln))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 
 	// try to login with admin name
-	resp, err = http.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=umpUtun&aud=remark", port))
+	time.Sleep(time.Second)
+	resp, err = client.Get(fmt.Sprintf("http://localhost:%d/auth/anonymous/login?user=umpUtun&aud=remark", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
 	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	// try to add a comment as anonymous with admin name
-	client = http.Client{Timeout: 10 * time.Second}
+	time.Sleep(time.Second)
-	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
+	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
 		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/blah1", "site": "remark"}}`))
 	require.NoError(t, err)
 
@@ -226,6 +261,9 @@ func TestServerApp_WithSSL(t *testing.T) {
 		"--ssl.type=static", "--ssl.cert=testdata/cert.pem", "--ssl.key=testdata/key.pem",
 		"--ssl.port=" + strconv.Itoa(sslPort), "--image.fs.path=/tmp"})
 	require.NoError(t, err)
+	defer os.Remove("/tmp/xyz")
+	defer os.Remove("/tmp/xyz/remark.db")
+	defer os.Remove("/tmp/ava-test.db")
 
 	// create app
 	app, err := opts.newServerApp(context.Background())
@@ -237,7 +275,7 @@ func TestServerApp_WithSSL(t *testing.T) {
 
 	client := http.Client{
 		// prevent http redirect
-		CheckRedirect: func(req *http.Request, via []*http.Request) error {
+		CheckRedirect: func(*http.Request, []*http.Request) error {
 			return http.ErrUseLastResponse
 		},
 
@@ -246,19 +284,20 @@ func TestServerApp_WithSSL(t *testing.T) {
 			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
 		},
 	}
+	defer client.CloseIdleConnections()
 
 	// check http to https redirect response
 	resp, err := client.Get(fmt.Sprintf("http://localhost:%d/blah?param=1", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 307, resp.StatusCode)
+	assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
 	assert.Equal(t, fmt.Sprintf("https://localhost:%d/blah?param=1", sslPort), resp.Header.Get("Location"))
 
 	// check https server
 	resp, err = client.Get(fmt.Sprintf("https://localhost:%d/ping", sslPort))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	body, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.Equal(t, "pong", string(body))
@@ -268,7 +307,6 @@ func TestServerApp_WithSSL(t *testing.T) {
 }
 
 func TestServerApp_WithRemote(t *testing.T) {
-
 	opts := ServerCommand{}
 	opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
 
@@ -277,7 +315,8 @@ func TestServerApp_WithRemote(t *testing.T) {
 	port := chooseRandomUnusedPort()
 	_, err := p.ParseArgs([]string{"--admin-passwd=password", "--cache.type=none",
 		"--store.type=rpc", "--store.rpc.api=http://127.0.0.1",
-		"--port=" + strconv.Itoa(port), "--admin.type=rpc", "--admin.rpc.api=http://127.0.0.1", "--avatar.fs.path=/tmp"})
+		"--port=" + strconv.Itoa(port), "--avatar.fs.path=/tmp",
+		"--admin.type=rpc", "--admin.rpc.secret_per_site", "--admin.rpc.api=http://127.0.0.1"})
 	require.NoError(t, err)
 	opts.Auth.Github.CSEC, opts.Auth.Github.CID = "csec", "cid"
 	opts.BackupLocation, opts.Image.FS.Path = "/tmp", "/tmp"
@@ -292,9 +331,10 @@ func TestServerApp_WithRemote(t *testing.T) {
 
 	// send ping
 	resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
+	defer http.DefaultClient.CloseIdleConnections()
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	body, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.Equal(t, "pong", string(body))
@@ -322,6 +362,7 @@ func TestServerApp_Failed(t *testing.T) {
 
 	_, err = p.ParseArgs([]string{"--store.bolt.path=/tmp", "--backup=/dev/null/not-writable"})
 	assert.NoError(t, err)
+	defer os.Remove("/tmp/remark.db")
 	_, err = opts.newServerApp(context.Background())
 	assert.EqualError(t, err, "failed to create backup store: can't make directory /dev/null/not-writable: mkdir /dev/null: not a directory")
 	t.Log(err)
@@ -336,6 +377,7 @@ func TestServerApp_Failed(t *testing.T) {
 	assert.EqualError(t, err, "invalid remark42 url demo.remark42.com")
 	t.Log(err)
 
+	// wrong store type
 	opts = ServerCommand{}
 	opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
 
@@ -359,6 +401,108 @@ func TestServerApp_Failed(t *testing.T) {
 			"problem subscribing to channel remark42-cache on address wrong_address: "+
 			"dial tcp: address wrong_address: missing port in address")
 	t.Log(err)
+
+	// wrong apple private key type
+	opts = ServerCommand{}
+	opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
+	p = flags.NewParser(&opts, flags.Default)
+	_, err = p.ParseArgs([]string{"--auth.apple.cid=123", "--auth.apple.tid=123",
+		"--auth.apple.kid=123", "--auth.apple.private-key-filepath=testdata/apple-bad.p8"})
+	assert.NoError(t, err)
+	_, err = opts.newServerApp(context.Background())
+	assert.EqualError(t, err,
+		"failed to make authenticator: an AppleProvider creating failed: "+
+			"provided private key is not ECDSA")
+	t.Log(err)
+
+	// incomplete custom oauth config
+	opts = ServerCommand{}
+	opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
+	p = flags.NewParser(&opts, flags.Default)
+	_, err = p.ParseArgs([]string{"--store.bolt.path=/tmp", "--backup=/tmp", "--image.fs.path=/tmp", "--auth.custom.name=oidc", "--auth.custom.cid=123"})
+	assert.NoError(t, err)
+	_, err = opts.newServerApp(context.Background())
+	assert.EqualError(t, err,
+		"failed to make authenticator: custom oauth provider configuration is incomplete, missing: "+
+			"AUTH_CUSTOM_CSEC, AUTH_CUSTOM_AUTH_URL, AUTH_CUSTOM_TOKEN_URL, AUTH_CUSTOM_INFO_URL")
+	t.Log(err)
+}
+
+func TestIsReservedCustomProviderName(t *testing.T) {
+	reserved := []string{
+		"email", "anonymous", "google", "github", "facebook", "yandex", "twitter",
+		"microsoft", "patreon", "discord", "telegram", "dev", "apple",
+	}
+
+	for _, name := range reserved {
+		t.Run(name, func(t *testing.T) {
+			assert.True(t, isReservedCustomProviderName(name))
+		})
+	}
+
+	assert.False(t, isReservedCustomProviderName("oidc"))
+}
+
+func TestIsValidCustomProviderName(t *testing.T) {
+	valid := []string{"oidc", "codeberg", "provider_1", "provider-1", "a1"}
+	for _, name := range valid {
+		t.Run("valid_"+name, func(t *testing.T) {
+			assert.True(t, isValidCustomProviderName(name))
+		})
+	}
+
+	invalid := []string{"", " has-space", "has space", "Uppercase", "provider!", "-provider", "_provider"}
+	for _, name := range invalid {
+		t.Run("invalid_"+strings.ReplaceAll(name, " ", "_"), func(t *testing.T) {
+			assert.False(t, isValidCustomProviderName(name))
+		})
+	}
+}
+
+func TestCustomProviderSourceID(t *testing.T) {
+	cfg := CustomAuthGroup{IDField: "sub", EmailField: "email", NameField: "name", PictureField: "picture"}
+
+	assert.Equal(t, "user-1", customProviderSourceID(provider.UserData{"sub": "user-1", "email": "a@example.com"}, cfg))
+	assert.Equal(t, "a@example.com", customProviderSourceID(provider.UserData{"email": "a@example.com"}, cfg))
+	assert.Equal(t, "alice", customProviderSourceID(provider.UserData{"name": "alice"}, cfg))
+	assert.Equal(t, "https://example.com/avatar.png", customProviderSourceID(provider.UserData{"picture": "https://example.com/avatar.png"}, cfg))
+	assert.Equal(t, `{"login":"alice"}`, customProviderSourceID(provider.UserData{"login": "alice"}, cfg))
+	assert.Equal(t, "{}", customProviderSourceID(provider.UserData{}, cfg))
+}
+
+func TestServerApp_InvalidCustomOAuthProviderName(t *testing.T) {
+	baseArgs := []string{
+		"--store.bolt.path=/tmp",
+		"--backup=/tmp",
+		"--image.fs.path=/tmp",
+		"--auth.custom.cid=123",
+		"--auth.custom.csec=456",
+		"--auth.custom.auth-url=https://example.com/oauth2/authorize",
+		"--auth.custom.token-url=https://example.com/oauth2/token",
+		"--auth.custom.info-url=https://example.com/oauth2/userinfo",
+	}
+
+	t.Run("reserved", func(t *testing.T) {
+		opts := ServerCommand{}
+		opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
+		p := flags.NewParser(&opts, flags.Default)
+		_, err := p.ParseArgs(append(baseArgs, "--auth.custom.name=twitter"))
+		require.NoError(t, err)
+
+		_, err = opts.newServerApp(context.Background())
+		assert.EqualError(t, err, `failed to make authenticator: custom oauth provider name "twitter" is reserved`)
+	})
+
+	t.Run("not_url_safe", func(t *testing.T) {
+		opts := ServerCommand{}
+		opts.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
+		p := flags.NewParser(&opts, flags.Default)
+		_, err := p.ParseArgs(append(baseArgs, "--auth.custom.name=bad name"))
+		require.NoError(t, err)
+
+		_, err = opts.newServerApp(context.Background())
+		assert.EqualError(t, err, `failed to make authenticator: custom oauth provider name "bad name" is invalid, expected pattern "^[a-z0-9][a-z0-9_-]*$"`)
+	})
 }
 
 func TestServerApp_Shutdown(t *testing.T) {
@@ -377,7 +521,6 @@ func TestServerApp_Shutdown(t *testing.T) {
 }
 
 func TestServerApp_MainSignal(t *testing.T) {
-
 	done := make(chan struct{})
 	go func() {
 		<-done
@@ -393,6 +536,8 @@ func TestServerApp_MainSignal(t *testing.T) {
 	port := chooseRandomUnusedPort()
 	args := []string{"test", "--store.bolt.path=/tmp/xyz", "--backup=/tmp", "--avatar.type=bolt",
 		"--avatar.bolt.file=/tmp/ava-test.db", "--port=" + strconv.Itoa(port), "--image.fs.path=/tmp"}
+	defer os.Remove("/tmp/xyz")
+	defer os.Remove("/tmp/xyz/remark.db")
 	defer os.Remove("/tmp/ava-test.db")
 	_, err := p.ParseArgs(args)
 	require.NoError(t, err)
@@ -410,7 +555,12 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
 	p := flags.NewParser(&s, flags.Default)
 	args := []string{
 		"test",
+		"--notify.type=email",
 		"--notify.type=telegram",
+		"--notify.users=none",
+		"--notify.admins=none",
+		"--img-proxy",
+		"--notify.email.notify_admin",
 		"--auth.email.host=smtp.example.org",
 		"--auth.email.port=666",
 		"--auth.email.tls",
@@ -420,6 +570,9 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
 		"--auth.email.template=file.tmpl",
 		"--notify.telegram.token=abcd",
 		"--notify.telegram.timeout=3m",
+		"--notify.telegram.api=http://example.org",
+		"--auth.twitter.cid=123",
+		"--auth.twitter.csec=456",
 	}
 	assert.Empty(t, s.SMTP.Host)
 	assert.Empty(t, s.SMTP.Port)
@@ -439,9 +592,14 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
 			{Old: "auth.email.passwd", New: "smtp.password", Version: "1.5"},
 			{Old: "auth.email.timeout", New: "smtp.timeout", Version: "1.5"},
 			{Old: "auth.email.template", Version: "1.5"},
+			{Old: "img-proxy", New: "image-proxy.http2https", Version: "1.5"},
+			{Old: "notify.email.notify_admin", New: "notify.admins=email", Version: "1.9"},
 			{Old: "notify.type", New: "notify.(users|admins)", Version: "1.9"},
 			{Old: "notify.telegram.token", New: "telegram.token", Version: "1.9"},
 			{Old: "notify.telegram.timeout", New: "telegram.timeout", Version: "1.9"},
+			{Old: "notify.telegram.api", Version: "1.9"},
+			{Old: "auth.twitter.cid", Version: "1.14"},
+			{Old: "auth.twitter.csec", Version: "1.14"},
 		},
 		deprecatedFlags)
 	assert.Equal(t, "smtp.example.org", s.SMTP.Host)
@@ -452,6 +610,62 @@ func TestServerApp_DeprecatedArgs(t *testing.T) {
 	assert.Equal(t, 15*time.Second, s.SMTP.TimeOut)
 }
 
+func TestServerApp_DeprecatedArgsCollisions(t *testing.T) {
+	s := ServerCommand{}
+	s.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
+
+	p := flags.NewParser(&s, flags.Default)
+	args := []string{
+		"test",
+		"--auth.email.host=smtp-old.example.org",
+		"--smtp.host=smtp-new.example.org",
+		"--auth.email.port=666",
+		"--smtp.port=999",
+		"--auth.email.user=test_user",
+		"--smtp.username=new_test_user",
+		"--auth.email.passwd=test_password",
+		"--smtp.password=new_test_password",
+		"--auth.email.timeout=15s",
+		"--smtp.timeout=20s",
+		"--notify.type=telegram",
+		"--notify.users=telegram",
+		"--notify.admins=none",
+		"--notify.telegram.token=abcd",
+		"--telegram.token=dcba",
+		"--notify.telegram.timeout=3m",
+		"--telegram.timeout=5m",
+	}
+	_, err := p.ParseArgs(args)
+	require.NoError(t, err)
+	deprecatedFlagsCollisions := s.findDeprecatedFlagsCollisions()
+	assert.ElementsMatch(t,
+		[]DeprecatedFlag{
+			{Old: "notify.type", New: "notify.(users|admins)", Collision: true},
+			{Old: "auth.email.host", New: "smtp.host", Collision: true},
+			{Old: "auth.email.port", New: "smtp.port", Collision: true},
+			{Old: "auth.email.user", New: "smtp.username", Collision: true},
+			{Old: "auth.email.passwd", New: "smtp.password", Collision: true},
+			{Old: "auth.email.timeout", New: "smtp.timeout", Collision: true},
+			{Old: "notify.telegram.token", New: "telegram.token", Collision: true},
+			{Old: "notify.telegram.timeout", New: "telegram.timeout", Collision: true},
+		},
+		deprecatedFlagsCollisions)
+
+	// case which should return nothing
+	s = ServerCommand{}
+	s.SetCommon(CommonOpts{RemarkURL: "https://demo.remark42.com", SharedSecret: "123456"})
+	p = flags.NewParser(&s, flags.Default)
+	args = []string{
+		"test",
+		"--auth.email.host=smtp-old.example.org",
+		"--smtp.host=''",
+	}
+	_, err = p.ParseArgs(args)
+	require.NoError(t, err)
+	deprecatedFlagsCollisions = s.findDeprecatedFlagsCollisions()
+	assert.Empty(t, []DeprecatedFlag{}, deprecatedFlagsCollisions)
+}
+
 func Test_ACMEEmail(t *testing.T) {
 	cmd := ServerCommand{}
 	cmd.SetCommon(CommonOpts{RemarkURL: "https://remark.com:443", SharedSecret: "123456"})
@@ -509,14 +723,14 @@ func TestServerAuthHooks(t *testing.T) {
 	tkService.TokenDuration = time.Second
 
 	claims := token.Claims{
-		StandardClaims: jwt.StandardClaims{
+		RegisteredClaims: jwt.RegisteredClaims{
-			Audience:  "remark",
+			Audience:  jwt.ClaimStrings{"remark"},
 			Issuer:    "remark",
-			ExpiresAt: time.Now().Add(time.Second).Unix(),
+			ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Second)),
-			NotBefore: time.Now().Add(-1 * time.Minute).Unix(),
+			NotBefore: jwt.NewNumericDate(time.Now().Add(-1 * time.Minute)),
 		},
 		User: &token.User{
-			ID:   "dev",
+			ID:   "github_dev",
 			Name: "developer one",
 		},
 	}
@@ -524,9 +738,11 @@ func TestServerAuthHooks(t *testing.T) {
 	require.NoError(t, err)
 	t.Log(tk)
 
-	// add comment
 	client := http.Client{Timeout: 10 * time.Second}
-	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
+	defer client.CloseIdleConnections()
+
+	// add comment
+	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
 		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/p/2018/12/29/podcast-630/", "site": "remark"}}`))
 	require.NoError(t, err)
 	req.Header.Set("X-JWT", tk)
@@ -535,13 +751,13 @@ func TestServerAuthHooks(t *testing.T) {
 	require.NoError(t, resp.Body.Close())
 	assert.Equal(t, http.StatusCreated, resp.StatusCode, "non-blocked user able to post")
 
-	// add comment with no-aud claim
+	// try to add comment with no-aud claim
-	claimsNoAud := claims
+	badClaimsNoAud := claims
-	claimsNoAud.Audience = ""
+	badClaimsNoAud.Audience = jwt.ClaimStrings{""}
-	tkNoAud, err := tkService.Token(claimsNoAud)
+	tkNoAud, err := tkService.Token(badClaimsNoAud)
 	require.NoError(t, err)
 	t.Logf("no-aud claims: %s", tkNoAud)
-	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
+	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
 		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/p/2018/12/29/podcast-631/",
 	"site": "remark"}}`))
 	require.NoError(t, err)
@@ -553,21 +769,58 @@ func TestServerAuthHooks(t *testing.T) {
 	require.NoError(t, resp.Body.Close())
 	assert.Equal(t, http.StatusUnauthorized, resp.StatusCode, "user without aud claim rejected, \n"+tkNoAud+"\n"+string(body))
 
-	// block user dev as admin
+	// try to add comment with multiple auds
+	badClaimsMultipleAud := claims
+	badClaimsMultipleAud.Audience = jwt.ClaimStrings{"remark", "second_aud"}
+	tkMultipleAuds, err := tkService.Token(badClaimsMultipleAud)
+	require.NoError(t, err)
+	t.Logf("multiple aud claims: %s", tkMultipleAuds)
+	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
+		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/p/2018/12/29/podcast-631/",
+	"site": "remark"}}`))
+	require.NoError(t, err)
+	req.Header.Set("X-JWT", tkMultipleAuds)
+	resp, err = client.Do(req)
+	require.NoError(t, err)
+	body, err = io.ReadAll(resp.Body)
+	require.NoError(t, err)
+	require.NoError(t, resp.Body.Close())
+	assert.Equal(t, http.StatusUnauthorized, resp.StatusCode, "user with multiple auds claim rejected, \n"+tkMultipleAuds+"\n"+string(body))
+
+	// try to add comment without user set
+	badClaimsNoUser := claims
+	badClaimsNoUser.Audience = jwt.ClaimStrings{"remark"}
+	badClaimsNoUser.User = nil
+	tkNoUser, err := tkService.Token(badClaimsNoUser)
+	require.NoError(t, err)
+	t.Logf("no user claims: %s", tkNoUser)
+	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
+		strings.NewReader(`{"text": "test 123", "locator":{"url": "https://radio-t.com/p/2018/12/29/podcast-631/",
+	"site": "remark"}}`))
+	require.NoError(t, err)
+	req.Header.Set("X-JWT", tkNoUser)
+	resp, err = client.Do(req)
+	require.NoError(t, err)
+	body, err = io.ReadAll(resp.Body)
+	require.NoError(t, err)
+	require.NoError(t, resp.Body.Close())
+	assert.Equal(t, http.StatusUnauthorized, resp.StatusCode, "user without user information rejected, \n"+tkNoUser+"\n"+string(body))
+
+	// block user github_dev as admin
 	req, err = http.NewRequest(http.MethodPut,
-		fmt.Sprintf("http://localhost:%d/api/v1/admin/user/dev?site=remark&block=1&ttl=10d", port), http.NoBody)
+		fmt.Sprintf("http://localhost:%d/api/v1/admin/user/github_dev?site=remark&block=1&ttl=10d", port), http.NoBody)
 	assert.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
 	resp, err = client.Do(req)
 	require.NoError(t, err)
-	assert.Equal(t, http.StatusOK, resp.StatusCode, "user dev blocked")
+	assert.Equal(t, http.StatusOK, resp.StatusCode, "user github_dev blocked")
 	b, err := io.ReadAll(resp.Body)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
 	t.Log(string(b))
 
 	// try add a comment with blocked user
-	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment", port),
+	req, err = http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/api/v1/comment?site=remark", port),
 		strings.NewReader(`{"text": "test 123 blah", "locator":{"url": "https://radio-t.com/blah1", "site": "remark"}}`))
 	require.NoError(t, err)
 	req.Header.Set("X-JWT", tk)
@@ -584,21 +837,7 @@ func TestServerAuthHooks(t *testing.T) {
 	client.CloseIdleConnections()
 }
 
-func TestServer_loadEmailTemplate(t *testing.T) {
-	cmd := ServerCommand{}
-	cmd.Auth.Email.MsgTemplate = "testdata/email.tmpl"
-	r, err := cmd.loadEmailTemplate()
-	assert.NoError(t, err)
-	assert.Equal(t, "The token is {{.Token}}", r)
-
-	cmd.Auth.Email.MsgTemplate = "badpath.tmpl"
-	r, err = cmd.loadEmailTemplate()
-	assert.EqualError(t, err, "failed to read file badpath.tmpl: open badpath.tmpl: no such file or directory")
-	assert.Equal(t, r, "")
-}
-
 func TestServerCommand_parseSameSite(t *testing.T) {
-
 	tbl := []struct {
 		inp string
 		res http.SameSite
@@ -613,7 +852,6 @@ func TestServerCommand_parseSameSite(t *testing.T) {
 
 	cmd := ServerCommand{}
 	for i, tt := range tbl {
-		tt := tt
 		t.Run(strconv.Itoa(i), func(t *testing.T) {
 			assert.Equal(t, tt.res, cmd.parseSameSite(tt.inp))
 		})
@@ -621,7 +859,6 @@ func TestServerCommand_parseSameSite(t *testing.T) {
 }
 
 func Test_splitAtCommas(t *testing.T) {
-
 	tbl := []struct {
 		inp string
 		res []string
@@ -643,8 +880,58 @@ func Test_splitAtCommas(t *testing.T) {
 	}
 }
 
+func Test_getAllowedDomains(t *testing.T) {
+	tbl := []struct {
+		s              ServerCommand
+		allowedDomains []string
+	}{
+		// correct example, parsed and returned as allowed domain
+		{ServerCommand{AllowedHosts: []string{}, CommonOpts: CommonOpts{RemarkURL: "https://remark42.example.org"}}, []string{"example.org"}},
+		{ServerCommand{AllowedHosts: []string{}, CommonOpts: CommonOpts{RemarkURL: "http://remark42.example.org"}}, []string{"example.org"}},
+		{ServerCommand{AllowedHosts: []string{}, CommonOpts: CommonOpts{RemarkURL: "http://localhost"}}, []string{"localhost"}},
+		// incorrect URLs, so Hostname is empty but returned list doesn't include empty string as it would allow any domain
+		{ServerCommand{AllowedHosts: []string{}, CommonOpts: CommonOpts{RemarkURL: "bad hostname"}}, []string{}},
+		{ServerCommand{AllowedHosts: []string{}, CommonOpts: CommonOpts{RemarkURL: "not_a_hostname"}}, []string{}},
+		// test removal of 'self', multiple AllowedHosts. No deduplication is expected
+		{ServerCommand{AllowedHosts: []string{"'self'", "example.org", "test.example.org", "remark42.com"}, CommonOpts: CommonOpts{RemarkURL: "https://example.org"}}, []string{"example.org", "test.example.org", "remark42.com", "example.org"}},
+	}
+	for i, tt := range tbl {
+		t.Run(strconv.Itoa(i), func(t *testing.T) {
+			assert.Equal(t, tt.allowedDomains, tt.s.getAllowedDomains())
+		})
+	}
+}
+
+func Test_getAllowedRedirectHosts(t *testing.T) {
+	tbl := []struct {
+		name  string
+		hosts []string
+		want  []string
+	}{
+		{name: "empty", hosts: nil, want: []string{}},
+		{name: "bare hostnames pass through", hosts: []string{"example.com", "admin.example.com"}, want: []string{"example.com", "admin.example.com"}},
+		{name: "https scheme stripped", hosts: []string{"https://example.com"}, want: []string{"example.com"}},
+		{name: "http scheme stripped", hosts: []string{"http://example.com"}, want: []string{"example.com"}},
+		{name: "scheme with path strips path", hosts: []string{"https://example.com/embed"}, want: []string{"example.com"}},
+		{name: "explicit port preserved as host:port", hosts: []string{"example.com:8080"}, want: []string{"example.com:8080"}},
+		{name: "scheme with explicit port preserved", hosts: []string{"https://example.com:8443"}, want: []string{"example.com:8443"}},
+		{name: "scheme without port stays bare host", hosts: []string{"https://example.com"}, want: []string{"example.com"}},
+		{name: "self sentinel filtered", hosts: []string{"'self'", "self", `"self"`, "example.com"}, want: []string{"example.com"}},
+		{name: "wildcards filtered", hosts: []string{"*", "*.example.com", "https://*.example.com", "example.com"}, want: []string{"example.com"}},
+		{name: "empty entries filtered", hosts: []string{"", "  ", "example.com"}, want: []string{"example.com"}},
+		{name: "mixed real-world", hosts: []string{"'self'", "https://blog.example.com", "admin.example.com:8443", "*.cdn.example.com"},
+			want: []string{"blog.example.com", "admin.example.com:8443"}},
+	}
+	for _, tt := range tbl {
+		t.Run(tt.name, func(t *testing.T) {
+			s := ServerCommand{AllowedHosts: tt.hosts}
+			assert.Equal(t, tt.want, s.getAllowedRedirectHosts())
+		})
+	}
+}
+
 func chooseRandomUnusedPort() (port int) {
-	for i := 0; i < 10; i++ {
+	for range 10 {
 		port = 40000 + int(rand.Int31n(10000))
 		if ln, err := net.Listen("tcp", fmt.Sprintf(":%d", port)); err == nil {
 			_ = ln.Close()
@@ -657,7 +944,8 @@ func chooseRandomUnusedPort() (port int) {
 func waitForHTTPServerStart(port int) {
 	// wait for up to 3 seconds for server to start before returning it
 	client := http.Client{Timeout: time.Second}
-	for i := 0; i < 300; i++ {
+	defer client.CloseIdleConnections()
+	for range 300 {
 		time.Sleep(time.Millisecond * 10)
 		if resp, err := client.Get(fmt.Sprintf("http://localhost:%d", port)); err == nil {
 			_ = resp.Body.Close()
@@ -668,7 +956,7 @@ func waitForHTTPServerStart(port int) {
 
 func waitForHTTPSServerStart(port int) {
 	// wait for up to 3 seconds for HTTPS server to start
-	for i := 0; i < 300; i++ {
+	for range 300 {
 		time.Sleep(time.Millisecond * 10)
 		conn, _ := net.DialTimeout("tcp", fmt.Sprintf("localhost:%d", port), time.Millisecond*10)
 		if conn != nil {
@@ -686,9 +974,10 @@ func prepServerApp(t *testing.T, fn func(o ServerCommand) ServerCommand) (*serve
 	p := flags.NewParser(&cmd, flags.Default)
 	_, err := p.ParseArgs([]string{"--admin-passwd=password", "--site=remark"})
 	require.NoError(t, err)
-	cmd.Avatar.FS.Path, cmd.Avatar.Type, cmd.BackupLocation, cmd.Image.FS.Path = "/tmp", "fs", "/tmp", "/tmp"
+	cmd.Avatar.FS.Path, cmd.Avatar.Type, cmd.BackupLocation, cmd.Image.FS.Path = "/tmp/remark42_test", "fs", "/tmp/remark42_test", "/tmp/remark42_test"
-	cmd.Store.Bolt.Path = fmt.Sprintf("/tmp/%d", cmd.Port)
 	cmd.Store.Bolt.Timeout = 10 * time.Second
+	cmd.Auth.Apple.CID, cmd.Auth.Apple.KID, cmd.Auth.Apple.TID = "cid", "kid", "tid"
+	cmd.Auth.Apple.PrivateKeyFilePath = "testdata/apple.p8"
 	cmd.Auth.Github.CSEC, cmd.Auth.Github.CID = "csec", "cid"
 	cmd.Auth.Google.CSEC, cmd.Auth.Google.CID = "csec", "cid"
 	cmd.Auth.Facebook.CSEC, cmd.Auth.Facebook.CID = "csec", "cid"
@@ -696,6 +985,7 @@ func prepServerApp(t *testing.T, fn func(o ServerCommand) ServerCommand) (*serve
 	cmd.Auth.Microsoft.CSEC, cmd.Auth.Microsoft.CID = "csec", "cid"
 	cmd.Auth.Twitter.CSEC, cmd.Auth.Twitter.CID = "csec", "cid"
 	cmd.Auth.Patreon.CSEC, cmd.Auth.Patreon.CID = "csec", "cid"
+	cmd.Auth.Discord.CSEC, cmd.Auth.Discord.CID = "csec", "cid"
 	cmd.Auth.Telegram = true
 	cmd.Telegram.Token = "token"
 	cmd.Auth.Email.Enable = true
@@ -716,23 +1006,37 @@ func prepServerApp(t *testing.T, fn func(o ServerCommand) ServerCommand) (*serve
 	cmd.RestrictedNames = []string{"umputun", "bobuk"}
 	cmd.emailMsgTemplatePath = "../../templates/email_reply.html.tmpl"
 	cmd.emailVerificationTemplatePath = "../../templates/email_confirmation_subscription.html.tmpl"
+
 	cmd = fn(cmd)
+	// as is uses port, call it after fn which could set it
+	cmd.Store.Bolt.Path = fmt.Sprintf("/tmp/%d", cmd.Port)
 
-	os.Remove(cmd.Store.Bolt.Path + "/remark.db")
+	app, ctx, cancel := createAppFromCmd(t, cmd)
 
-	return createAppFromCmd(t, cmd)
+	// cleanup the remark.db file after context is canceled
+	go func() {
+		<-ctx.Done()
+		os.RemoveAll(cmd.Store.Bolt.Path)
+		os.RemoveAll(cmd.Avatar.FS.Path)
+
+	}()
+
+	return app, ctx, cancel
 }
 
 func createAppFromCmd(t *testing.T, cmd ServerCommand) (*serverApp, context.Context, context.CancelFunc) {
-	app, err := cmd.newServerApp(context.Background())
-	require.NoError(t, err)
-
 	ctx, cancel := context.WithCancel(context.Background())
-	rand.Seed(time.Now().UnixNano())
+	app, err := cmd.newServerApp(ctx)
+	require.NoError(t, err)
 	return app, ctx, cancel
 }
 
 func TestMain(m *testing.M) {
 	// ignore is added only for GitHub Actions, can't reproduce locally
-	goleak.VerifyTestMain(m, goleak.IgnoreTopFunction("net/http.(*Server).Shutdown"))
+	goleak.VerifyTestMain(
+		m,
+		goleak.IgnoreTopFunction("net/http.(*Server).Shutdown"),
+		// this will be fixed in https://github.com/hashicorp/golang-lru/issues/159
+		goleak.IgnoreTopFunction("github.com/hashicorp/golang-lru/v2/expirable.NewLRU[...].func1"),
+	)
 }

backend/app/cmd/testdata/apple-bad.p8

@@ -0,0 +1,16 @@
+-----BEGIN PRIVATE KEY-----
+MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKNwapOQ6rQJHetP
+HRlJBIh1OsOsUBiXb3rXXE3xpWAxAha0MH+UPRblOko+5T2JqIb+xKf9Vi3oTM3t
+KvffaOPtzKXZauscjq6NGzA3LgeiMy6q19pvkUUOlGYK6+Xfl+B7Xw6+hBMkQuGE
+nUS8nkpR5mK4ne7djIyfHFfMu4ptAgMBAAECgYA+s0PPtMq1osG9oi4xoxeAGikf
+JB3eMUptP+2DYW7mRibc+ueYKhB9lhcUoKhlQUhL8bUUFVZYakP8xD21thmQqnC4
+f63asad0ycteJMLb3r+z26LHuCyOdPg1pyLk3oQ32lVQHBCYathRMcVznxOG16VK
+I8BFfstJTaJu0lK/wQJBANYFGusBiZsJQ3utrQMVPpKmloO2++4q1v6ZR4puDQHx
+TjLjAIgrkYfwTJBLBRZxec0E7TmuVQ9uJ+wMu/+7zaUCQQDDf2xMnQqYknJoKGq+
+oAnyC66UqWC5xAnQS32mlnJ632JXA0pf9pb1SXAYExB1p9Dfqd3VAwQDwBsDDgP6
+HD8pAkEA0lscNQZC2TaGtKZk2hXkdcH1SKru/g3vWTkRHxfCAznJUaza1fx0wzdG
+GcES1Bdez0tbW4llI5By/skZc2eE3QJAFl6fOskBbGHde3Oce0F+wdZ6XIJhEgCP
+iukIcKZoZQzoiMJUoVRrA5gqnmaYDI5uRRl/y57zt6YksR3KcLUIuQJAd242M/WF
+6YAZat3q/wEeETeQq1wrooew+8lHl05/Nt0cCpV48RGEhJ83pzBm3mnwHf8lTBJH
+x6XroMXsmbnsEw==
+-----END PRIVATE KEY-----

backend/app/cmd/testdata/apple.p8

@@ -0,0 +1,6 @@
+-----BEGIN PRIVATE KEY-----
+MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQgGH2MylyZjjRdauTk
+xxXW6p8VSHqIeVRRKSJPg1xn6+KgCgYIKoZIzj0DAQehRANCAAS/mNzQ7aBbIBr3
+DiHiJGIDEzi6+q3mmyhH6ZWQWFdFei2qgdyM1V6qtRPVq+yHBNSBebbR4noE/IYO
+hMdWYrKn
+-----END PRIVATE KEY-----

backend/app/cmd/web/index.html

@@ -0,0 +1 @@
+This stub page would be replaced by the frontend statically built HTML during the Docker image build.

backend/app/main.go

@@ -1,6 +1,7 @@
 package main
 
 import (
+	"errors"
 	"fmt"
 	"os"
 	"os/signal"
@@ -8,7 +9,7 @@ import (
 	"syscall"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/umputun/go-flags"
+	"github.com/jessevdk/go-flags"
 
 	"github.com/umputun/remark42/backend/app/cmd"
 )
@@ -23,7 +24,8 @@ type Opts struct {
 	CleanupCmd cmd.CleanupCommand `command:"cleanup"`
 	RemapCmd   cmd.RemapCommand   `command:"remap"`
 
-	RemarkURL    string `long:"url" env:"REMARK_URL" required:"true" description:"url to remark"`
+	RemarkURL string `long:"url" env:"REMARK_URL" required:"true" description:"url to remark"`
+	// SharedSecret is only used in server command, but defined for all commands for historical reasons
 	SharedSecret string `long:"secret" env:"SECRET" required:"true" description:"the shared secret key used to sign JWT, should be a random, long, hard-to-guess string"`
 
 	Dbg bool `long:"dbg" env:"DEBUG" description:"debug mode"`
@@ -45,13 +47,7 @@ func main() {
 			SharedSecret: opts.SharedSecret,
 			Revision:     revision,
 		})
-		for _, entry := range c.HandleDeprecatedFlags() {
+		logDeprecatedParams(c.HandleDeprecatedFlags())
-			deprecationNote := fmt.Sprintf("[WARN] --%s is deprecated since v%s and will be removed in the future", entry.Old, entry.Version)
-			if entry.New != "" {
-				deprecationNote += fmt.Sprintf(", please use --%s instead", entry.New)
-			}
-			log.Print(deprecationNote)
-		}
 		err := c.Execute(args)
 		if err != nil {
 			log.Printf("[ERROR] failed with %+v", err)
@@ -60,11 +56,11 @@ func main() {
 	}
 
 	if _, err := p.Parse(); err != nil {
-		if flagsErr, ok := err.(*flags.Error); ok && flagsErr.Type == flags.ErrHelp {
+		var flagsErr *flags.Error
+		if errors.As(err, &flagsErr) && flagsErr.Type == flags.ErrHelp {
 			os.Exit(0)
-		} else {
-			os.Exit(1)
 		}
+		os.Exit(1)
 	}
 }
 
@@ -76,18 +72,31 @@ func setupLog(dbg bool) {
 	log.Setup(log.Msec, log.LevelBraces)
 }
 
+// logs usual and "collision" deprecated parameters
+func logDeprecatedParams(params []cmd.DeprecatedFlag) {
+	for _, entry := range params {
+		var deprecationNote string
+		if entry.Collision {
+			deprecationNote = fmt.Sprintf("[ERROR] deprecated --%s and new --%s options are set to different values, old one is ignored: please remove it", entry.Old, entry.New)
+		} else {
+			deprecationNote = fmt.Sprintf("[WARN] --%s is deprecated since v%s and will be removed in the future", entry.Old, entry.Version)
+			if entry.New != "" {
+				deprecationNote += fmt.Sprintf(", please use --%s instead", entry.New)
+			}
+		}
+		log.Print(deprecationNote)
+	}
+}
+
 // getDump reads runtime stack and returns as a string
 func getDump() string {
 	maxSize := 5 * 1024 * 1024
 	stacktrace := make([]byte, maxSize)
-	length := runtime.Stack(stacktrace, true)
+	length := min(runtime.Stack(stacktrace, true), maxSize)
-	if length > maxSize {
-		length = maxSize
-	}
 	return string(stacktrace[:length])
 }
 
-// nolint:gochecknoinits // can't avoid it in this place
+//nolint:gochecknoinits // can't avoid it in this place
 func init() {
 	// catch SIGQUIT and print stack traces
 	sigChan := make(chan os.Signal, 1)

backend/app/main_test.go

@@ -3,7 +3,6 @@ package main
 import (
 	"fmt"
 	"io"
-	"io/ioutil"
 	"math/rand"
 	"net"
 	"net/http"
@@ -22,8 +21,7 @@ import (
 )
 
 func Test_Main(t *testing.T) {
-
+	dir, err := os.MkdirTemp(os.TempDir(), "remark42")
-	dir, err := ioutil.TempDir(os.TempDir(), "remark42")
 	require.NoError(t, err)
 	defer os.RemoveAll(dir)
 
@@ -54,19 +52,19 @@ func Test_Main(t *testing.T) {
 	resp, err := http.Get(fmt.Sprintf("http://localhost:%d/api/v1/ping", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	body, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.Equal(t, "pong", string(body))
 }
 
 func TestMain_WithWebhook(t *testing.T) {
-	dir, err := ioutil.TempDir(os.TempDir(), "remark42")
+	dir, err := os.MkdirTemp(os.TempDir(), "remark42")
 	require.NoError(t, err)
 	defer os.RemoveAll(dir)
 
 	var webhookSent int32
-	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+	ts := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
 		atomic.StoreInt32(&webhookSent, 1)
 		assert.Equal(t, "application/json", r.Header.Get("Content-Type"))
 
@@ -100,9 +98,6 @@ func TestMain_WithWebhook(t *testing.T) {
 	finished := make(chan struct{})
 	go func() {
 		main()
-		assert.Eventually(t, func() bool {
-			return atomic.LoadInt32(&webhookSent) == int32(1)
-		}, time.Second, 100*time.Millisecond, "webhook was not sent")
 		close(finished)
 	}()
 
@@ -118,19 +113,24 @@ func TestMain_WithWebhook(t *testing.T) {
 		strings.NewReader(`{"text": "env test", "locator":{"url": "https://radio-t.com", "site": "remark"}}`))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 201, resp.StatusCode)
+	assert.Equal(t, http.StatusCreated, resp.StatusCode)
+
+	// wait for webhook to be sent before shutting down
+	assert.Eventually(t, func() bool {
+		return atomic.LoadInt32(&webhookSent) == int32(1)
+	}, time.Second, 100*time.Millisecond, "webhook was not sent")
 }
 
 func TestGetDump(t *testing.T) {
 	dump := getDump()
-	assert.True(t, strings.Contains(dump, "goroutine"))
+	assert.Contains(t, dump, "goroutine")
-	assert.True(t, strings.Contains(dump, "[running]"))
+	assert.Contains(t, dump, "[running]")
-	assert.True(t, strings.Contains(dump, "backend/app/main.go"))
+	assert.Contains(t, dump, "backend/app/main.go")
 	t.Logf("\n dump: %s", dump)
 }
 
 func chooseRandomUnusedPort() (port int) {
-	for i := 0; i < 10; i++ {
+	for range 10 {
 		port = 40000 + int(rand.Int31n(10000))
 		if ln, err := net.Listen("tcp", fmt.Sprintf(":%d", port)); err == nil {
 			_ = ln.Close()
@@ -143,7 +143,8 @@ func chooseRandomUnusedPort() (port int) {
 func waitForHTTPServerStart(port int) {
 	// wait for up to 10 seconds for server to start before returning it
 	client := http.Client{Timeout: time.Second}
-	for i := 0; i < 100; i++ {
+	defer client.CloseIdleConnections()
+	for range 100 {
 		time.Sleep(time.Millisecond * 100)
 		if resp, err := client.Get(fmt.Sprintf("http://localhost:%d", port)); err == nil {
 			_ = resp.Body.Close()
@@ -158,5 +159,7 @@ func TestMain(m *testing.M) {
 		m,
 		goleak.IgnoreTopFunction("github.com/umputun/remark42/backend/app.init.0.func1"),
 		goleak.IgnoreTopFunction("net/http.(*Server).Shutdown"),
+		// this will be fixed in https://github.com/hashicorp/golang-lru/issues/159
+		goleak.IgnoreTopFunction("github.com/hashicorp/golang-lru/v2/expirable.NewLRU[...].func1"),
 	)
 }

backend/app/migrator/backup.go

@@ -10,7 +10,6 @@ import (
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 )
 
 // AutoBackup struct handles daily backups params for siteID
@@ -48,20 +47,20 @@ func (ab AutoBackup) Do(ctx context.Context) {
 func (ab AutoBackup) makeBackup() (string, error) {
 	log.Printf("[DEBUG] make backup for %s", ab.SiteID)
 	backupFile := fmt.Sprintf("%s/backup-%s-%s.gz", ab.BackupLocation, ab.SiteID, time.Now().Format("20060102"))
-	fh, err := os.Create(backupFile)
+	fh, err := os.Create(backupFile) //nolint:gosec // harmless
 	if err != nil {
-		return "", errors.Wrapf(err, "can't create backup file %s", backupFile)
+		return "", fmt.Errorf("can't create backup file %s: %w", backupFile, err)
 	}
 	gz := gzip.NewWriter(fh)
 
 	if _, err = ab.Exporter.Export(gz, ab.SiteID); err != nil {
-		return "", errors.Wrapf(err, "export failed for %s", ab.SiteID)
+		return "", fmt.Errorf("export failed for %s: %w", ab.SiteID, err)
 	}
 	if err = gz.Close(); err != nil {
-		return "", errors.Wrapf(err, "can't close gz for %s", backupFile)
+		return "", fmt.Errorf("can't close gz for %s: %w", backupFile, err)
 	}
 	if err = fh.Close(); err != nil {
-		return "", errors.Wrapf(err, "can't close file handler for %s", backupFile)
+		return "", fmt.Errorf("can't close file handler for %s: %w", backupFile, err)
 	}
 	log.Printf("[DEBUG] created backup file %s", backupFile)
 	return backupFile, nil
@@ -84,7 +83,7 @@ func (ab AutoBackup) removeOldBackupFiles() {
 			backFiles = append(backFiles, info)
 		}
 	}
-	sort.Slice(backFiles, func(i int, j int) bool { return backFiles[i].Name() < backFiles[j].Name() })
+	sort.Slice(backFiles, func(i, j int) bool { return backFiles[i].Name() < backFiles[j].Name() })
 
 	if len(backFiles) > ab.KeepMax {
 		for i := 0; i < len(backFiles)-ab.KeepMax; i++ {

backend/app/migrator/backup_test.go

@@ -6,6 +6,7 @@ import (
 	"io"
 	"os"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	"github.com/stretchr/testify/assert"
@@ -59,19 +60,21 @@ func TestBackup_Do(t *testing.T) {
 	defer os.RemoveAll(loc)
 	assert.NoError(t, os.MkdirAll(loc, 0o700))
 
-	ctx, cancel := context.WithCancel(context.Background())
+	synctest.Test(t, func(t *testing.T) {
-	go func() {
+		ctx, cancel := context.WithCancel(context.Background())
-		time.Sleep(time.Second)
+		go func() {
-		cancel()
+			time.Sleep(time.Second)
-	}()
+			cancel()
+		}()
 
-	bk := AutoBackup{BackupLocation: loc, SiteID: "site1", KeepMax: 3, Exporter: &mockExporter{}, Duration: 600 * time.Millisecond}
+		bk := AutoBackup{BackupLocation: loc, SiteID: "site1", KeepMax: 3, Exporter: &mockExporter{}, Duration: 600 * time.Millisecond}
-	bk.Do(ctx)
+		bk.Do(ctx)
 
-	expFile := fmt.Sprintf("/tmp/remark-backups.test/backup-site1-%s.gz", time.Now().Format("20060102"))
+		expFile := fmt.Sprintf("/tmp/remark-backups.test/backup-site1-%s.gz", time.Now().Format("20060102"))
-	fi, err := os.Lstat(expFile)
+		fi, err := os.Lstat(expFile)
-	assert.NoError(t, err)
+		assert.NoError(t, err)
-	assert.Equal(t, int64(52), fi.Size())
+		assert.Equal(t, int64(52), fi.Size())
+	})
 }
 
 type mockExporter struct{}

backend/app/migrator/commento.go

@@ -2,10 +2,11 @@ package migrator
 
 import (
 	"encoding/json"
+	"fmt"
 	"io"
+	"net/url"
 	"time"
 
-	"github.com/pkg/errors"
 	"github.com/umputun/remark42/backend/app/store"
 
 	log "github.com/go-pkgz/lgr"
@@ -47,7 +48,7 @@ type commentoCommenter struct {
 	Link         string    `json:"link"`
 	Photo        string    `json:"photo"`
 	Provider     string    `json:"provider,omitempty"`
-	JoinDate     time.Time `json:"joinDate,omitempty"`
+	JoinDate     time.Time `json:"joinDate"`
 	IsModerator  bool      `json:"isModerator"`
 }
 
@@ -68,9 +69,9 @@ func (d *Commento) Import(r io.Reader, siteID string) (size int, err error) {
 	}
 
 	if failed > 0 {
-		err = errors.Errorf("failed to save %d comments", failed)
+		err = fmt.Errorf("failed to save %d comments", failed)
 		if passed == 0 {
-			err = errors.New("import failed")
+			err = fmt.Errorf("import failed")
 		}
 	}
 
@@ -85,7 +86,6 @@ func (d *Commento) convert(r io.Reader, siteID string) (ch chan store.Comment) {
 	decoder := json.NewDecoder(r)
 
 	go func() {
-
 		var exportedData commentoExport
 		err := decoder.Decode(&exportedData)
 		if err != nil {
@@ -101,6 +101,11 @@ func (d *Commento) convert(r io.Reader, siteID string) (ch chan store.Comment) {
 			}
 		}
 
+		usersMap["anonymous"] = store.User{
+			Name: "Anonymous",
+			ID:   "commento_" + store.EncodeID("anonymous"),
+		}
+
 		for _, comment := range exportedData.Comments {
 			u, ok := usersMap[comment.CommenterHex]
 			if !ok {
@@ -111,16 +116,28 @@ func (d *Commento) convert(r io.Reader, siteID string) (ch chan store.Comment) {
 				continue
 			}
 
+			parentID := comment.ParentHex
+			// comments with ParentHex == "root" are top-level comments
+			if parentID == "root" {
+				parentID = ""
+			}
+
+			commentURL, e := url.JoinPath("https://", comment.Domain, comment.Path)
+			if e != nil {
+				log.Printf("[WARN] can't construct comment URL in commento import, %s", err.Error())
+			}
+			log.Printf("[ERROR] commentoURL: %s", commentURL)
+
 			c := store.Comment{
 				ID: comment.CommentHex,
 				Locator: store.Locator{
-					URL:    comment.Path,
+					URL:    commentURL,
 					SiteID: siteID,
 				},
 				User:      u,
 				Text:      comment.Markdown,
 				Timestamp: comment.CreationDate,
-				ParentID:  comment.ParentHex,
+				ParentID:  parentID,
 				Imported:  true,
 			}
 

backend/app/migrator/commento_test.go

@@ -7,11 +7,12 @@ import (
 
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
+	bolt "go.etcd.io/bbolt"
+
 	"github.com/umputun/remark42/backend/app/store"
 	"github.com/umputun/remark42/backend/app/store/admin"
 	"github.com/umputun/remark42/backend/app/store/engine"
 	"github.com/umputun/remark42/backend/app/store/service"
-	bolt "go.etcd.io/bbolt"
 )
 
 func TestCommento_Import(t *testing.T) {
@@ -26,11 +27,11 @@ func TestCommento_Import(t *testing.T) {
 	require.NoError(t, err)
 	size, err := d.Import(fh, "test")
 	assert.NoError(t, err)
-	assert.Equal(t, 2, size)
+	assert.Equal(t, 3, size)
 
 	last, err := dataStore.Last("test", 10, time.Time{}, adminUser)
 	assert.NoError(t, err)
-	require.Equal(t, 2, len(last), "2 comments imported")
+	require.Equal(t, 3, len(last), "3 comments imported")
 
 	t.Log(last[0])
 
@@ -43,11 +44,24 @@ func TestCommento_Import(t *testing.T) {
 	assert.Equal(t, "commento_35369aeb6ac5255de30410a0f86dc71eb9c6d0ca", c.User.ID)
 	assert.True(t, c.Imported)
 
+	c = last[2] // anonymous comment
+	assert.Equal(t, "Example comment created by user.", c.Text)
+	assert.Equal(t, "e7069a7dfcfaed43caf62300a9b0edb1c124ad79d0f5887c93649c15d7f69945", c.ID)
+	assert.Equal(t, "", c.ParentID)
+	assert.Equal(t, store.Locator{SiteID: "test", URL: "https://example.com/blog/post/2"}, c.Locator)
+	assert.Equal(t, "Anonymous", c.User.Name)
+	assert.Equal(t, "commento_0a92fab3230134cca6eadd9898325b9b2ae67998", c.User.ID)
+	assert.True(t, c.Imported)
+
 	posts, err := dataStore.List("test", 0, 0)
 	assert.NoError(t, err)
-	assert.Equal(t, 1, len(posts), "1 post")
+	assert.Equal(t, 2, len(posts), "2 posts")
 
 	count, err := dataStore.Count(store.Locator{SiteID: "test", URL: "https://example.com/blog/post/1"})
 	assert.NoError(t, err)
 	assert.Equal(t, 2, count)
+
+	count, err = dataStore.Count(store.Locator{SiteID: "test", URL: "https://example.com/blog/post/2"})
+	assert.NoError(t, err)
+	assert.Equal(t, 1, count)
 }

backend/app/migrator/disqus.go

@@ -2,12 +2,12 @@ package migrator
 
 import (
 	"encoding/xml"
+	"fmt"
 	"io"
 	"strings"
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store"
 )
@@ -68,9 +68,9 @@ func (d *Disqus) Import(r io.Reader, siteID string) (size int, err error) {
 	}
 
 	if failed > 0 {
-		err = errors.Errorf("failed to save %d comments", failed)
+		err = fmt.Errorf("failed to save %d comments", failed)
 		if passed == 0 {
-			err = errors.New("import failed")
+			err = fmt.Errorf("import failed")
 		}
 	}
 
@@ -82,7 +82,6 @@ func (d *Disqus) Import(r io.Reader, siteID string) (size int, err error) {
 // convert disqus stream (xml) from reader and fill channel of comments.
 // runs async and closes channel on completion.
 func (d *Disqus) convert(r io.Reader, siteID string) (ch chan store.Comment) {
-
 	postsMap := map[string]string{} // tid:url
 	decoder := xml.NewDecoder(r)
 	commentsCh := make(chan store.Comment)
@@ -176,7 +175,7 @@ func (d *Disqus) convert(r io.Reader, siteID string) (ch chan store.Comment) {
 
 func (*Disqus) cleanText(text string) string {
 	text = strings.TrimSpace(text)
-	text = strings.Replace(text, "\n", "", -1)
+	text = strings.ReplaceAll(text, "\n", "")
-	text = strings.Replace(text, "\t", "", -1)
+	text = strings.ReplaceAll(text, "\t", "")
 	return text
 }

backend/app/migrator/disqus_test.go

@@ -122,7 +122,7 @@ func TestDisqus_Convert(t *testing.T) {
 	require.NoError(t, err)
 	ch := d.convert(fh, "test")
 
-	res := []store.Comment{}
+	res := make([]store.Comment, 0, 4)
 	for comment := range ch {
 		res = append(res, comment)
 	}

backend/app/migrator/mapper.go

@@ -1,7 +1,7 @@
 package migrator
 
 import (
-	"errors"
+	"fmt"
 	"io"
 	"strings"
 )
@@ -38,11 +38,11 @@ func (u *URLMapper) loadRules(reader io.Reader) error {
 
 	u.rules = make(map[string]string)
 
-	for _, row := range strings.Split(rulesText, "\n") {
+	for row := range strings.SplitSeq(rulesText, "\n") {
 		row = strings.TrimSpace(row)
 		urls := strings.Split(row, " ")
 		if len(urls) != 2 {
-			return errors.New("bad row " + row)
+			return fmt.Errorf("bad row %s", row)
 		}
 
 		from, to := strings.TrimSpace(urls[0]), strings.TrimSpace(urls[1])
@@ -64,8 +64,8 @@ func (u *URLMapper) URL(url string) string {
 		}
 		oldURL = strings.TrimSuffix(oldURL, "*")
 		newURL = strings.TrimSuffix(newURL, "*")
-		if strings.HasPrefix(url, oldURL) {
+		if after, ok := strings.CutPrefix(url, oldURL); ok {
-			return newURL + strings.TrimPrefix(url, oldURL)
+			return newURL + after
 		}
 	}
 	// search failed, return given url

backend/app/migrator/migrator.go

@@ -4,11 +4,11 @@
 package migrator
 
 import (
+	"fmt"
 	"io"
 	"os"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store"
 	"github.com/umputun/remark42/backend/app/store/service"
@@ -38,7 +38,7 @@ type MapperMaker func(reader io.Reader) (Mapper, error)
 type Store interface {
 	Create(comment store.Comment) (commentID string, err error)
 	Find(locator store.Locator, sort string, user store.User) ([]store.Comment, error)
-	List(siteID string, limit int, skip int) ([]store.PostInfo, error)
+	List(siteID string, limit, skip int) ([]store.PostInfo, error)
 	DeleteAll(siteID string) error
 	Metas(siteID string) (umetas []service.UserMetaData, pmetas []service.PostMetaData, err error)
 	SetMetas(siteID string, umetas []service.UserMetaData, pmetas []service.PostMetaData) error
@@ -69,12 +69,12 @@ func ImportComments(p ImportParams) (int, error) {
 	case "native":
 		importer = &Native{DataStore: p.DataStore}
 	default:
-		return 0, errors.Errorf("unsupported import provider %s", p.Provider)
+		return 0, fmt.Errorf("unsupported import provider %s", p.Provider)
 	}
 
 	fh, err := os.Open(p.InputFile)
 	if err != nil {
-		return 0, errors.Wrapf(err, "can't open import file %s", p.InputFile)
+		return 0, fmt.Errorf("can't open import file %s: %w", p.InputFile, err)
 	}
 
 	defer func() { //nolint:gosec // false positive on defer without error check when it's checked here

backend/app/migrator/migrator_test.go

@@ -77,11 +77,11 @@ func TestMigrator_ImportCommento(t *testing.T) {
 		Provider:  "commento",
 	})
 	assert.NoError(t, err)
-	assert.Equal(t, 2, size)
+	assert.Equal(t, 3, size)
 
 	last, err := dataStore.Last("test", 10, time.Time{}, store.User{})
 	assert.NoError(t, err)
-	assert.Equal(t, 2, len(last), "2 comments imported")
+	assert.Equal(t, 3, len(last), "3 comments imported")
 }
 
 func TestMigrator_ImportNative(t *testing.T) {

backend/app/migrator/native.go

@@ -4,12 +4,12 @@ import (
 	"bytes"
 	"context"
 	"encoding/json"
+	"fmt"
 	"io"
 	"sync/atomic"
 
 	log "github.com/go-pkgz/lgr"
 	"github.com/go-pkgz/syncs"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store"
 	"github.com/umputun/remark42/backend/app/store/service"
@@ -35,9 +35,8 @@ type meta struct {
 // Export all comments to writer as json strings. Each comment is one string, separated by "\n"
 // The final file is a valid json
 func (n *Native) Export(w io.Writer, siteID string) (size int, err error) {
-
 	if err = n.exportMeta(siteID, w); err != nil {
-		return 0, errors.Wrapf(err, "failed to export meta for site %s", siteID)
+		return 0, fmt.Errorf("failed to export meta for site %s: %w", siteID, err)
 	}
 
 	topics, err := n.DataStore.List(siteID, 0, 0)
@@ -55,16 +54,15 @@ func (n *Native) Export(w io.Writer, siteID string) (size int, err error) {
 		}
 
 		for _, comment := range comments {
-
 			buf := &bytes.Buffer{}
 			enc := json.NewEncoder(buf)
 			enc.SetEscapeHTML(false)
 
 			if err = enc.Encode(comment); err != nil {
-				return commentsCount, errors.Wrapf(err, "can't marshal %v", comments)
+				return commentsCount, fmt.Errorf("can't marshal %v: %w", comments, err)
 			}
 			if _, err = w.Write(buf.Bytes()); err != nil {
-				return commentsCount, errors.Wrap(err, "can't write comment data")
+				return commentsCount, fmt.Errorf("can't write comment data: %w", err)
 			}
 			commentsCount++
 		}
@@ -78,11 +76,11 @@ func (n *Native) exportMeta(siteID string, w io.Writer) (err error) {
 	m := meta{Version: nativeVersion}
 	m.Users, m.Posts, err = n.DataStore.Metas(siteID)
 	if err != nil {
-		return errors.Wrap(err, "can't get meta")
+		return fmt.Errorf("can't get meta: %w", err)
 	}
 
 	if err = json.NewEncoder(w).Encode(m); err != nil {
-		return errors.Wrap(err, "can't encode meta")
+		return fmt.Errorf("can't encode meta: %w", err)
 	}
 	return nil
 }
@@ -133,11 +131,11 @@ func (n *Native) Import(reader io.Reader, siteID string) (size int, err error) {
 	m := meta{}
 	dec := json.NewDecoder(reader)
 	if err = dec.Decode(&m); err != nil {
-		return 0, errors.Wrapf(err, "failed to import meta for site %s", siteID)
+		return 0, fmt.Errorf("failed to import meta for site %s: %w", siteID, err)
 	}
 
 	if m.Version != nativeVersion && m.Version != 0 { // this version allows back compatibility with 0 version
-		return 0, errors.Errorf("unexpected import file version %d", m.Version)
+		return 0, fmt.Errorf("unexpected import file version %d", m.Version)
 	}
 
 	if e := n.DataStore.DeleteAll(siteID); e != nil {
@@ -180,13 +178,12 @@ func (n *Native) Import(reader io.Reader, siteID string) (size int, err error) {
 				log.Printf("[DEBUG] imported %d comments", num)
 			}
 		})
-
 	}
 
 	grp.Wait()
 
 	if failed > 0 {
-		return int(comments), errors.Errorf("failed to save %d comments", failed)
+		return int(comments), fmt.Errorf("failed to save %d comments", failed)
 	}
 	log.Printf("[INFO] imported %d comments from %d records", comments, total)
 

backend/app/migrator/native_test.go

@@ -153,7 +153,6 @@ func TestNative_ImportWrongVersion(t *testing.T) {
 	size, err := r.Import(strings.NewReader(inp), "radio-t")
 	assert.EqualError(t, err, "unexpected import file version 2")
 	assert.Equal(t, 0, size)
-
 }
 func TestNative_ImportManyWithError(t *testing.T) {
 	b, teardown := prep(t) // write 2 comments
@@ -163,8 +162,8 @@ func TestNative_ImportManyWithError(t *testing.T) {
 
 	buf := &bytes.Buffer{}
 	buf.WriteString(`{"version":1, "users":[], "posts":[]}` + "\n")
-	for i := 0; i < 100; i++ {
+	for i := range 100 {
-		buf.WriteString(fmt.Sprintf(goodRec, i))
+		fmt.Fprintf(buf, goodRec, i)
 	}
 	buf.WriteString("{}\n")
 	buf.WriteString("{}\n")
@@ -181,7 +180,6 @@ func TestNative_ImportManyWithError(t *testing.T) {
 
 // makes new boltdb, put two records
 func prep(t *testing.T) (ds *service.DataStore, teardown func()) {
-
 	testDB := fmt.Sprintf("/tmp/migrator-%d.db", rand.Intn(999999999))
 
 	boltStore, err := engine.NewBoltDB(bolt.Options{}, engine.BoltSite{SiteID: "radio-t", FileName: testDB})

backend/app/migrator/testdata/commento.json

@@ -4,7 +4,7 @@
 	{
 	  "commentHex": "e7069a7dfcfaed43caf62300a9b0edb1c124ad79d0f5887c93649c15d7f69945",
 	  "domain": "example.com",
-	  "url": "https://example.com/blog/post/1",
+	  "url": "/blog/post/2",
 	  "commenterHex": "anonymous",
 	  "markdown": "Example comment created by user.",
 	  "html": "",
@@ -18,7 +18,7 @@
 	{
 	  "commentHex": "7d77e39fcd813241d6281478cc8f21ab5f807d043c750bc1a936bc23b34fb854",
 	  "domain": "example.com",
-	  "url": "https://example.com/blog/post/1",
+	  "url": "/blog/post/1",
 	  "commenterHex": "a1ac58ed1146bd7fe3feff6a7276f73955c3bfd23cacee00e2e0a7a89b1a8c10",
 	  "markdown": "Example 2 comment created by user.",
 	  "html": "",
@@ -32,7 +32,7 @@
 	{
 	  "commentHex": "ea5f7bcd6ac9bb7b657f7d0569831104e1bcf9c253d03c1e16bf9654c49a5ce9",
 	  "domain": "example.com",
-	  "url": "https://example.com/blog/post/1",
+	  "url": "/blog/post/1",
 	  "commenterHex": "bd1290ab5c858cf2a05903c2a9a61fd63399c6635db38cc6597002195e22e061",
 	  "markdown": "Great reply!",
 	  "html": "",

backend/app/migrator/wordpress.go

@@ -2,12 +2,12 @@ package migrator
 
 import (
 	"encoding/xml"
+	"fmt"
 	"html"
 	"io"
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/store"
 )
@@ -16,7 +16,8 @@ const wpTimeLayout = "2006-01-02 15:04:05"
 
 // WordPress implements Importer from WP xml
 type WordPress struct {
-	DataStore Store
+	DataStore                  Store
+	DisableFancyTextFormatting bool
 }
 
 type wpItem struct {
@@ -60,7 +61,6 @@ func (w *WordPress) Convert(text string) string {
 
 // Import comments from WP and save to store
 func (w *WordPress) Import(r io.Reader, siteID string) (size int, err error) {
-
 	if e := w.DataStore.DeleteAll(siteID); e != nil {
 		return 0, e
 	}
@@ -76,9 +76,9 @@ func (w *WordPress) Import(r io.Reader, siteID string) (size int, err error) {
 	}
 
 	if failed > 0 {
-		err = errors.Errorf("failed to save %d comments", failed)
+		err = fmt.Errorf("failed to save %d comments", failed)
 		if passed == 0 {
-			err = errors.New("import failed")
+			err = fmt.Errorf("import failed")
 		}
 	}
 
@@ -88,7 +88,6 @@ func (w *WordPress) Import(r io.Reader, siteID string) (size int, err error) {
 }
 
 func (w *WordPress) convert(r io.Reader, siteID string) chan store.Comment {
-
 	decoder := xml.NewDecoder(r)
 	commentsCh := make(chan store.Comment)
 
@@ -140,7 +139,7 @@ func (w *WordPress) convert(r io.Reader, siteID string) chan store.Comment {
 								ParentID:  comment.PID,
 								Imported:  true,
 							}
-							commentsCh <- commentFormatter.Format(c)
+							commentsCh <- commentFormatter.Format(c, w.DisableFancyTextFormatting)
 							stats.inpComments++
 							if stats.inpComments%1000 == 0 {
 								log.Printf("[DEBUG] processed %d comments", stats.inpComments)

backend/app/migrator/wordpress_test.go

@@ -24,7 +24,7 @@ func TestWordPress_Import(t *testing.T) {
 
 	dataStore := service.DataStore{Engine: b, AdminStore: admin.NewStaticStore("12345", nil, []string{}, "")}
 	defer dataStore.Close()
-	wp := WordPress{DataStore: &dataStore}
+	wp := WordPress{DataStore: &dataStore, DisableFancyTextFormatting: false}
 	size, err := wp.Import(strings.NewReader(xmlTestWP), siteID)
 	assert.NoError(t, err)
 	assert.Equal(t, 3, size)
@@ -41,7 +41,7 @@ func TestWordPress_Import(t *testing.T) {
 	assert.Equal(t, "e8b1e92bbcf5b9bb88472f9bdb82d1b8c7ed39d6", c.User.IP)
 	ts, _ := time.Parse(wpTimeLayout, "2010-08-18 15:19:14")
 	assert.Equal(t, ts, c.Timestamp)
-	assert.Equal(t, c.Text, "<p>Mekkatorque was over in that tent up to the right</p>\n")
+	assert.Equal(t, "<p>«Mekkatorque» was over in that tent up to the right</p>\n", c.Text)
 	assert.True(t, c.Imported)
 
 	posts, err := dataStore.List(siteID, 0, 0)
@@ -54,13 +54,25 @@ func TestWordPress_Import(t *testing.T) {
 	count, err := dataStore.Count(store.Locator{URL: "https://realmenweardress.es/2010/07/do-you-rp/", SiteID: siteID})
 	assert.NoError(t, err)
 	assert.Equal(t, 3, count)
+
+	// test with DisableFancyTextFormatting
+	wp = WordPress{DataStore: &dataStore, DisableFancyTextFormatting: true}
+	size, err = wp.Import(strings.NewReader(xmlTestWP), siteID)
+	assert.NoError(t, err)
+	assert.Equal(t, 3, size)
+
+	last, err = dataStore.Last(siteID, 10, time.Time{}, adminUser)
+	assert.NoError(t, err)
+	require.Equal(t, 3, len(last), "3 comments imported")
+
+	assert.Equal(t, "<p>&#34;Mekkatorque&#34; was over in that tent up to the right</p>\n", last[0].Text)
 }
 
 func TestWordPress_Convert(t *testing.T) {
 	wp := WordPress{}
 	ch := wp.convert(strings.NewReader(xmlTestWP), "testWP")
 
-	comments := []store.Comment{}
+	comments := make([]store.Comment, 0, 3)
 	for c := range ch {
 		comments = append(comments, c)
 	}
@@ -88,7 +100,7 @@ func TestWP_Convert_MD(t *testing.T) {
 	wp := WordPress{}
 	ch := wp.convert(strings.NewReader(xmlTestWPmd), "siteID")
 
-	comments := []store.Comment{}
+	comments := make([]store.Comment, 0, 3)
 	for c := range ch {
 		comments = append(comments, c)
 	}
@@ -247,7 +259,7 @@ var xmlTestWP = `
 			<wp:comment_author_IP><![CDATA[128.243.253.117]]></wp:comment_author_IP>
 			<wp:comment_date><![CDATA[2010-08-18 15:19:14]]></wp:comment_date>
 			<wp:comment_date_gmt><![CDATA[2010-08-18 15:19:14]]></wp:comment_date_gmt>
-			<wp:comment_content><![CDATA[Mekkatorque was over in that tent up to the right]]></wp:comment_content>
+			<wp:comment_content><![CDATA["Mekkatorque" was over in that tent up to the right]]></wp:comment_content>
 			<wp:comment_approved><![CDATA[1]]></wp:comment_approved>
 			<wp:comment_type><![CDATA[]]></wp:comment_type>
 			<wp:comment_parent>13</wp:comment_parent>

backend/app/notify/email.go

@@ -3,20 +3,15 @@ package notify
 import (
 	"bytes"
 	"context"
-	"crypto/tls"
 	"fmt"
-	"io"
+	"net/url"
-	"mime"
-	"mime/quotedprintable"
-	"net"
-	"net/smtp"
 	"text/template"
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/go-pkgz/repeater"
+	ntf "github.com/go-pkgz/notify"
+	"github.com/go-pkgz/repeater/v2"
 	"github.com/hashicorp/go-multierror"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/templates"
 )
@@ -31,53 +26,18 @@ type EmailParams struct {
 	SubscribeURL             string   // full subscribe handler URL
 	UnsubscribeURL           string   // full unsubscribe handler URL
 
-	TokenGenFn func(userID, email, site string) (string, error) // Unsubscribe token generation function
+	TokenGenFn func(userID, email, site string) (string, error) // unsubscribe token generation function
-}
-
-// SMTPParams contain settings for smtp server connection
-type SMTPParams struct {
-	Host     string        // SMTP host
-	Port     int           // SMTP port
-	TLS      bool          // TLS auth
-	Username string        // user name
-	Password string        // password
-	TimeOut  time.Duration // TCP connection timeout
 }
 
 // Email implements notify.Destination for email
 type Email struct {
-	EmailParams
+	*ntf.Email
-	SMTPParams
 
-	smtp       smtpClientCreator
+	EmailParams
 	msgTmpl    *template.Template // parsed request message template
 	verifyTmpl *template.Template // parsed verification message template
 }
 
-// default email client implementation
-type emailClient struct{ smtpClientCreator }
-
-// smtpClient interface defines subset of net/smtp used by email client
-type smtpClient interface {
-	Mail(string) error
-	Auth(smtp.Auth) error
-	Rcpt(string) error
-	Data() (io.WriteCloser, error)
-	Quit() error
-	Close() error
-}
-
-// smtpClientCreator interface defines function for creating new smtpClients
-type smtpClientCreator interface {
-	Create(SMTPParams) (smtpClient, error)
-}
-
-type emailMessage struct {
-	from    string
-	to      string
-	message string
-}
-
 // msgTmplData store data for message from request template execution
 type msgTmplData struct {
 	UserName          string
@@ -113,15 +73,14 @@ const (
 )
 
 // NewEmail makes new Email object, returns error in case of e.MsgTemplate or e.VerificationTemplate parsing error
-func NewEmail(emailParams EmailParams, smtpParams SMTPParams) (*Email, error) {
+func NewEmail(emailParams EmailParams, smtpParams ntf.SMTPParams) (*Email, error) {
 	// set up Email emailParams
-	res := Email{EmailParams: emailParams}
+	if smtpParams.TimeOut <= 0 {
-	res.smtp = &emailClient{}
+		smtpParams.TimeOut = defaultEmailTimeout
-	res.SMTPParams = smtpParams
-	if res.TimeOut <= 0 {
-		res.TimeOut = defaultEmailTimeout
 	}
 
+	res := Email{Email: ntf.NewEmail(smtpParams), EmailParams: emailParams}
+
 	if res.VerificationSubject == "" {
 		res.VerificationSubject = defaultVerificationSubject
 	}
@@ -129,7 +88,7 @@ func NewEmail(emailParams EmailParams, smtpParams SMTPParams) (*Email, error) {
 	// initialize templates
 	err := res.setTemplates()
 	if err != nil {
-		return nil, errors.Wrap(err, "can't set templates")
+		return nil, fmt.Errorf("can't set templates: %w", err)
 	}
 
 	log.Printf("[DEBUG] Create new email notifier for server %s with user %s, timeout=%s",
@@ -141,7 +100,6 @@ func NewEmail(emailParams EmailParams, smtpParams SMTPParams) (*Email, error) {
 func (e *Email) setTemplates() error {
 	var err error
 	var msgTmplFile, verifyTmplFile []byte
-	fs := templates.NewFS()
 
 	if e.VerificationTemplatePath == "" {
 		e.VerificationTemplatePath = defaultEmailVerificationTemplatePath
@@ -151,17 +109,17 @@ func (e *Email) setTemplates() error {
 		e.MsgTemplatePath = defaultEmailTemplatePath
 	}
 
-	if msgTmplFile, err = fs.ReadFile(e.MsgTemplatePath); err != nil {
+	if msgTmplFile, err = templates.Read(e.MsgTemplatePath); err != nil {
-		return errors.Wrapf(err, "can't read message template")
+		return fmt.Errorf("can't read message template: %w", err)
 	}
-	if verifyTmplFile, err = fs.ReadFile(e.VerificationTemplatePath); err != nil {
+	if verifyTmplFile, err = templates.Read(e.VerificationTemplatePath); err != nil {
-		return errors.Wrapf(err, "can't read verification template")
+		return fmt.Errorf("can't read verification template: %w", err)
 	}
 	if e.msgTmpl, err = template.New("msgTmpl").Parse(string(msgTmplFile)); err != nil {
-		return errors.Wrapf(err, "can't parse message template")
+		return fmt.Errorf("can't parse message template: %w", err)
 	}
 	if e.verifyTmpl, err = template.New("verifyTmpl").Parse(string(verifyTmplFile)); err != nil {
-		return errors.Wrapf(err, "can't parse verification template")
+		return fmt.Errorf("can't parse verification template: %w", err)
 	}
 
 	return nil
@@ -173,7 +131,7 @@ func (e *Email) setTemplates() error {
 func (e *Email) Send(ctx context.Context, req Request) error {
 	select {
 	case <-ctx.Done():
-		return errors.Errorf("sending email messages about comment %q aborted due to canceled context", req.Comment.ID)
+		return fmt.Errorf("sending email messages about comment %q aborted due to canceled context", req.Comment.ID)
 	default:
 	}
 
@@ -181,12 +139,16 @@ func (e *Email) Send(ctx context.Context, req Request) error {
 
 	for _, email := range req.Emails {
 		err := e.buildAndSendMessage(ctx, req, email, false)
-		result = multierror.Append(errors.Wrapf(err, "problem sending user email notification to %q", email))
+		if err != nil {
+			result = multierror.Append(fmt.Errorf("problem sending user email notification to %q: %w", email, err))
+		}
 	}
 
 	for _, email := range e.AdminEmails {
 		err := e.buildAndSendMessage(ctx, req, email, true)
-		result = multierror.Append(errors.Wrapf(err, "problem sending admin email notification to %q", email))
+		if err != nil {
+			result = multierror.Append(fmt.Errorf("problem sending admin email notification to %q: %w", email, err))
+		}
 	}
 
 	return result.ErrorOrNil()
@@ -199,10 +161,19 @@ func (e *Email) buildAndSendMessage(ctx context.Context, req Request, email stri
 		return err
 	}
 
-	return repeater.NewDefault(5, time.Millisecond*250).Do(
+	return repeater.NewFixed(5, time.Millisecond*250).Do(
 		ctx,
 		func() error {
-			return e.sendMessage(emailMessage{from: e.From, to: email, message: msg})
+			return e.Email.Send(
+				ctx,
+				fmt.Sprintf("mailto:%s?from=%s&unsubscribeLink=%s&subject=%s",
+					email,
+					url.QueryEscape(e.From),
+					url.QueryEscape(msg.unsubscribeLink),
+					url.QueryEscape(msg.subject),
+				),
+				msg.body,
+			)
 		})
 }
 
@@ -215,7 +186,7 @@ func (e *Email) SendVerification(ctx context.Context, req VerificationRequest) e
 	}
 	select {
 	case <-ctx.Done():
-		return errors.Errorf("sending message to %q aborted due to canceled context", req.User)
+		return fmt.Errorf("sending message to %q aborted due to canceled context", req.User)
 	default:
 	}
 
@@ -225,16 +196,23 @@ func (e *Email) SendVerification(ctx context.Context, req VerificationRequest) e
 		return err
 	}
 
-	return repeater.NewDefault(5, time.Millisecond*250).Do(
+	return repeater.NewFixed(5, time.Millisecond*250).Do(
 		ctx,
 		func() error {
-			return e.sendMessage(emailMessage{from: e.From, to: req.Email, message: msg})
+			return e.Email.Send(
+				ctx,
+				fmt.Sprintf("mailto:%s?from=%s&subject=%s",
+					req.Email,
+					url.QueryEscape(e.From),
+					url.QueryEscape(e.VerificationSubject),
+				),
+				msg,
+			)
 		})
 }
 
 // buildVerificationMessage generates verification email message based on given input
 func (e *Email) buildVerificationMessage(user, email, token, site string) (string, error) {
-	subject := e.VerificationSubject
 	msg := bytes.Buffer{}
 	err := e.verifyTmpl.Execute(&msg, verifyTmplData{
 		User:         user,
@@ -244,13 +222,19 @@ func (e *Email) buildVerificationMessage(user, email, token, site string) (strin
 		SubscribeURL: e.SubscribeURL,
 	})
 	if err != nil {
-		return "", errors.Wrapf(err, "error executing template to build verification message")
+		return "", fmt.Errorf("error executing template to build verification message: %w", err)
 	}
-	return e.buildMessage(subject, msg.String(), email, "text/html", "")
+	return msg.String(), nil
+}
+
+type commentMessage struct {
+	subject         string
+	body            string
+	unsubscribeLink string
 }
 
 // buildMessageFromRequest generates email message based on Request using e.MsgTemplate
-func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool) (string, error) {
+func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool) (commentMessage, error) {
 	subject := "New reply to your comment"
 	if forAdmin {
 		subject = "New comment to your site"
@@ -261,7 +245,7 @@ func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool
 
 	token, err := e.TokenGenFn(req.parent.User.ID, email, req.Comment.Locator.SiteID)
 	if err != nil {
-		return "", errors.Wrapf(err, "error creating token for unsubscribe link")
+		return commentMessage{}, fmt.Errorf("error creating token for unsubscribe link: %w", err)
 	}
 	unsubscribeLink := e.UnsubscribeURL + "?site=" + req.Comment.Locator.SiteID + "&tkn=" + token
 	if forAdmin {
@@ -291,141 +275,11 @@ func (e *Email) buildMessageFromRequest(req Request, email string, forAdmin bool
 	}
 	err = e.msgTmpl.Execute(&msg, tmplData)
 	if err != nil {
-		return "", errors.Wrapf(err, "error executing template to build comment reply message")
+		return commentMessage{}, fmt.Errorf("error executing template to build comment reply message: %w", err)
 	}
-	return e.buildMessage(subject, msg.String(), email, "text/html", unsubscribeLink)
+	return commentMessage{
-}
+		subject:         subject,
-
+		body:            msg.String(),
-// buildMessage generates email message to send using net/smtp.Data()
+		unsubscribeLink: unsubscribeLink,
-func (e *Email) buildMessage(subject, body, to, contentType, unsubscribeLink string) (message string, err error) {
+	}, err
-	addHeader := func(msg, h, v string) string {
-		msg += fmt.Sprintf("%s: %s\n", h, v)
-		return msg
-	}
-	message = addHeader(message, "From", e.From)
-	message = addHeader(message, "To", to)
-	message = addHeader(message, "Subject", mime.BEncoding.Encode("utf-8", subject))
-	message = addHeader(message, "Content-Transfer-Encoding", "quoted-printable")
-
-	if contentType != "" {
-		message = addHeader(message, "MIME-version", "1.0")
-		message = addHeader(message, "Content-Type", contentType+`; charset="UTF-8"`)
-	}
-
-	if unsubscribeLink != "" {
-		// https://support.google.com/mail/answer/81126 -> "Include option to unsubscribe"
-		message = addHeader(message, "List-Unsubscribe-Post", "List-Unsubscribe=One-Click")
-		message = addHeader(message, "List-Unsubscribe", "<"+unsubscribeLink+">")
-	}
-
-	message = addHeader(message, "Date", time.Now().Format(time.RFC1123Z))
-
-	buff := &bytes.Buffer{}
-	qp := quotedprintable.NewWriter(buff)
-	if _, err := qp.Write([]byte(body)); err != nil {
-		return "", err
-	}
-	// flush now, must NOT use defer, for small body, defer may cause buff.String() got empty body
-	if err := qp.Close(); err != nil {
-		return "", fmt.Errorf("quotedprintable Write failed: %w", err)
-	}
-	m := buff.String()
-	message += "\n" + m
-	return message, nil
-}
-
-// sendMessage sends messages to server in a new connection, closing the connection after finishing.
-// Thread safe.
-func (e *Email) sendMessage(m emailMessage) error {
-	if e.smtp == nil {
-		return errors.New("sendMessage called without client set")
-	}
-	client, err := e.smtp.Create(e.SMTPParams)
-	if err != nil {
-		return errors.Wrap(err, "failed to make smtp Create")
-	}
-
-	defer func() {
-		if err = client.Quit(); err != nil {
-			log.Printf("[WARN] failed to send quit command to %s:%d, %v", e.Host, e.Port, err)
-			if err = client.Close(); err != nil {
-				log.Printf("[WARN] can't close smtp connection, %v", err)
-			}
-		}
-	}()
-
-	if err = client.Mail(m.from); err != nil {
-		return errors.Wrapf(err, "bad from address %q", m.from)
-	}
-	if err = client.Rcpt(m.to); err != nil {
-		return errors.Wrapf(err, "bad to address %q", m.to)
-	}
-
-	writer, err := client.Data()
-	if err != nil {
-		return errors.Wrap(err, "can't make email writer")
-	}
-
-	defer func() {
-		if err = writer.Close(); err != nil {
-			log.Printf("[WARN] can't close smtp body writer, %v", err)
-		}
-	}()
-
-	buf := bytes.NewBufferString(m.message)
-	if _, err = buf.WriteTo(writer); err != nil {
-		return errors.Wrapf(err, "failed to send email body to %q", m.to)
-	}
-
-	return nil
-}
-
-// String representation of Email object
-func (e *Email) String() string {
-	return fmt.Sprintf("email: from %q with username '%s' at server %s:%d", e.From, e.Username, e.Host, e.Port)
-}
-
-// Create establish SMTP connection with server using credentials in smtpClientWithCreator.SMTPParams
-// and returns pointer to it. Thread safe.
-func (s *emailClient) Create(params SMTPParams) (smtpClient, error) {
-	authenticate := func(c *smtp.Client) error {
-		if params.Username == "" || params.Password == "" {
-			return nil
-		}
-		auth := smtp.PlainAuth("", params.Username, params.Password, params.Host)
-		if err := c.Auth(auth); err != nil {
-			return errors.Wrapf(err, "failed to auth to smtp %s:%d", params.Host, params.Port)
-		}
-		return nil
-	}
-
-	var c *smtp.Client
-	srvAddress := fmt.Sprintf("%s:%d", params.Host, params.Port)
-	if params.TLS {
-		tlsConf := &tls.Config{
-			InsecureSkipVerify: false,
-			ServerName:         params.Host,
-			MinVersion:         tls.VersionTLS12,
-		}
-		conn, err := tls.Dial("tcp", srvAddress, tlsConf)
-		if err != nil {
-			return nil, errors.Wrapf(err, "failed to dial smtp tls to %s", srvAddress)
-		}
-		if c, err = smtp.NewClient(conn, params.Host); err != nil {
-			return nil, errors.Wrapf(err, "failed to make smtp client for %s", srvAddress)
-		}
-		return c, authenticate(c)
-	}
-
-	conn, err := net.DialTimeout("tcp", srvAddress, params.TimeOut)
-	if err != nil {
-		return nil, errors.Wrapf(err, "timeout connecting to %s", srvAddress)
-	}
-
-	c, err = smtp.NewClient(conn, params.Host)
-	if err != nil {
-		return nil, errors.Wrap(err, "failed to dial")
-	}
-
-	return c, authenticate(c)
 }

backend/app/notify/email_test.go

@@ -1,16 +1,12 @@
 package notify
 
 import (
-	"bytes"
 	"context"
-	"errors"
+	"fmt"
-	"io"
-	"net/smtp"
-	"sync"
 	"testing"
 	"text/template"
-	"time"
 
+	ntf "github.com/go-pkgz/notify"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
@@ -23,13 +19,13 @@ func TestEmailNew(t *testing.T) {
 		VerificationTemplatePath: "testdata/verification.html.tmpl",
 		MsgTemplatePath:          "testdata/msg.html.tmpl",
 	}
-	smtpParams := SMTPParams{
+	smtpParams := ntf.SMTPParams{
 		Host:     "test@host",
 		Port:     1000,
 		TLS:      true,
+		StartTLS: true,
 		Username: "test@username",
 		Password: "test@password",
-		TimeOut:  time.Second,
 	}
 
 	email, err := NewEmail(emailParams, smtpParams)
@@ -38,7 +34,7 @@ func TestEmailNew(t *testing.T) {
 	assert.NotNil(t, email, "email returned")
 
 	assert.NotNil(t, email.msgTmpl, "e.template is set")
-	assert.Equal(t, emailParams.From, email.EmailParams.From, "emailParams.From unchanged after creation")
+	assert.Equal(t, emailParams.From, email.From, "emailParams.From unchanged after creation")
 	if smtpParams.TimeOut == 0 {
 		assert.Equal(t, defaultEmailTimeout, email.TimeOut, "empty emailParams.TimeOut changed to default")
 	} else {
@@ -49,6 +45,8 @@ func TestEmailNew(t *testing.T) {
 	assert.Equal(t, smtpParams.Password, email.Password, "emailParams.Password unchanged after creation")
 	assert.Equal(t, smtpParams.Port, email.Port, "emailParams.Port unchanged after creation")
 	assert.Equal(t, smtpParams.TLS, email.TLS, "emailParams.TLS unchanged after creation")
+	assert.Equal(t, smtpParams.StartTLS, email.StartTLS, "emailParams.TLS unchanged after creation")
+	assert.Equal(t, "email: with username 'test@username' at server test@host:1000 with TLS with StartTLS", email.String())
 }
 
 func Test_initTemplatesErr(t *testing.T) {
@@ -59,18 +57,16 @@ func Test_initTemplatesErr(t *testing.T) {
 	}{
 		{
 			name:    "with wrong path to verification template",
-			errText: "can't read verification template: open notfount.tmpl: no such file or directory",
+			errText: "notfound.tmpl: file does not exist",
 			emailParams: EmailParams{
-				VerificationTemplatePath: "notfount.tmpl",
+				VerificationTemplatePath: "notfound.tmpl",
-				MsgTemplatePath:          "testdata/msg.html.tmpl",
 			},
 		},
 		{
 			name:    "with wrong path to message template",
-			errText: "can't read message template: open notfount.tmpl: no such file or directory",
+			errText: "notfound.tmpl: file does not exist",
 			emailParams: EmailParams{
-				VerificationTemplatePath: "testdata/verification.html.tmpl",
+				MsgTemplatePath: "notfound.tmpl",
-				MsgTemplatePath:          "notfount.tmpl",
 			},
 		},
 		{
@@ -92,11 +88,10 @@ func Test_initTemplatesErr(t *testing.T) {
 	}
 
 	for _, d := range testSet {
-		d := d
 		t.Run(d.name, func(t *testing.T) {
-			e := Email{EmailParams: d.emailParams}
+			e, err := NewEmail(d.emailParams, ntf.SMTPParams{})
-			err := e.setTemplates()
 			require.Error(t, err)
+			require.Nil(t, e)
 			assert.Contains(t, err.Error(), d.errText)
 		})
 	}
@@ -125,7 +120,6 @@ func TestEmailSendErrors(t *testing.T) {
 	assert.EqualError(t, e.Send(ctx, Request{Comment: store.Comment{ID: "999"}, parent: store.Comment{User: store.User{ID: "test"}}, Emails: []string{"bad@example.org"}}),
 		"sending email messages about comment \"999\" aborted due to canceled context")
 
-	e.smtp = &fakeTestSMTP{}
 	assert.EqualError(t, e.Send(context.Background(), Request{Comment: store.Comment{ID: "999"}, parent: store.Comment{User: store.User{ID: "error"}}, Emails: []string{"bad@example.org"}}),
 		"1 error occurred:\n\t* problem sending user email notification to \"bad@example.org\":"+
 			" error creating token for unsubscribe link: token generation error\n\n")
@@ -135,7 +129,7 @@ func TestEmailSend_ExitConditions(t *testing.T) {
 	email, err := NewEmail(EmailParams{
 		VerificationTemplatePath: "testdata/verification.html.tmpl",
 		MsgTemplatePath:          "testdata/msg.html.tmpl",
-	}, SMTPParams{})
+	}, ntf.SMTPParams{})
 	assert.NoError(t, err)
 	assert.NotNil(t, email, "expecting email returned")
 	// prevent triggering e.autoFlush creation
@@ -144,67 +138,14 @@ func TestEmailSend_ExitConditions(t *testing.T) {
 		"Message without Emails and AdminEmails is not sent and returns nil")
 }
 
-func TestEmailSendClientError(t *testing.T) {
-	var testSet = []struct {
-		name string
-		smtp *fakeTestSMTP
-		err  string
-	}{
-		{name: "failed to verify receiver", smtp: &fakeTestSMTP{fail: map[string]bool{"mail": true}},
-			err: "bad from address \"\": failed to verify sender"},
-		{name: "failed to verify sender", smtp: &fakeTestSMTP{fail: map[string]bool{"rcpt": true}},
-			err: "bad to address \"\": failed to verify receiver"},
-		{name: "failed to close connection", smtp: &fakeTestSMTP{fail: map[string]bool{"quit": true, "close": true}}},
-		{name: "failed to make email writer", smtp: &fakeTestSMTP{fail: map[string]bool{"data": true}},
-			err: "can't make email writer: failed to send"},
-	}
-	for _, d := range testSet {
-		d := d
-		t.Run(d.name, func(t *testing.T) {
-			e := Email{smtp: d.smtp}
-			if d.err != "" {
-				assert.EqualError(t, e.sendMessage(emailMessage{}), d.err,
-					"expected error for e.sendMessage")
-			} else {
-				assert.NoError(t, e.sendMessage(emailMessage{}),
-					"expected no error for e.sendMessage")
-			}
-		})
-	}
-	e := Email{}
-	e.smtp = nil
-	assert.Error(t, e.sendMessage(emailMessage{}),
-		"nil e.smtp should return error")
-	e.smtp = &fakeTestSMTP{}
-	assert.NoError(t, e.sendMessage(emailMessage{}), "",
-		"no error expected for e.sendMessage in normal flow")
-	e.smtp = &fakeTestSMTP{fail: map[string]bool{"quit": true}}
-	assert.NoError(t, e.sendMessage(emailMessage{}), "",
-		"no error expected for e.sendMessage with failed smtpClient.Quit but successful smtpClient.Close")
-	e.smtp = &fakeTestSMTP{fail: map[string]bool{"create": true}}
-	assert.EqualError(t, e.sendMessage(emailMessage{}), "failed to make smtp Create: failed to create client",
-		"e.send called without smtpClient set returns error")
-}
-
-func TestEmail_DefaultTemplates(t *testing.T) {
-	email, err := NewEmail(EmailParams{}, SMTPParams{})
-	assert.Error(t, err)
-	assert.Nil(t, email)
-	email, err = NewEmail(EmailParams{VerificationTemplatePath: "testdata/verification.html.tmpl"}, SMTPParams{})
-	assert.Error(t, err)
-	assert.Nil(t, email)
-}
-
 func TestEmail_Send(t *testing.T) {
 	email, err := NewEmail(EmailParams{
 		From:                     "from@example.org",
 		VerificationTemplatePath: "testdata/verification.html.tmpl",
 		MsgTemplatePath:          "testdata/msg.html.tmpl",
-	}, SMTPParams{})
+	}, ntf.SMTPParams{})
 	assert.NoError(t, err)
 	assert.NotNil(t, email)
-	fakeSMTP := fakeTestSMTP{}
-	email.smtp = &fakeSMTP
 	email.TokenGenFn = TokenGenFn
 	email.UnsubscribeURL = "https://remark42.com/api/v1/email/unsubscribe"
 	req := Request{
@@ -212,22 +153,21 @@ func TestEmail_Send(t *testing.T) {
 		parent:  store.Comment{ID: "1", User: store.User{ID: "999", Name: "parent_user"}},
 		Emails:  []string{"test@example.org"},
 	}
-	assert.NoError(t, email.Send(context.TODO(), req))
+	assert.Contains(t, email.Send(context.Background(), req).Error(), "problem sending user email notification to \"test@example.org\"")
-	assert.Equal(t, "from@example.org", fakeSMTP.readMail())
-	assert.Equal(t, 1, fakeSMTP.readQuitCount())
-	assert.Equal(t, "test@example.org", fakeSMTP.readRcpt())
 	// test buildMessageFromRequest separately for message text
-	res, err := email.buildMessageFromRequest(req, req.Emails[0], false)
+	msg, err := email.buildMessageFromRequest(req, req.Emails[0], false)
 	assert.NoError(t, err)
-	assert.Contains(t, res, `From: from@example.org
+	assert.Equal(t, `
-To: test@example.org
+	New reply from test_user on your comment to «test_title»
-Subject: New reply to your comment for "test_title"
+
-Content-Transfer-Encoding: quoted-printable
+User: test_user
-MIME-version: 1.0
+01.01.0001 at 00:00
-Content-Type: text/html; charset="UTF-8"
+Comment: 
-List-Unsubscribe-Post: List-Unsubscribe=One-Click
+test@example.org  for parent_user
-List-Unsubscribe: <https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token>
+Unsubscribe link: https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token
-Date: `)
+`, msg.body)
+	assert.Equal(t, "https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token", msg.unsubscribeLink)
+	assert.Equal(t, `New reply to your comment for "test_title"`, msg.subject)
 
 	// send email to both user and admin, without parent set
 	email.AdminEmails = []string{"admin@example.org"}
@@ -235,51 +175,19 @@ Date: `)
 		Comment: store.Comment{ID: "999", User: store.User{ID: "1", Name: "test_user"}, PostTitle: "test_title"},
 		Emails:  []string{"test@example.org"},
 	}
-	assert.NoError(t, email.Send(context.TODO(), req))
+	assert.Error(t, email.Send(context.Background(), req))
-	assert.Equal(t, "from@example.org", fakeSMTP.readMail())
+	msg, err = email.buildMessageFromRequest(req, email.AdminEmails[0], true)
-	assert.Equal(t, 3, fakeSMTP.readQuitCount(), "plus two emails: one for user and one for admin")
-	assert.Equal(t, "admin@example.org", fakeSMTP.readRcpt())
-	res, err = email.buildMessageFromRequest(req, email.AdminEmails[0], true)
 	assert.NoError(t, err)
-	assert.Contains(t, res, `From: from@example.org
+	assert.Equal(t, `
-To: admin@example.org
+New comment from test_user on your site  to «test_title»
-Subject: New comment to your site for "test_title"
-Content-Transfer-Encoding: quoted-printable
-MIME-version: 1.0
-Content-Type: text/html; charset="UTF-8"
-Date: `)
-}
 
-func TestEmail_SendWithUnicodeInSubject(t *testing.T) {
+User: test_user
-	email, err := NewEmail(EmailParams{
+01.01.0001 at 00:00
-		From:                     "from@example.org",
+Comment: 
-		VerificationTemplatePath: "testdata/verification.html.tmpl",
+admin@example.org 
-		MsgTemplatePath:          "testdata/msg.html.tmpl",
+`, msg.body)
-	}, SMTPParams{})
+	assert.Equal(t, `New comment to your site for "test_title"`, msg.subject)
-	assert.NoError(t, err)
+	assert.Empty(t, msg.unsubscribeLink)
-	assert.NotNil(t, email)
-	fakeSMTP := fakeTestSMTP{}
-	email.smtp = &fakeSMTP
-	email.TokenGenFn = TokenGenFn
-	email.UnsubscribeURL = "https://remark42.com/api/v1/email/unsubscribe"
-	req := Request{
-		Comment: store.Comment{ID: "999", User: store.User{ID: "1", Name: "test_user"}, ParentID: "1", PostTitle: "Привет"},
-		parent:  store.Comment{ID: "1", User: store.User{ID: "999", Name: "parent_user"}},
-		Emails:  []string{"test@example.org"},
-	}
-	// test buildMessageFromRequest separately for message text
-	res, err := email.buildMessageFromRequest(req, req.Emails[0], false)
-	assert.NoError(t, err)
-	// `=?utf-8?b?TmV3IHJlcGx5IHRvIHlvdXIgY29tbWVudCBmb3IgItCf0YDQuNCy0LXRgiI=?=` -> `New reply to your comment for "Привет"` in base64 + required prefix and suffix
-	assert.Contains(t, res, `From: from@example.org
-To: test@example.org
-Subject: =?utf-8?b?TmV3IHJlcGx5IHRvIHlvdXIgY29tbWVudCBmb3IgItCf0YDQuNCy0LXRgiI=?=
-Content-Transfer-Encoding: quoted-printable
-MIME-version: 1.0
-Content-Type: text/html; charset="UTF-8"
-List-Unsubscribe-Post: List-Unsubscribe=One-Click
-List-Unsubscribe: <https://remark42.com/api/v1/email/unsubscribe?site=&tkn=token>
-Date: `)
 }
 
 func TestEmail_SendVerification(t *testing.T) {
@@ -287,11 +195,9 @@ func TestEmail_SendVerification(t *testing.T) {
 		From:                     "from@example.org",
 		VerificationTemplatePath: "testdata/verification.html.tmpl",
 		MsgTemplatePath:          "testdata/msg.html.tmpl",
-	}, SMTPParams{})
+	}, ntf.SMTPParams{})
 	assert.NoError(t, err)
 	assert.NotNil(t, email)
-	fakeSMTP := fakeTestSMTP{}
-	email.smtp = &fakeSMTP
 	email.TokenGenFn = TokenGenFn
 	// proper VerificationRequest without email
 	req := VerificationRequest{
@@ -299,149 +205,41 @@ func TestEmail_SendVerification(t *testing.T) {
 		User:   "test_username",
 		Token:  "secret_",
 	}
-	assert.NoError(t, email.SendVerification(context.TODO(), req))
+	assert.NoError(t, email.SendVerification(context.Background(), req))
-	assert.Equal(t, "", fakeSMTP.readMail())
-	assert.Equal(t, 0, fakeSMTP.readQuitCount())
-	assert.Equal(t, "", fakeSMTP.readRcpt())
 
 	// proper VerificationRequest with email
 	req.Email = "test@example.org"
-	assert.NoError(t, email.SendVerification(context.TODO(), req))
+	assert.Error(t, email.SendVerification(context.Background(), req), "failed to make smtp client")
-	assert.Equal(t, "from@example.org", fakeSMTP.readMail())
-	assert.Equal(t, 1, fakeSMTP.readQuitCount())
-	assert.Equal(t, "test@example.org", fakeSMTP.readRcpt())
 
 	// VerificationRequest with canceled context
-	ctx, cancel := context.WithCancel(context.TODO())
+	ctx, cancel := context.WithCancel(context.Background())
 	cancel()
 	assert.EqualError(t, email.SendVerification(ctx, req), "sending message to \"test_username\" aborted due to canceled context")
 
 	// test buildVerificationMessage separately for message text
 	res, err := email.buildVerificationMessage(req.User, req.Email, req.Token, req.SiteID)
 	assert.NoError(t, err)
-	assert.Contains(t, res, `From: from@example.org
+	assert.Equal(t, res, `Confirmation for test_username on site remark
-To: test@example.org
+Token:secret_
-Subject: Email verification
+Sent to test@example.org
-Content-Transfer-Encoding: quoted-printable
+
-MIME-version: 1.0
+`)
-Content-Type: text/html; charset="UTF-8"
-Date: `)
 	assert.Contains(t, res, `secret_`)
 	assert.NotContains(t, res, `https://example.org/`)
 	email.SubscribeURL = "https://example.org/subscribe.html?token="
 	res, err = email.buildVerificationMessage(req.User, req.Email, req.Token, req.SiteID)
 	assert.NoError(t, err)
-	assert.Contains(t, res, `From: from@example.org
+	assert.Equal(t, res, `Confirmation for test_username on site remark
-To: test@example.org
+Subscribe url: https://example.org/subscribe.html?token=secret_
-Subject: Email verification
+Token:secret_
-Content-Transfer-Encoding: quoted-printable
+Sent to test@example.org
-MIME-version: 1.0
-Content-Type: text/html; charset="UTF-8"
-Date: `)
-	assert.Contains(t, res, `https://example.org/subscribe.html?token=3Dsecret_`)
-}
 
-func Test_emailClient_Create(t *testing.T) {
+`)
-	creator := emailClient{}
-	client, err := creator.Create(SMTPParams{})
-	assert.Error(t, err, "absence of address to connect results in error")
-	assert.Nil(t, client, "no client returned in case of error")
-}
-
-type fakeTestSMTP struct {
-	fail map[string]bool
-
-	buff       bytes.Buffer
-	mail, rcpt string
-	auth       bool
-	close      bool
-	quitCount  int
-	lock       sync.RWMutex
-}
-
-func (f *fakeTestSMTP) Create(SMTPParams) (smtpClient, error) {
-	if f.fail["create"] {
-		return nil, errors.New("failed to create client")
-	}
-	return f, nil
-}
-
-func (f *fakeTestSMTP) Auth(smtp.Auth) error { f.auth = true; return nil }
-
-func (f *fakeTestSMTP) Mail(m string) error {
-	f.lock.Lock()
-	f.mail = m
-	f.lock.Unlock()
-	if f.fail["mail"] {
-		return errors.New("failed to verify sender")
-	}
-	return nil
-}
-
-func (f *fakeTestSMTP) Rcpt(r string) error {
-	f.lock.Lock()
-	f.rcpt = r
-	f.lock.Unlock()
-	if f.fail["rcpt"] {
-		return errors.New("failed to verify receiver")
-	}
-	return nil
-}
-
-func (f *fakeTestSMTP) Quit() error {
-	f.lock.Lock()
-	f.quitCount++
-	f.lock.Unlock()
-	if f.fail["quit"] {
-		return errors.New("failed to quit")
-	}
-	return nil
-}
-
-func (f *fakeTestSMTP) Close() error {
-	f.close = true
-	if f.fail["close"] {
-		return errors.New("failed to close")
-	}
-	return nil
-}
-
-func (f *fakeTestSMTP) Data() (io.WriteCloser, error) {
-	if f.fail["data"] {
-		return nil, errors.New("failed to send")
-	}
-	return nopCloser{&f.buff}, nil
-}
-
-func (f *fakeTestSMTP) readRcpt() string {
-	f.lock.RLock()
-	defer f.lock.RUnlock()
-	return f.rcpt
-}
-
-func (f *fakeTestSMTP) readMail() string {
-	f.lock.RLock()
-	defer f.lock.RUnlock()
-	return f.mail
-}
-
-func (f *fakeTestSMTP) readQuitCount() int {
-	f.lock.RLock()
-	defer f.lock.RUnlock()
-	return f.quitCount
 }
 
 func TokenGenFn(user, _, _ string) (string, error) {
 	if user == "error" {
-		return "", errors.New("token generation error")
+		return "", fmt.Errorf("token generation error")
 	}
 	return "token", nil
 }
-
-type nopCloser struct {
-	io.Writer
-}
-
-func (nopCloser) Close() error {
-	return nil
-}

backend/app/notify/notify.go

@@ -34,8 +34,8 @@ type Destination interface {
 // Store defines the minimal interface accessing stored comments used by notifier
 type Store interface {
 	Get(locator store.Locator, id string, user store.User) (store.Comment, error)
-	GetUserEmail(siteID string, userID string) (string, error)
+	GetUserEmail(siteID, userID string) (string, error)
-	GetUserTelegram(siteID string, userID string) (string, error)
+	GetUserTelegram(siteID, userID string) (string, error)
 }
 
 // used for email and telegram retrieval from user details
@@ -143,6 +143,12 @@ func (s *Service) SubmitVerification(req VerificationRequest) {
 // Close queue channel and wait for completion
 func (s *Service) Close() {
 	if s.queue != nil {
+		// don't panic in case service is already closed
+		select {
+		case <-s.ctx.Done():
+			return
+		default:
+		}
 		log.Print("[DEBUG] close notifier")
 		close(s.queue)
 		close(s.verificationQueue)

backend/app/notify/notify_mock.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"fmt"
 	"sync"
-	"time"
 
 	log "github.com/go-pkgz/lgr"
 )
@@ -22,14 +21,13 @@ type MockDest struct {
 func (m *MockDest) Send(ctx context.Context, r Request) error {
 	m.lock.Lock()
 	defer m.lock.Unlock()
-	select {
+	if err := ctx.Err(); err != nil {
-	case <-time.After(10 * time.Millisecond):
-		m.data = append(m.data, r)
-		log.Printf("sent %s -> %d", r.Comment.ID, m.id)
-	case <-ctx.Done():
 		log.Printf("ctx closed %d", m.id)
 		m.closed = true
+		return nil
 	}
+	m.data = append(m.data, r)
+	log.Printf("sent %s -> %d", r.Comment.ID, m.id)
 	return nil
 }
 
@@ -37,14 +35,13 @@ func (m *MockDest) Send(ctx context.Context, r Request) error {
 func (m *MockDest) SendVerification(ctx context.Context, v VerificationRequest) error {
 	m.lock.Lock()
 	defer m.lock.Unlock()
-	select {
+	if err := ctx.Err(); err != nil {
-	case <-time.After(10 * time.Millisecond):
-		m.verificationData = append(m.verificationData, v)
-		log.Printf("sent verification %s -> %d", v.User, m.id)
-	case <-ctx.Done():
 		log.Printf("verification ctx closed %d", m.id)
 		m.closed = true
+		return nil
 	}
+	m.verificationData = append(m.verificationData, v)
+	log.Printf("sent verification %s -> %d", v.User, m.id)
 	return nil
 }
 
@@ -66,4 +63,15 @@ func (m *MockDest) GetVerify() []VerificationRequest {
 	return res
 }
 
-func (m *MockDest) String() string { return fmt.Sprintf("mock id=%d, closed=%v", m.id, m.closed) }
+// IsClosed returns closed status safely
+func (m *MockDest) IsClosed() bool {
+	m.lock.Lock()
+	defer m.lock.Unlock()
+	return m.closed
+}
+
+func (m *MockDest) String() string {
+	m.lock.Lock()
+	defer m.lock.Unlock()
+	return fmt.Sprintf("mock id=%d, closed=%v", m.id, m.closed)
+}

backend/app/notify/notify_test.go

@@ -1,12 +1,10 @@
 package notify
 
 import (
-	"errors"
 	"fmt"
-	"math/rand"
 	"sync/atomic"
 	"testing"
-	"time"
+	"testing/synctest"
 
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
@@ -22,251 +20,261 @@ func TestService_NoDestinations(t *testing.T) {
 	s.Submit(Request{Comment: store.Comment{ID: "123"}})
 	s.Submit(Request{Comment: store.Comment{ID: "123"}})
 	s.Close()
+	// second call should not result in panic
+	s.Close()
 }
 
 func TestService_WithDestinations(t *testing.T) {
-	d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
+	synctest.Test(t, func(t *testing.T) {
-	s := NewService(nil, 1, d1, d2)
+		d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
-	assert.NotNil(t, s)
+		s := NewService(nil, 1, d1, d2)
+		assert.NotNil(t, s)
 
-	s.Submit(Request{Comment: store.Comment{ID: "100"}})
+		s.Submit(Request{Comment: store.Comment{ID: "100"}})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
-	s.Submit(Request{Comment: store.Comment{ID: "101"}})
+		s.Submit(Request{Comment: store.Comment{ID: "101"}})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
-	s.Submit(Request{Comment: store.Comment{ID: "102"}})
+		s.Submit(Request{Comment: store.Comment{ID: "102"}})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
-	s.Close()
+		s.Close()
 
-	require.Equal(t, 3, len(d1.Get()), "got all comments to d1")
+		require.Equal(t, 3, len(d1.Get()), "got all comments to d1")
-	require.Equal(t, 3, len(d2.Get()), "got all comments to d2")
+		require.Equal(t, 3, len(d2.Get()), "got all comments to d2")
 
-	assert.Equal(t, "100", d1.Get()[0].Comment.ID)
+		assert.Equal(t, "100", d1.Get()[0].Comment.ID)
-	assert.Equal(t, "101", d1.Get()[1].Comment.ID)
+		assert.Equal(t, "101", d1.Get()[1].Comment.ID)
-	assert.Equal(t, "102", d1.Get()[2].Comment.ID)
+		assert.Equal(t, "102", d1.Get()[2].Comment.ID)
+	})
 }
 
 func TestService_WithDrops(t *testing.T) {
-	d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
+	synctest.Test(t, func(t *testing.T) {
-	s := NewService(nil, 1, d1, d2)
+		d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
-	assert.NotNil(t, s)
+		s := NewService(nil, 1, d1, d2)
+		assert.NotNil(t, s)
 
-	s.Submit(Request{Comment: store.Comment{ID: "100"}})
+		s.Submit(Request{Comment: store.Comment{ID: "100"}})
-	s.Submit(Request{Comment: store.Comment{ID: "101"}})
+		s.Submit(Request{Comment: store.Comment{ID: "101"}})
-	s.Submit(Request{Comment: store.Comment{ID: "102"}})
+		s.Submit(Request{Comment: store.Comment{ID: "102"}})
-	time.Sleep(time.Millisecond * 21)
+		synctest.Wait()
-	s.Close()
+		s.Close()
 
-	s.Submit(Request{Comment: store.Comment{ID: "111"}}) // safe to send after close
+		s.Submit(Request{Comment: store.Comment{ID: "111"}}) // safe to send after close
 
-	assert.LessOrEqual(t, len(d1.Get()), 2, "at least one comment from three dropped from d1, got: %v", d1.Get())
+		assert.LessOrEqual(t, len(d1.Get()), 2, "at least one comment from three dropped from d1, got: %v", d1.Get())
-	assert.LessOrEqual(t, len(d2.Get()), 2, "at least one comment from three dropped from d2, got: %v", d2.Get())
+		assert.LessOrEqual(t, len(d2.Get()), 2, "at least one comment from three dropped from d2, got: %v", d2.Get())
+	})
 }
 
 func TestService_SubmitVerificationWithDrops(t *testing.T) {
-	d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
+	synctest.Test(t, func(t *testing.T) {
-	s := NewService(nil, 1, d1, d2)
+		d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
-	assert.NotNil(t, s)
+		s := NewService(nil, 1, d1, d2)
+		assert.NotNil(t, s)
 
-	s.SubmitVerification(VerificationRequest{
+		s.SubmitVerification(VerificationRequest{
-		SiteID: "remark",
+			SiteID: "remark",
-		User:   "testUser",
+			User:   "testUser",
-		Email:  "test@example.org",
+			Email:  "test@example.org",
-		Token:  "testToken",
+			Token:  "testToken",
+		})
+		s.SubmitVerification(VerificationRequest{})
+		s.SubmitVerification(VerificationRequest{})
+		synctest.Wait()
+		s.Close()
+
+		s.SubmitVerification(VerificationRequest{}) // safe to send after close
+
+		assert.LessOrEqual(t, len(d2.GetVerify()), 2, "one request from three dropped from d2, got: %v", d2.GetVerify())
+
+		verifyDest := d1.GetVerify()
+		require.LessOrEqual(t, len(verifyDest), 2, "one request from three dropped from d1, got: %v", verifyDest)
+		assert.Equal(t, "remark", verifyDest[0].SiteID)
+		assert.Equal(t, "testUser", verifyDest[0].User)
+		assert.Equal(t, "test@example.org", verifyDest[0].Email)
+		assert.Equal(t, "testToken", verifyDest[0].Token)
 	})
-	s.SubmitVerification(VerificationRequest{})
-	s.SubmitVerification(VerificationRequest{})
-	time.Sleep(time.Millisecond * 21)
-	s.Close()
-
-	s.SubmitVerification(VerificationRequest{}) // safe to send after close
-
-	assert.LessOrEqual(t, len(d2.GetVerify()), 2, "one request from three dropped from d2, got: %v", d2.GetVerify())
-
-	verifyDest := d1.GetVerify()
-	require.LessOrEqual(t, len(verifyDest), 2, "one request from three dropped from d1, got: %v", verifyDest)
-	assert.Equal(t, "remark", verifyDest[0].SiteID)
-	assert.Equal(t, "testUser", verifyDest[0].User)
-	assert.Equal(t, "test@example.org", verifyDest[0].Email)
-	assert.Equal(t, "testToken", verifyDest[0].Token)
 }
 
 func TestService_Many(t *testing.T) {
-	d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
+	synctest.Test(t, func(t *testing.T) {
-	s := NewService(nil, 5, d1, d2)
+		d1, d2 := &MockDest{id: 1}, &MockDest{id: 2}
-	assert.NotNil(t, s)
+		s := NewService(nil, 5, d1, d2)
+		assert.NotNil(t, s)
 
-	for i := 0; i < 10; i++ {
+		for i := range 10 {
-		s.Submit(Request{Comment: store.Comment{ID: fmt.Sprintf("%d", 100+i)}})
+			s.Submit(Request{Comment: store.Comment{ID: fmt.Sprintf("%d", 100+i)}})
-		s.SubmitVerification(VerificationRequest{User: fmt.Sprintf("%d", 100+i)})
+			s.SubmitVerification(VerificationRequest{User: fmt.Sprintf("%d", 100+i)})
-		time.Sleep(time.Millisecond * time.Duration(rand.Int31n(20)))
+		}
-	}
+		s.Close()
-	s.Close()
-	time.Sleep(time.Millisecond * 10)
 
-	assert.NotEqual(t, 10, len(d1.Get()), "some comments dropped from d1")
+		assert.NotEqual(t, 10, len(d1.Get()), "some comments dropped from d1")
-	assert.NotEqual(t, 10, len(d1.GetVerify()), "some verifications dropped from d1")
+		assert.NotEqual(t, 10, len(d1.GetVerify()), "some verifications dropped from d1")
-	assert.NotEqual(t, 10, len(d2.Get()), "some comments dropped from d2")
+		assert.NotEqual(t, 10, len(d2.Get()), "some comments dropped from d2")
-	assert.NotEqual(t, 10, len(d2.GetVerify()), "some verifications dropped from d2")
+		assert.NotEqual(t, 10, len(d2.GetVerify()), "some verifications dropped from d2")
-
+	})
-	assert.True(t, d1.closed)
-	assert.True(t, d2.closed)
-	assert.Equal(t, "mock id=1, closed=true", d1.String())
 }
 
 func TestService_WithParent(t *testing.T) {
-	dest := &MockDest{id: 1}
+	synctest.Test(t, func(t *testing.T) {
-	dataStore := &mockStore{data: map[string]store.Comment{}}
+		dest := &MockDest{id: 1}
+		dataStore := &mockStore{data: map[string]store.Comment{}}
 
-	dataStore.data["p1"] = store.Comment{ID: "p1"}
+		dataStore.data["p1"] = store.Comment{ID: "p1"}
-	dataStore.data["p2"] = store.Comment{ID: "p2"}
+		dataStore.data["p2"] = store.Comment{ID: "p2"}
 
-	s := NewService(dataStore, 1, dest)
+		s := NewService(dataStore, 1, dest)
-	assert.NotNil(t, s)
+		assert.NotNil(t, s)
 
-	s.Submit(Request{Comment: store.Comment{ID: "c1", ParentID: "p1"}})
+		s.Submit(Request{Comment: store.Comment{ID: "c1", ParentID: "p1"}})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
-	s.Submit(Request{Comment: store.Comment{ID: "c11", ParentID: "p11"}})
+		s.Submit(Request{Comment: store.Comment{ID: "c11", ParentID: "p11"}})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
-	s.Close()
+		s.Close()
 
-	destRes := dest.Get()
+		destRes := dest.Get()
-	require.Equal(t, 2, len(destRes), "two comment notified")
+		require.Equal(t, 2, len(destRes), "two comment notified")
-	assert.Equal(t, "p1", destRes[0].Comment.ParentID)
+		assert.Equal(t, "p1", destRes[0].Comment.ParentID)
-	assert.Equal(t, "p1", destRes[0].parent.ID)
+		assert.Equal(t, "p1", destRes[0].parent.ID)
-	assert.Equal(t, "p11", destRes[1].Comment.ParentID)
+		assert.Equal(t, "p11", destRes[1].Comment.ParentID)
-	assert.Equal(t, "", destRes[1].parent.ID)
+		assert.Equal(t, "", destRes[1].parent.ID)
+	})
 }
 
 func TestService_EmailRetrieval(t *testing.T) {
-	dest := &MockDest{id: 1}
+	synctest.Test(t, func(t *testing.T) {
-	dataStore := &mockStore{data: map[string]store.Comment{}, userDetails: map[string]string{}}
+		dest := &MockDest{id: 1}
+		dataStore := &mockStore{data: map[string]store.Comment{}, userDetails: map[string]string{}}
 
-	dataStore.data["p1"] = store.Comment{ID: "p1", User: store.User{ID: "u1"}}
+		dataStore.data["p1"] = store.Comment{ID: "p1", User: store.User{ID: "u1"}}
-	dataStore.data["p2"] = store.Comment{ID: "p2", ParentID: "p1", User: store.User{ID: "u1"}}
+		dataStore.data["p2"] = store.Comment{ID: "p2", ParentID: "p1", User: store.User{ID: "u1"}}
-	dataStore.data["p3"] = store.Comment{ID: "p3", ParentID: "p1", User: store.User{ID: "u2"}}
+		dataStore.data["p3"] = store.Comment{ID: "p3", ParentID: "p1", User: store.User{ID: "u2"}}
-	dataStore.data["p4"] = store.Comment{ID: "p4", ParentID: "p3", User: store.User{ID: "u1"}}
+		dataStore.data["p4"] = store.Comment{ID: "p4", ParentID: "p3", User: store.User{ID: "u1"}}
-	dataStore.userDetails["u1"] = "u1@example.com"
+		dataStore.userDetails["u1"] = "u1@example.com"
 
-	s := NewService(dataStore, 1, dest)
+		s := NewService(dataStore, 1, dest)
-	assert.NotNil(t, s)
+		assert.NotNil(t, s)
 
-	// one comment, one notification
+		// one comment, one notification
-	s.Submit(Request{Comment: dataStore.data["p1"]})
+		s.Submit(Request{Comment: dataStore.data["p1"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes := dest.Get()
+		destRes := dest.Get()
-	require.Equal(t, 1, len(destRes), "one comment notified")
+		require.Equal(t, 1, len(destRes), "one comment notified")
-	assert.Equal(t, "p1", destRes[0].Comment.ID)
+		assert.Equal(t, "p1", destRes[0].Comment.ID)
-	assert.Empty(t, destRes[0].parent)
+		assert.Empty(t, destRes[0].parent)
-	assert.Empty(t, destRes[0].Emails)
+		assert.Empty(t, destRes[0].Emails)
 
-	// reply to the first comment, same comment as one in original comment
+		// reply to the first comment, same comment as one in original comment
-	s.Submit(Request{Comment: dataStore.data["p2"]})
+		s.Submit(Request{Comment: dataStore.data["p2"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 2, len(destRes), "two comment notified")
+		require.Equal(t, 2, len(destRes), "two comment notified")
-	assert.Equal(t, "p2", destRes[1].Comment.ID)
+		assert.Equal(t, "p2", destRes[1].Comment.ID)
-	assert.Equal(t, "p1", destRes[1].parent.ID)
+		assert.Equal(t, "p1", destRes[1].parent.ID)
-	assert.Equal(t, "u1", destRes[1].parent.User.ID)
+		assert.Equal(t, "u1", destRes[1].parent.User.ID)
-	assert.Empty(t, destRes[1].Emails, "u1 is not notified they are the one who left the comment")
+		assert.Empty(t, destRes[1].Emails, "u1 is not notified they are the one who left the comment")
 
-	// another reply to the first comment, another user
+		// another reply to the first comment, another user
-	s.Submit(Request{Comment: dataStore.data["p3"]})
+		s.Submit(Request{Comment: dataStore.data["p3"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 3, len(destRes), "three comment notified")
+		require.Equal(t, 3, len(destRes), "three comment notified")
-	assert.Equal(t, "p3", destRes[2].Comment.ID)
+		assert.Equal(t, "p3", destRes[2].Comment.ID)
-	assert.Equal(t, "p1", destRes[2].parent.ID)
+		assert.Equal(t, "p1", destRes[2].parent.ID)
-	assert.Equal(t, "u1", destRes[2].parent.User.ID)
+		assert.Equal(t, "u1", destRes[2].parent.User.ID)
-	assert.ElementsMatch(t, []string{"u1@example.com"}, destRes[2].Emails)
+		assert.ElementsMatch(t, []string{"u1@example.com"}, destRes[2].Emails)
 
-	// reply to the last comment by another user, should trigger email retrieval error
+		// reply to the last comment by another user, should trigger email retrieval error
-	s.Submit(Request{Comment: dataStore.data["p4"]})
+		s.Submit(Request{Comment: dataStore.data["p4"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 4, len(destRes), "four comment notified")
+		require.Equal(t, 4, len(destRes), "four comment notified")
-	assert.Equal(t, "p4", destRes[3].Comment.ID)
+		assert.Equal(t, "p4", destRes[3].Comment.ID)
-	assert.Equal(t, "p3", destRes[3].parent.ID)
+		assert.Equal(t, "p3", destRes[3].parent.ID)
-	assert.Equal(t, "u2", destRes[3].parent.User.ID)
+		assert.Equal(t, "u2", destRes[3].parent.User.ID)
-	assert.Empty(t, destRes[3].Emails, "no email can be retrieved for u2")
+		assert.Empty(t, destRes[3].Emails, "no email can be retrieved for u2")
 
-	s.Close()
+		s.Close()
+	})
 }
 
 func TestService_Recursive(t *testing.T) {
-	dest := &MockDest{id: 1}
+	synctest.Test(t, func(t *testing.T) {
-	dataStore := &mockStore{data: map[string]store.Comment{}, userDetails: map[string]string{}}
+		dest := &MockDest{id: 1}
+		dataStore := &mockStore{data: map[string]store.Comment{}, userDetails: map[string]string{}}
 
-	dataStore.data["p1"] = store.Comment{ID: "p1", User: store.User{ID: "u1"}}
+		dataStore.data["p1"] = store.Comment{ID: "p1", User: store.User{ID: "u1"}}
-	dataStore.data["p2"] = store.Comment{ID: "p2", ParentID: "p1", User: store.User{ID: "u2"}}
+		dataStore.data["p2"] = store.Comment{ID: "p2", ParentID: "p1", User: store.User{ID: "u2"}}
-	dataStore.data["p3"] = store.Comment{ID: "p3", ParentID: "p2", User: store.User{ID: "u3"}}
+		dataStore.data["p3"] = store.Comment{ID: "p3", ParentID: "p2", User: store.User{ID: "u3"}}
-	dataStore.data["p4"] = store.Comment{ID: "p4", ParentID: "p3", User: store.User{ID: "u1"}}
+		dataStore.data["p4"] = store.Comment{ID: "p4", ParentID: "p3", User: store.User{ID: "u1"}}
-	dataStore.data["p5"] = store.Comment{ID: "p5", ParentID: "p4", User: store.User{ID: "u4"}}
+		dataStore.data["p5"] = store.Comment{ID: "p5", ParentID: "p4", User: store.User{ID: "u4"}}
-	dataStore.userDetails["u1"] = "u1@example.com"
+		dataStore.userDetails["u1"] = "u1@example.com"
-	// second comment goes without email address for notification
+		// second comment goes without email address for notification
-	dataStore.userDetails["u3"] = "u3@example.com"
+		dataStore.userDetails["u3"] = "u3@example.com"
 
-	s := NewService(dataStore, 1, dest)
+		s := NewService(dataStore, 1, dest)
-	assert.NotNil(t, s)
+		assert.NotNil(t, s)
 
-	// one comment from u1 with email set
+		// one comment from u1 with email set
-	s.Submit(Request{Comment: dataStore.data["p1"]})
+		s.Submit(Request{Comment: dataStore.data["p1"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes := dest.Get()
+		destRes := dest.Get()
-	require.Equal(t, 1, len(destRes), "one comment notified")
+		require.Equal(t, 1, len(destRes), "one comment notified")
-	assert.Equal(t, "p1", destRes[0].Comment.ID)
+		assert.Equal(t, "p1", destRes[0].Comment.ID)
-	assert.Empty(t, destRes[0].parent)
+		assert.Empty(t, destRes[0].parent)
-	assert.Empty(t, destRes[0].Emails)
+		assert.Empty(t, destRes[0].Emails)
 
-	// reply to the first comment from u2 without email set
+		// reply to the first comment from u2 without email set
-	s.Submit(Request{Comment: dataStore.data["p2"]})
+		s.Submit(Request{Comment: dataStore.data["p2"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 2, len(destRes), "two comment notified")
+		require.Equal(t, 2, len(destRes), "two comment notified")
-	assert.Equal(t, "p2", destRes[1].Comment.ID)
+		assert.Equal(t, "p2", destRes[1].Comment.ID)
-	assert.Equal(t, "p1", destRes[1].parent.ID)
+		assert.Equal(t, "p1", destRes[1].parent.ID)
-	assert.Equal(t, "u1", destRes[1].parent.User.ID)
+		assert.Equal(t, "u1", destRes[1].parent.User.ID)
-	assert.ElementsMatch(t, []string{"u1@example.com"}, destRes[1].Emails)
+		assert.ElementsMatch(t, []string{"u1@example.com"}, destRes[1].Emails)
 
-	// reply to the second comment from u3 with email set
+		// reply to the second comment from u3 with email set
-	s.Submit(Request{Comment: dataStore.data["p3"]})
+		s.Submit(Request{Comment: dataStore.data["p3"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 3, len(destRes), "three comment notified")
+		require.Equal(t, 3, len(destRes), "three comment notified")
-	assert.Equal(t, "p3", destRes[2].Comment.ID)
+		assert.Equal(t, "p3", destRes[2].Comment.ID)
-	assert.Equal(t, "p2", destRes[2].parent.ID)
+		assert.Equal(t, "p2", destRes[2].parent.ID)
-	assert.Equal(t, "u2", destRes[2].parent.User.ID)
+		assert.Equal(t, "u2", destRes[2].parent.User.ID)
-	assert.ElementsMatch(t, []string{"u1@example.com"}, destRes[2].Emails)
+		assert.ElementsMatch(t, []string{"u1@example.com"}, destRes[2].Emails)
 
-	// reply to the third comment from u1 (author of the first comment), only u3 should be notified
+		// reply to the third comment from u1 (author of the first comment), only u3 should be notified
-	s.Submit(Request{Comment: dataStore.data["p4"]})
+		s.Submit(Request{Comment: dataStore.data["p4"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 4, len(destRes), "four comment notified once each")
+		require.Equal(t, 4, len(destRes), "four comment notified once each")
-	assert.Equal(t, "p4", destRes[3].Comment.ID)
+		assert.Equal(t, "p4", destRes[3].Comment.ID)
-	assert.Equal(t, "p3", destRes[3].parent.ID)
+		assert.Equal(t, "p3", destRes[3].parent.ID)
-	assert.Equal(t, "u3", destRes[3].parent.User.ID)
+		assert.Equal(t, "u3", destRes[3].parent.User.ID)
-	assert.ElementsMatch(t, []string{"u3@example.com"}, destRes[3].Emails, "u1 is not notified they are the one who left the comment")
+		assert.ElementsMatch(t, []string{"u3@example.com"}, destRes[3].Emails, "u1 is not notified they are the one who left the comment")
 
-	// reply to the fourth comment from u4, u1 and u3 should be notified once as a result
+		// reply to the fourth comment from u4, u1 and u3 should be notified once as a result
-	s.Submit(Request{Comment: dataStore.data["p5"]})
+		s.Submit(Request{Comment: dataStore.data["p5"]})
-	time.Sleep(time.Millisecond * 110)
+		synctest.Wait()
 
-	destRes = dest.Get()
+		destRes = dest.Get()
-	require.Equal(t, 5, len(destRes), "four comment notified once each")
+		require.Equal(t, 5, len(destRes), "four comment notified once each")
-	assert.Equal(t, "p5", destRes[4].Comment.ID)
+		assert.Equal(t, "p5", destRes[4].Comment.ID)
-	assert.Equal(t, "p4", destRes[4].parent.ID)
+		assert.Equal(t, "p4", destRes[4].parent.ID)
-	assert.Equal(t, "u1", destRes[4].parent.User.ID)
+		assert.Equal(t, "u1", destRes[4].parent.User.ID)
-	assert.ElementsMatch(t, []string{"u1@example.com", "u3@example.com"}, destRes[4].Emails, "u3 and u1 notified once")
+		assert.ElementsMatch(t, []string{"u1@example.com", "u3@example.com"}, destRes[4].Emails, "u3 and u1 notified once")
 
-	s.Close()
+		s.Close()
+	})
 }
 
 func TestService_Nop(t *testing.T) {
@@ -284,7 +292,7 @@ type mockStore struct {
 func (m mockStore) getUserDetail(userID string) (string, error) {
 	detail, ok := m.userDetails[userID]
 	if !ok {
-		return "", errors.New("no such user")
+		return "", fmt.Errorf("no such user")
 	}
 	return detail, nil
 }
@@ -292,7 +300,7 @@ func (m mockStore) getUserDetail(userID string) (string, error) {
 func (m mockStore) Get(_ store.Locator, id string, _ store.User) (store.Comment, error) {
 	res, ok := m.data[id]
 	if !ok {
-		return store.Comment{}, errors.New("no such id")
+		return store.Comment{}, fmt.Errorf("no such id")
 	}
 	return res, nil
 }

backend/app/notify/prune_html.go

@@ -0,0 +1,77 @@
+package notify
+
+import (
+	"fmt"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+// pruneHTML prunes string keeping HTML closing tags.
+// maxLength applies to visible text only, not HTML tags.
+func pruneHTML(htmlText string, maxLength int) string {
+	var result strings.Builder
+	var endTokens []string
+	visibleLen := 0
+
+	suffix := "..."
+	suffixLen := len(suffix)
+
+	tokenizer := html.NewTokenizer(strings.NewReader(htmlText))
+	for {
+		if tokenizer.Next() == html.ErrorToken {
+			return result.String()
+		}
+		token := tokenizer.Token()
+
+		switch token.Type {
+		case html.CommentToken, html.DoctypeToken:
+			continue
+
+		case html.StartTagToken:
+			endTokens = append([]string{fmt.Sprintf("</%s>", token.Data)}, endTokens...)
+			result.WriteString(token.String())
+
+		case html.EndTagToken:
+			if len(endTokens) > 0 {
+				endTokens = endTokens[1:]
+			}
+			result.WriteString(token.String())
+
+		case html.SelfClosingTagToken:
+			result.WriteString(token.String())
+
+		case html.TextToken:
+			text := token.String()
+			if visibleLen+len(text)+suffixLen > maxLength {
+				remaining := maxLength - visibleLen - suffixLen
+				text = pruneStringToWord(text, remaining)
+				result.WriteString(text)
+				result.WriteString(suffix)
+				for _, endTag := range endTokens {
+					result.WriteString(endTag)
+				}
+				return result.String()
+			}
+			visibleLen += len(text)
+			result.WriteString(text)
+		}
+	}
+}
+
+// pruneStringToWord prunes string to specified length respecting word boundaries
+func pruneStringToWord(text string, maxLength int) string {
+	if maxLength <= 0 {
+		return ""
+	}
+	if len(text) <= maxLength {
+		return text
+	}
+
+	// find last space at or before maxLength to cut at word boundary
+	lastSpace := strings.LastIndex(text[:maxLength+1], " ")
+	if lastSpace <= 0 {
+		return ""
+	}
+	return text[:lastSpace]
+}

backend/app/notify/prune_html_test.go

@@ -0,0 +1,47 @@
+package notify
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestPruneHTML(t *testing.T) {
+	tests := []struct {
+		name      string
+		html      string
+		maxLength int
+		expected  string
+	}{
+		{"within limit", "<p>Hello</p>", 20, "<p>Hello</p>"},
+		{"exceeds limit", "<p>Hello world, this is a long text</p>", 15, "<p>Hello world,...</p>"},
+		{"nested tags", "<div><p>Hello world</p><p>More text</p></div>", 20, "<div><p>Hello world</p><p>More...</p></div>"},
+		{"html comment stripped", "<!-- comment --><p>Hello</p>", 20, "<p>Hello</p>"},
+		{"self-closing tag", "<p>Hello<br/>World</p>", 8, "<p>Hello<br/>...</p>"},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			assert.Equal(t, tt.expected, pruneHTML(tt.html, tt.maxLength))
+		})
+	}
+}
+
+func TestPruneStringToWord(t *testing.T) {
+	tests := []struct {
+		name      string
+		text      string
+		maxLength int
+		expected  string
+	}{
+		{"within limit", "hello world", 15, "hello world"},
+		{"cut at word boundary", "hello world and more", 11, "hello world"},
+		{"zero length", "hello", 0, ""},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			assert.Equal(t, tt.expected, pruneStringToWord(tt.text, tt.maxLength))
+		})
+	}
+}

backend/app/notify/slack.go

@@ -2,43 +2,32 @@ package notify
 
 import (
 	"context"
+	"fmt"
+	"net/url"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
+	ntf "github.com/go-pkgz/notify"
-	"github.com/slack-go/slack"
 )
 
 // Slack implements notify.Destination for Slack
 type Slack struct {
-	channelID   string
+	*ntf.Slack
+
 	channelName string
-	client      *slack.Client
 }
 
 // NewSlack makes Slack bot for notifications
-func NewSlack(token, channelName string, opts ...slack.Option) (*Slack, error) {
+func NewSlack(token, channelName string) *Slack {
-
+	log.Printf("[DEBUG] create new slack notifier for chan %s", channelName)
 	if channelName == "" {
 		channelName = "general"
 	}
 
-	client := slack.New(token, opts...)
+	return &Slack{Slack: ntf.NewSlack(token), channelName: channelName}
-	res := &Slack{client: client, channelName: channelName}
-
-	channelID, err := res.findChannelIDByName(channelName)
-	if err != nil {
-		return nil, errors.Wrap(err, "can not find slack channel '"+channelName+"'")
-	}
-
-	res.channelID = channelID
-	log.Printf("[DEBUG] create new slack notifier for chan %s", channelID)
-
-	return res, nil
 }
 
 // Send to Slack channel
-func (t *Slack) Send(ctx context.Context, req Request) error {
+func (s *Slack) Send(ctx context.Context, req Request) error {
-
 	log.Printf("[DEBUG] send slack notification, comment id %s", req.Comment.ID)
 
 	user := req.Comment.User.Name
@@ -51,51 +40,22 @@ func (t *Slack) Send(ctx context.Context, req Request) error {
 		title = "↦ " + req.Comment.PostTitle
 	}
 
-	_, _, err := t.client.PostMessageContext(ctx, t.channelID,
+	destination := fmt.Sprintf(
-		slack.MsgOptionText("New comment from "+user, false),
+		"slack:%s?title=%s&attachmentText=%s&titleLink=%s",
-		slack.MsgOptionAttachments(
+		s.channelName,
-			slack.Attachment{
+		url.QueryEscape(title),
-				TitleLink: req.Comment.Locator.URL + uiNav + req.Comment.ID,
+		url.QueryEscape(req.Comment.Orig),
-				Title:     title,
+		url.QueryEscape(req.Comment.Locator.URL+uiNav+req.Comment.ID),
-				Text:      req.Comment.Orig,
-			},
-		),
 	)
 
-	return err
+	return s.Slack.Send(ctx, destination, "New comment from "+user)
-
 }
 
 // SendVerification is not implemented for Slack
-func (t *Slack) SendVerification(_ context.Context, _ VerificationRequest) error {
+func (s *Slack) SendVerification(_ context.Context, _ VerificationRequest) error {
 	return nil
 }
 
-func (t *Slack) String() string {
+func (s *Slack) String() string {
-	return "slack: " + t.channelName + " (" + t.channelID + ")"
+	return s.Slack.String() + " for channel " + s.channelName + ""
-}
-
-func (t *Slack) findChannelIDByName(name string) (string, error) {
-
-	params := slack.GetConversationsParameters{}
-	for {
-
-		chans, next, err := t.client.GetConversations(&params)
-		if err != nil {
-			return "", err
-		}
-
-		for _, channel := range chans {
-			if channel.Name == name {
-				return channel.ID, nil
-			}
-		}
-
-		if next == "" {
-			break
-		}
-		params.Cursor = next
-
-	}
-	return "", errors.New("no such channel")
 }

backend/app/notify/slack_test.go

@@ -2,169 +2,39 @@ package notify
 
 import (
 	"context"
-	"log"
-	"net/http"
-	"net/http/httptest"
 	"testing"
 
-	"github.com/go-chi/chi/v5"
-	"github.com/slack-go/slack"
 	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
 
 	"github.com/umputun/remark42/backend/app/store"
 )
 
 func TestSlack_New(t *testing.T) {
-
+	ts := NewSlack("", "")
-	ts := newMockSlackServer()
+	assert.NotNil(t, ts)
-	defer ts.Close()
+	assert.Equal(t, "general", ts.channelName)
-
-	tb, err := ts.newClient("general")
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-	assert.Equal(t, "C12345678", tb.channelID)
-
-	_, err = ts.newClient("unknown-channel")
-	require.Error(t, err)
-	assert.Contains(t, err.Error(), "no such channel")
-
 }
 
 func TestSlack_Send(t *testing.T) {
+	ts := NewSlack("", "")
 
-	ts := newMockSlackServer()
+	c := store.Comment{PostTitle: "test title", Text: "some text", ParentID: "1", ID: "999"}
-	defer ts.Close()
-
-	tb, err := ts.newClient("general")
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-
-	c := store.Comment{Text: "some text", ParentID: "1", ID: "999"}
 	c.User.Name = "from"
 	cp := store.Comment{Text: "some parent text"}
 	cp.User.Name = "to"
 
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
+	ctx, cancel := context.WithCancel(context.Background())
-	assert.NoError(t, err)
+	cancel()
-	c.PostTitle = "test title"
+	err := ts.Send(ctx, Request{Comment: c, parent: cp})
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
+	assert.Error(t, err)
-	assert.NoError(t, err)
-
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
-	assert.NoError(t, err)
-	c.PostTitle = "[test title]"
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
-	assert.NoError(t, err)
-
-	tb, err = ts.newClient("general")
-	assert.NoError(t, err)
-	ts.isServerDown = true
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
-	require.Error(t, err)
-	assert.Contains(t, err.Error(), "slack server error", "send on broken client")
-
 }
 
 func TestSlack_Name(t *testing.T) {
-	ts := newMockSlackServer()
+	tb := NewSlack("", "test-channel")
-	defer ts.Close()
+	assert.Equal(t, "slack notifications destination for channel test-channel", tb.String())
-
-	tb, err := ts.newClient("general")
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-	assert.Equal(t, "slack: general (C12345678)", tb.String())
 }
 
 func TestSlack_SendVerification(t *testing.T) {
-	ts := newMockSlackServer()
+	ts := NewSlack("", "")
-	defer ts.Close()
+	assert.NoError(t, ts.SendVerification(context.Background(), VerificationRequest{}))
-
-	tb, err := ts.newClient("general")
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-
-	err = tb.SendVerification(context.TODO(), VerificationRequest{})
-	assert.NoError(t, err)
-}
-
-type mockSlackServer struct {
-	*httptest.Server
-	isServerDown bool
-}
-
-func (ts *mockSlackServer) newClient(channelName string) (*Slack, error) {
-	return NewSlack("any-token", channelName, slack.OptionAPIURL(ts.URL+"/"))
-}
-
-func newMockSlackServer() *mockSlackServer {
-
-	mockServer := mockSlackServer{}
-	router := chi.NewRouter()
-	router.Post("/conversations.list", func(w http.ResponseWriter, r *http.Request) {
-		s := `{
-		    "ok": true,
-		    "channels": [
-		        {
-		            "id": "C12345678",
-		            "name": "general",
-		            "is_channel": true,
-		            "is_group": false,
-		            "is_im": false,
-		            "created": 1503888888,
-		            "is_archived": false,
-		            "is_general": false,
-		            "unlinked": 0,
-		            "name_normalized": "random",
-		            "is_shared": false,
-		            "parent_conversation": null,
-		            "creator": "U12345678",
-		            "is_ext_shared": false,
-		            "is_org_shared": false,
-		            "pending_shared": [],
-		            "pending_connected_team_ids": [],
-		            "is_pending_ext_shared": false,
-		            "is_member": false,
-		            "is_private": false,
-		            "is_mpim": false,
-		            "previous_names": [],
-		            "num_members": 1
-		        }
-		    ],
-		    "response_metadata": {
-		        "next_cursor": ""
-		    }
-		}`
-		_, _ = w.Write([]byte(s))
-	})
-
-	router.Post("/chat.postMessage", func(w http.ResponseWriter, r *http.Request) {
-
-		if mockServer.isServerDown {
-			w.WriteHeader(500)
-
-		} else {
-			s := `{
-			    "ok": true,
-			    "channel": "C12345678",
-			    "ts": "1617008342.000100",
-			    "message": {
-			        "type": "message",
-			        "subtype": "bot_message",
-			        "text": "wowo",
-			        "ts": "1617008342.000100",
-			        "username": "slackbot",
-			        "bot_id": "B12345678"
-			    }
-			}`
-			_, _ = w.Write([]byte(s))
-		}
-	})
-
-	router.NotFound(func(w http.ResponseWriter, r *http.Request) {
-		log.Printf("..... 404 for %s .....\n", r.URL)
-	})
-
-	mockServer.Server = httptest.NewServer(router)
-	return &mockServer
 }

backend/app/notify/telegram.go

@@ -1,28 +1,17 @@
 package notify
 
 import (
-	"bytes"
 	"context"
-	"encoding/json"
 	"fmt"
-	"golang.org/x/net/html"
-	"io"
-	"net/http"
-	neturl "net/url"
-	"strconv"
-	"strings"
-	"sync"
-	"sync/atomic"
 	"time"
 
-	"github.com/hashicorp/go-multierror"
-	"github.com/microcosm-cc/bluemonday"
-
 	log "github.com/go-pkgz/lgr"
-	"github.com/go-pkgz/repeater"
+	ntf "github.com/go-pkgz/notify"
-	"github.com/pkg/errors"
+	"github.com/hashicorp/go-multierror"
 )
 
+const commentTextLengthLimit = 100
+
 // TelegramParams contain settings for telegram notifications
 type TelegramParams struct {
 	AdminChannelID       string        // unique identifier for the target chat or username of the target channel (in the format @channelusername)
@@ -30,78 +19,29 @@ type TelegramParams struct {
 	Timeout              time.Duration // http client timeout
 	UserNotifications    bool          // flag which enables user notifications
 	ErrorMsg, SuccessMsg string        // messages for successful and unsuccessful subscription requests to bot
-
-	apiPrefix string // changed only in tests
 }
 
 // Telegram implements notify.Destination for telegram
 type Telegram struct {
-	TelegramParams
+	*ntf.Telegram
 
-	// Identifier of the first update to be requested.
+	AdminChannelID    string // unique identifier for the target chat or username of the target channel (in the format @channelusername)
-	// Should be equal to LastSeenUpdateID + 1
+	UserNotifications bool   // flag which enables user notifications
-	// See https://core.telegram.org/bots/api#getupdates
-	updateOffset           int
-	apiPollInterval        time.Duration // interval to check updates from Telegram API and answer to users
-	expiredCleanupInterval time.Duration // interval to check and clean up expired notification requests
-	username               string        // bot username
-	run                    int32         // non-zero if Run goroutine has started
-	requests               struct {
-		sync.RWMutex
-		data map[string]tgAuthRequest
-	}
 }
 
-// telegramMsg is used to send message trough Telegram bot API
-type telegramMsg struct {
-	Text      string `json:"text"`
-	ParseMode string `json:"parse_mode,omitempty"`
-}
-
-type tgAuthRequest struct {
-	confirmed  bool // whether login request has been confirmed and user info set
-	expires    time.Time
-	telegramID string
-	user       string
-	site       string
-}
-
-// TelegramBotInfo structure contains information about telegram bot, which is used from whole telegram API response
-type TelegramBotInfo struct {
-	Username string `json:"username"`
-}
-
-const telegramTimeOut = 5000 * time.Millisecond
-const telegramAPIPrefix = "https://api.telegram.org/bot"
-const tgPollInterval = time.Second * 5
-const tgCleanupInterval = time.Minute * 5
-
 // NewTelegram makes telegram bot for notifications
 func NewTelegram(params TelegramParams) (*Telegram, error) {
-	res := Telegram{TelegramParams: params}
+	client, err := ntf.NewTelegram(ntf.TelegramParams{
-
+		Token:      params.Token,
-	if res.apiPrefix == "" {
+		Timeout:    params.Timeout,
-		res.apiPrefix = telegramAPIPrefix
+		ErrorMsg:   params.ErrorMsg,
-	}
+		SuccessMsg: params.SuccessMsg,
-	if res.Timeout == 0 {
+	})
-		res.Timeout = telegramTimeOut
-	}
-	res.apiPollInterval = tgPollInterval
-	res.expiredCleanupInterval = tgCleanupInterval
-	log.Printf("[DEBUG] create new telegram notifier for api=%s, timeout=%s", res.apiPrefix, res.Timeout)
-
-	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
-	defer cancel()
-
-	botInfo, err := res.botInfo(ctx)
 	if err != nil {
-		return nil, errors.Wrapf(err, "can't retrieve bot info from Telegram API")
+		return nil, err
 	}
-	res.username = botInfo.Username
 
-	res.requests.data = make(map[string]tgAuthRequest)
+	return &Telegram{Telegram: client, AdminChannelID: params.AdminChannelID, UserNotifications: params.UserNotifications}, nil
-
-	return &res, nil
 }
 
 // Send to telegram recipients
@@ -109,121 +49,55 @@ func (t *Telegram) Send(ctx context.Context, req Request) error {
 	log.Printf("[DEBUG] send telegram notification for comment ID %s", req.Comment.ID)
 	result := new(multierror.Error)
 
-	msg, err := buildMessage(req)
+	msg := t.buildMessage(req)
-	if err != nil {
-		return errors.Wrapf(err, "failed to make telegram message body for comment ID %s", req.Comment.ID)
-	}
 
 	if t.AdminChannelID != "" {
-		err := t.sendMessage(ctx, msg, t.AdminChannelID)
+		err := t.Telegram.Send(ctx, fmt.Sprintf("telegram:%s?parseMode=HTML", t.AdminChannelID), msg)
-		result = multierror.Append(errors.Wrapf(err,
+		if err != nil {
-			"problem sending admin telegram notification about comment ID %s to %s", req.Comment.ID, t.AdminChannelID),
+			result = multierror.Append(result,
-		)
+				fmt.Errorf("problem sending admin telegram notification about comment ID %s to %s: %w",
+					req.Comment.ID, t.AdminChannelID, err,
+				),
+			)
+		}
 	}
 
 	if t.UserNotifications {
 		for _, user := range req.Telegrams {
-			err := t.sendMessage(ctx, msg, user)
+			err := t.Telegram.Send(ctx, fmt.Sprintf("telegram:%s?parseMode=HTML", user), msg)
-			result = multierror.Append(errors.Wrapf(err,
+			if err != nil {
-				"problem sending user telegram notification about comment ID %s to %q", req.Comment.ID, user),
+				result = multierror.Append(result,
-			)
+					fmt.Errorf("problem sending user telegram notification about comment ID %s to %q: %w",
+						req.Comment.ID, user, err,
+					),
+				)
+			}
 		}
 	}
 	return result.ErrorOrNil()
 }
 
-func (t *Telegram) sendMessage(ctx context.Context, b []byte, chatID string) error {
-	if _, err := strconv.ParseInt(chatID, 10, 64); err != nil {
-		chatID = "@" + chatID // if chatID not a number enforce @ prefix
-	}
-
-	url := fmt.Sprintf("sendMessage?chat_id=%s&disable_web_page_preview=true", chatID)
-	return t.Request(ctx, url, b, &struct{}{})
-}
-
 // buildMessage generates message for generic notification about new comment
-func buildMessage(req Request) ([]byte, error) {
+func (t *Telegram) buildMessage(req Request) string {
 	commentURLPrefix := req.Comment.Locator.URL + uiNav
 
-	msg := fmt.Sprintf(`<a href=%q>%s</a>`, commentURLPrefix+req.Comment.ID, escapeTelegramText(req.Comment.User.Name))
+	msg := fmt.Sprintf(`<a href=%q>%s</a>`, commentURLPrefix+req.Comment.ID, ntf.EscapeTelegramText(req.Comment.User.Name))
 
 	if req.Comment.ParentID != "" {
-		msg += fmt.Sprintf(" -> <a href=%q>%s</a>", commentURLPrefix+req.parent.ID, escapeTelegramText(req.parent.User.Name))
+		msg += fmt.Sprintf(" -> <a href=%q>%s</a>", commentURLPrefix+req.parent.ID, ntf.EscapeTelegramText(req.parent.User.Name))
 	}
 
-	msg += fmt.Sprintf("\n\n%s", telegramSupportedHTML(req.Comment.Text))
+	msg += fmt.Sprintf("\n\n%s", pruneHTML(ntf.TelegramSupportedHTML(req.Comment.Text), commentTextLengthLimit))
 
 	if req.Comment.ParentID != "" {
-		msg += fmt.Sprintf("\n\n\"<i>%s</i>\"", telegramSupportedHTML(req.parent.Text))
+		msg += fmt.Sprintf("\n\n\"<i>%s</i>\"", pruneHTML(ntf.TelegramSupportedHTML(req.parent.Text), commentTextLengthLimit))
 	}
 
 	if req.Comment.PostTitle != "" {
-		msg += fmt.Sprintf("\n\n↦  <a href=%q>%s</a>", req.Comment.Locator.URL, escapeTelegramText(req.Comment.PostTitle))
+		msg += fmt.Sprintf("\n\n↦  <a href=%q>%s</a>", req.Comment.Locator.URL, ntf.EscapeTelegramText(req.Comment.PostTitle))
 	}
 
-	body := telegramMsg{Text: msg, ParseMode: "HTML"}
+	return msg
-	b, err := json.Marshal(body)
-	if err != nil {
-		return nil, err
-	}
-	return b, nil
-}
-
-// returns HTML with only tags allowed in Telegram HTML message payload, also trims ending newlines
-// https://core.telegram.org/bots/api#html-style
-func telegramSupportedHTML(htmlText string) string {
-	adjustedHTMLText := adjustHTMLTags(htmlText)
-	p := bluemonday.NewPolicy()
-	p.AllowElements("b", "strong", "i", "em", "u", "ins", "s", "strike", "del", "a", "code", "pre")
-	p.AllowAttrs("href").OnElements("a")
-	p.AllowAttrs("class").OnElements("code")
-	return strings.TrimRight(p.Sanitize(adjustedHTMLText), "\n")
-}
-
-// returns text sanitized of symbols not allowed inside other HTML tags in Telegram HTML message payload
-// https://core.telegram.org/bots/api#html-style
-func escapeTelegramText(text string) string {
-	// order is important
-	text = strings.ReplaceAll(text, "&", "&amp;")
-	text = strings.ReplaceAll(text, "<", "&lt;")
-	text = strings.ReplaceAll(text, ">", "&gt;")
-	return text
-}
-
-// telegram not allow h1-h6 tags
-// replace these tags with a combination of <b> and <i> for visual distinction
-func adjustHTMLTags(htmlText string) string {
-	buff := strings.Builder{}
-	tokenizer := html.NewTokenizer(strings.NewReader(htmlText))
-	for {
-		if tokenizer.Next() == html.ErrorToken {
-			return buff.String()
-		}
-		token := tokenizer.Token()
-		switch token.Type {
-		case html.StartTagToken, html.EndTagToken:
-			switch token.Data {
-			case "h1", "h2", "h3":
-				if token.Type == html.StartTagToken {
-					buff.WriteString("<b>")
-				}
-				if token.Type == html.EndTagToken {
-					buff.WriteString("</b>")
-				}
-			case "h4", "h5", "h6":
-				if token.Type == html.StartTagToken {
-					buff.WriteString("<i><b>")
-				}
-				if token.Type == html.EndTagToken {
-					buff.WriteString("</b></i>")
-				}
-			default:
-				buff.WriteString(token.String())
-			}
-		default:
-			buff.WriteString(token.String())
-		}
-	}
 }
 
 // SendVerification is not needed for telegram
@@ -231,132 +105,8 @@ func (t *Telegram) SendVerification(_ context.Context, _ VerificationRequest) er
 	return nil
 }
 
-// TelegramUpdate contains update information, which is used from whole telegram API response
-type TelegramUpdate struct {
-	Result []struct {
-		UpdateID int `json:"update_id"`
-		Message  struct {
-			Chat struct {
-				ID   int    `json:"id"`
-				Name string `json:"first_name"`
-				Type string `json:"type"`
-			} `json:"chat"`
-			Text string `json:"text"`
-		} `json:"message"`
-	} `json:"result"`
-}
-
-// GetBotUsername returns bot username
-func (t *Telegram) GetBotUsername() string {
-	return t.username
-}
-
-// AddToken adds token
-func (t *Telegram) AddToken(token, user, site string, expires time.Time) {
-	t.requests.Lock()
-	t.requests.data[token] = tgAuthRequest{
-		expires: expires,
-		user:    user,
-		site:    site,
-	}
-	t.requests.Unlock()
-}
-
-// CheckToken verifies incoming token, returns the user address if it's confirmed and empty string otherwise
-func (t *Telegram) CheckToken(token, user string) (telegram, site string, err error) {
-	t.requests.RLock()
-	authRequest, ok := t.requests.data[token]
-	t.requests.RUnlock()
-
-	if !ok {
-		return "", "", errors.New("request is not found")
-	}
-
-	if time.Now().After(authRequest.expires) {
-		t.requests.Lock()
-		delete(t.requests.data, token)
-		t.requests.Unlock()
-		return "", "", errors.New("request expired")
-	}
-
-	if !authRequest.confirmed {
-		return "", "", errors.New("request is not verified yet")
-	}
-
-	if authRequest.user != user {
-		return "", "", errors.New("user does not match original requester")
-	}
-
-	// Delete request
-	t.requests.Lock()
-	delete(t.requests.data, token)
-	t.requests.Unlock()
-
-	return authRequest.telegramID, authRequest.site, nil
-}
-
-// Run starts processing login requests sent in Telegram, required for user notifications to work
-// Blocks caller
-func (t *Telegram) Run(ctx context.Context) {
-	atomic.AddInt32(&t.run, 1)
-	processUpdatedTicker := time.NewTicker(t.apiPollInterval)
-	cleanupTicker := time.NewTicker(t.expiredCleanupInterval)
-
-	for {
-		select {
-		case <-ctx.Done():
-			processUpdatedTicker.Stop()
-			cleanupTicker.Stop()
-			atomic.AddInt32(&t.run, -1)
-			return
-		case <-processUpdatedTicker.C:
-			updates, err := t.getUpdates(ctx)
-			if err != nil {
-				log.Printf("[WARN] Error while getting telegram updates: %v", err)
-				continue
-			}
-			t.processUpdates(ctx, updates)
-		case <-cleanupTicker.C:
-			now := time.Now()
-			t.requests.Lock()
-			for key, req := range t.requests.data {
-				if now.After(req.expires) {
-					delete(t.requests.data, key)
-				}
-			}
-			t.requests.Unlock()
-		}
-	}
-}
-
-// ProcessUpdate is alternative to Run, it processes provided plain text update from Telegram
-// so that caller could get updates and send it not only there but to multiple sources
-func (t *Telegram) ProcessUpdate(ctx context.Context, textUpdate string) error {
-	if atomic.LoadInt32(&t.run) != 0 {
-		return errors.New("Run goroutine should not be used with ProcessUpdate")
-	}
-	defer func() {
-		// as Run goroutine is not running, clean up old requests on each update
-		// even if we hit json decode error
-		now := time.Now()
-		t.requests.Lock()
-		for key, req := range t.requests.data {
-			if now.After(req.expires) {
-				delete(t.requests.data, key)
-			}
-		}
-		t.requests.Unlock()
-	}()
-	var updates TelegramUpdate
-	if err := json.Unmarshal([]byte(textUpdate), &updates); err != nil {
-		return errors.Wrap(err, "failed to decode provided telegram update")
-	}
-	t.processUpdates(ctx, &updates)
-	return nil
-}
-
 func (t *Telegram) String() string {
-	result := "telegram"
+	result := t.Telegram.String()
 	if t.AdminChannelID != "" {
 		result += " with admin notifications to " + t.AdminChannelID
 	}
@@ -365,134 +115,3 @@ func (t *Telegram) String() string {
 	}
 	return result
 }
-
-// getUpdates fetches incoming updates
-func (t *Telegram) getUpdates(ctx context.Context) (*TelegramUpdate, error) {
-	url := `getUpdates?allowed_updates=["message"]`
-	if t.updateOffset != 0 {
-		url += fmt.Sprintf("&offset=%d", t.updateOffset)
-	}
-
-	var result TelegramUpdate
-
-	err := t.Request(ctx, url, nil, &result)
-	if err != nil {
-		return nil, errors.Wrap(err, "failed to fetch updates")
-	}
-
-	for _, u := range result.Result {
-		if u.UpdateID >= t.updateOffset {
-			t.updateOffset = u.UpdateID + 1
-		}
-	}
-
-	return &result, nil
-}
-
-// processUpdates processes a batch of updates from telegram servers
-func (t *Telegram) processUpdates(ctx context.Context, updates *TelegramUpdate) {
-	for _, update := range updates.Result {
-		if update.Message.Chat.Type != "private" {
-			continue
-		}
-
-		if !strings.HasPrefix(update.Message.Text, "/start ") {
-			continue
-		}
-
-		token := strings.TrimPrefix(update.Message.Text, "/start ")
-
-		t.requests.RLock()
-		authRequest, ok := t.requests.data[token]
-		if !ok { // No such token
-			t.requests.RUnlock()
-			if t.ErrorMsg != "" {
-				if err := t.sendText(ctx, update.Message.Chat.ID, t.ErrorMsg); err != nil {
-					log.Printf("[WARN] failed to notify telegram peer: %v", err)
-				}
-			}
-			continue
-		}
-		t.requests.RUnlock()
-
-		authRequest.confirmed = true
-		authRequest.telegramID = strconv.Itoa(update.Message.Chat.ID)
-
-		t.requests.Lock()
-		t.requests.data[token] = authRequest
-		t.requests.Unlock()
-
-		if err := t.sendText(ctx, update.Message.Chat.ID, t.SuccessMsg); err != nil {
-			log.Printf("[ERROR] failed to notify telegram peer: %v", err)
-		}
-	}
-}
-
-// sendText sends a plain text message to telegram peer
-func (t *Telegram) sendText(ctx context.Context, recipientID int, msg string) error {
-	url := fmt.Sprintf("sendMessage?chat_id=%d&text=%s", recipientID, neturl.PathEscape(msg))
-	return t.Request(ctx, url, nil, &struct{}{})
-}
-
-// botInfo returns info about configured bot
-func (t *Telegram) botInfo(ctx context.Context) (*TelegramBotInfo, error) {
-	var resp = struct {
-		Result *TelegramBotInfo `json:"result"`
-	}{}
-
-	err := t.Request(ctx, "getMe", nil, &resp)
-	if err != nil {
-		return nil, err
-	}
-	if resp.Result == nil {
-		return nil, errors.New("received empty result")
-	}
-
-	return resp.Result, nil
-}
-
-// Request makes a request to the Telegram API and return the result
-func (t *Telegram) Request(ctx context.Context, method string, b []byte, data interface{}) error {
-	return repeater.NewDefault(3, time.Millisecond*250).Do(ctx, func() error {
-		url := fmt.Sprintf("%s%s/%s", t.apiPrefix, t.Token, method)
-
-		var req *http.Request
-		var err error
-		if b == nil {
-			req, err = http.NewRequestWithContext(ctx, "GET", url, http.NoBody)
-		} else {
-			req, err = http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(b))
-			req.Header.Set("Content-Type", "application/json; charset=utf-8")
-		}
-		if err != nil {
-			return errors.Wrap(err, "failed to create request")
-		}
-
-		client := http.Client{Timeout: t.Timeout}
-		resp, err := client.Do(req)
-		if err != nil {
-			return errors.Wrap(err, "failed to send request")
-		}
-		defer resp.Body.Close()
-
-		if resp.StatusCode != http.StatusOK {
-			return t.parseError(resp.Body, resp.StatusCode)
-		}
-
-		if err = json.NewDecoder(resp.Body).Decode(data); err != nil {
-			return errors.Wrap(err, "failed to decode json response")
-		}
-
-		return nil
-	})
-}
-
-func (t *Telegram) parseError(r io.Reader, statusCode int) error {
-	tgErr := struct {
-		Description string `json:"description"`
-	}{}
-	if err := json.NewDecoder(r).Decode(&tgErr); err != nil {
-		return errors.Errorf("unexpected telegram API status code %d", statusCode)
-	}
-	return errors.Errorf("unexpected telegram API status code %d, error: %q", statusCode, tgErr.Description)
-}

backend/app/notify/telegram_test.go

@@ -2,507 +2,70 @@ package notify
 
 import (
 	"context"
-	"net/http"
-	"net/http/httptest"
-	"strings"
 	"testing"
-	"time"
 
-	"github.com/go-chi/chi/v5"
+	ntf "github.com/go-pkgz/notify"
 	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
 
 	"github.com/umputun/remark42/backend/app/store"
 )
 
-func TestTelegram_New(t *testing.T) {
+func TestTelegram_NewError(t *testing.T) {
-	ts := mockTelegramServer(nil)
+	tb, err := NewTelegram(TelegramParams{})
-	defer ts.Close()
-
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "good-token",
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-	assert.Equal(t, tb.Timeout, time.Second*5)
-	assert.Equal(t, "remark_test", tb.AdminChannelID, "@ added")
-
-	_, err = NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "empty-json",
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.EqualError(t, err, "can't retrieve bot info from Telegram API: received empty result")
-
-	st := time.Now()
-	_, err = NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "non-json-resp",
-		Timeout:        2 * time.Second,
-		apiPrefix:      ts.URL + "/",
-	})
 	assert.Error(t, err)
-	assert.Contains(t, err.Error(), "failed to decode json response:")
+	assert.Nil(t, tb)
-	assert.True(t, time.Since(st) >= 250*3*time.Millisecond)
-
-	_, err = NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "404",
-		Timeout:        2 * time.Second,
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.EqualError(t, err, "can't retrieve bot info from Telegram API: unexpected telegram API status code 404")
-
-	_, err = NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "no-such-thing",
-		apiPrefix:      "http://127.0.0.1:4321/",
-	})
-	require.Error(t, err)
-	assert.Contains(t, err.Error(), "can't retrieve bot info from Telegram API")
-	assert.Contains(t, err.Error(), "dial tcp 127.0.0.1:4321: connect: connection refused")
-
-	_, err = NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "no-such-thing",
-		apiPrefix:      "",
-	})
-	assert.Error(t, err, "empty api url not allowed")
-
-	_, err = NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "good-token",
-		Timeout:        2 * time.Second,
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.NoError(t, err, "0 timeout allowed as default")
-
-	tb, err = NewTelegram(TelegramParams{
-		AdminChannelID: "1234567890",
-		Token:          "good-token",
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-	assert.Equal(t, "1234567890", tb.AdminChannelID, "no @ prefix")
-}
-
-func TestTelegram_GetBotUsername(t *testing.T) {
-	ts := mockTelegramServer(nil)
-	defer ts.Close()
-
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "good-token",
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-	assert.Equal(t, "remark42_test_bot", tb.GetBotUsername())
 }
 
 func TestTelegram_Send(t *testing.T) {
-	ts := mockTelegramServer(nil)
+	tb := Telegram{
-	defer ts.Close()
-
-	tb, err := NewTelegram(TelegramParams{
 		AdminChannelID:    "remark_test",
-		Token:             "good-token",
 		UserNotifications: true,
-		apiPrefix:         ts.URL + "/",
+		Telegram:          &ntf.Telegram{}, // broken sender due to unset API
-	})
+	}
-	assert.NoError(t, err)
+	assert.Equal(t, "telegram notifications destination with admin notifications to remark_test with user notifications enabled", tb.String())
-	assert.NotNil(t, tb)
+	c := store.Comment{Text: "some text", ParentID: "1", ID: "999", PostTitle: "[test title]", Locator: store.Locator{URL: "http://example.org/"}}
-	c := store.Comment{Text: "some text", ParentID: "1", ID: "999", Locator: store.Locator{URL: "http://example.org/"}}
 	c.User.Name = "from"
 	cp := store.Comment{Text: `<p>some parent text with a <a href="http://example.org">link</a> and special text:<br>& < > &</p>`}
 	cp.User.Name = "to"
 
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
+	err := tb.Send(context.Background(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
-	assert.NoError(t, err)
-	c.PostTitle = "test title"
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
-	assert.NoError(t, err)
-
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
-	assert.NoError(t, err)
-	c.PostTitle = "[test title]"
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp})
-	assert.NoError(t, err)
-
-	tb, err = NewTelegram(TelegramParams{
-		AdminChannelID:    "remark_test",
-		Token:             "non-json-resp",
-		UserNotifications: true,
-		apiPrefix:         ts.URL + "/",
-	})
-	assert.Nil(t, tb)
-	assert.Error(t, err, "should fail")
-	tb = &Telegram{
-		TelegramParams: TelegramParams{
-			AdminChannelID:    "remark_test",
-			Token:             "non-json-resp",
-			UserNotifications: true,
-			apiPrefix:         ts.URL + "/",
-		}}
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
-	require.Error(t, err)
-	assert.Contains(t, err.Error(), "unexpected telegram API status code 404", "send on broken tg")
-
-	assert.Equal(t, "telegram with admin notifications to remark_test with user notifications enabled", tb.String())
-
-	// bad API URL
-	tb.apiPrefix = "http://non-existent"
-	err = tb.Send(context.TODO(), Request{Comment: c, parent: cp, Telegrams: []string{"test_user_channel"}})
 	assert.Error(t, err)
+	assert.Contains(t, err.Error(), "2 errors occurred")
+	assert.Contains(t, err.Error(), "problem sending user telegram notification about comment ID 999 to \"test_user_channel\"")
+	assert.Contains(t, err.Error(), "problem sending admin telegram notification about comment ID 999 to remark_test")
 
 	// test buildMessage separately for message text
-	res, err := buildMessage(Request{Comment: c, parent: cp})
+	res := tb.buildMessage(Request{Comment: c, parent: cp})
-	assert.NoError(t, err)
+	assert.Equal(t, `<a href="http://example.org/#remark42__comment-999">from</a> -> <a href="http://example.org/#remark42__comment-">to</a>
-	assert.Equal(t, `{"text":"\u003ca href=\"http://example.org/#remark42__comment-999\"\u003efrom\u003c/a\u003e -\u003e \u003ca href=\"http://example.org/#remark42__comment-\"\u003eto\u003c/a\u003e\n\n`+
+
-		`some text\n\n`+
+some text
-		`\"\u003ci\u003esome parent text with a \u003ca href=\"http://example.org\"\u003elink\u003c/a\u003e and special text:\u0026amp; \u0026lt; \u0026gt; \u0026amp;\u003c/i\u003e\"\n\n`+
+
-		`↦  \u003ca href=\"http://example.org/\"\u003e[test title]\u003c/a\u003e","parse_mode":"HTML"}`,
+"<i>some parent text with a <a href="http://example.org">link</a> and special text:&amp; &lt; &gt; &amp;</i>"
-		string(res))
+
+↦  <a href="http://example.org/">[test title]</a>`,
+		res)
 
 	// special case for text with h1-h6 header
 	ch := store.Comment{Text: "<h1>Hello</h1><h6>World</h6>", ID: "555", Locator: store.Locator{URL: "http://example.org/"}}
 	ch.User.Name = "from"
-	res, err = buildMessage(Request{Comment: ch})
+	res = tb.buildMessage(Request{Comment: ch})
-	assert.NoError(t, err)
+	assert.Equal(t, `<a href="http://example.org/#remark42__comment-555">from</a>
-	assert.Equal(t, `{"text":"\u003ca href=\"http://example.org/#remark42__comment-555\"\u003efrom\u003c/a\u003e\n\n`+
+
-		`\u003cb\u003eHello\u003c/b\u003e\u003ci\u003e\u003cb\u003eWorld\u003c/b\u003e\u003c/i\u003e`+
+<b>Hello</b><i><b>World</b></i>`,
-		`","parse_mode":"HTML"}`,
+		res)
-		string(res))
+
+	// prune string keeping HTML closing tags
+	c = store.Comment{
+		Text: "<b>Lorem ipsum <i>dolor sit amet</i>, consectetur adipiscing <code>elit, sed do eiusmod tempor incididunt</code> ut labore et dolore magna aliqua.</b>",
+	}
+	res = tb.buildMessage(Request{Comment: c})
+	assert.Equal(t, `<a href="#remark42__comment-"></a>
+
+<b>Lorem ipsum <i>dolor sit amet</i>, consectetur adipiscing <code>elit, sed do eiusmod tempor incididunt</code> ut...</b>`, res)
 }
 
 func TestTelegram_SendVerification(t *testing.T) {
-	ts := mockTelegramServer(nil)
+	tb := Telegram{}
-	defer ts.Close()
+	// empty VerificationRequest should return no error and do nothing, as well as any other
-
+	assert.NoError(t, tb.SendVerification(context.Background(), VerificationRequest{}))
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "good-token",
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-
-	// empty VerificationRequest should return no error and no nothing, as well as any other
-	assert.NoError(t, tb.SendVerification(context.TODO(), VerificationRequest{}))
-}
-
-const getUpdatesResp = `{
-  "ok": true,
-  "result": [
-     {
-        "update_id": 998,
-        "message": {
-           "chat": {
-              "type": "group"
-           }
-        }
-     },
-     {
-        "update_id": 999,
-        "message": {
-					 "text": "not starting with /start",
-           "chat": {
-              "type": "private"
-           }
-        }
-     },
-     {
-        "update_id": 1000,
-        "message": {
-           "message_id": 4,
-           "from": {
-              "id": 313131313,
-              "is_bot": false,
-              "first_name": "Joe",
-              "username": "joe123",
-              "language_code": "en"
-           },
-           "chat": {
-              "id": 313131313,
-              "first_name": "Joe",
-              "username": "joe123",
-              "type": "private"
-           },
-           "date": 1601665548,
-           "text": "/start token",
-           "entities": [
-              {
-                 "offset": 0,
-                 "length": 6,
-                 "type": "bot_command"
-              }
-           ]
-        }
-     }
-  ]
-}`
-
-func TestTelegram_GetUpdatesFlow(t *testing.T) {
-	first := true
-	ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
-		if strings.Contains(r.URL.String(), "sendMessage") {
-			// respond normally to processUpdates attempt to send message back to user
-			_, _ = w.Write([]byte("{}"))
-			return
-		}
-		// responses to get updates calls to API
-		if first {
-			assert.Equal(t, "", r.URL.Query().Get("offset"))
-			first = false
-		} else {
-			assert.Equal(t, "1001", r.URL.Query().Get("offset"))
-		}
-		_, _ = w.Write([]byte(getUpdatesResp))
-	})
-	defer ts.Close()
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID:    "remark_test",
-		Token:             "xxxsupersecretxxx",
-		UserNotifications: true,
-		apiPrefix:         ts.URL + "/",
-	})
-	assert.NoError(t, err)
-
-	// send request with no offset
-	upd, err := tb.getUpdates(context.Background())
-	assert.NoError(t, err)
-
-	assert.Len(t, upd.Result, 3)
-	assert.Equal(t, 1001, tb.updateOffset)
-	assert.Equal(t, "/start token", upd.Result[len(upd.Result)-1].Message.Text)
-
-	tb.AddToken("token", "user", "site", time.Now().Add(time.Minute))
-	_, _, err = tb.CheckToken("token", "user")
-	assert.Error(t, err)
-	tb.processUpdates(context.Background(), upd)
-	tgID, site, err := tb.CheckToken("token", "user")
-	assert.NoError(t, err)
-	assert.Equal(t, "313131313", tgID)
-	assert.Equal(t, "site", site)
-
-	// send request with offset
-	_, err = tb.getUpdates(context.Background())
-	assert.NoError(t, err)
-}
-
-func TestTelegram_ProcessUpdateFlow(t *testing.T) {
-	ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
-		// respond normally to processUpdates attempt to send message back to user
-		_, _ = w.Write([]byte("{}"))
-	})
-	defer ts.Close()
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID:    "remark_test",
-		Token:             "xxxsupersecretxxx",
-		UserNotifications: true,
-		apiPrefix:         ts.URL + "/",
-	})
-	assert.NoError(t, err)
-
-	tb.AddToken("token", "user", "site", time.Now().Add(time.Minute))
-	tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
-	assert.Len(t, tb.requests.data, 2)
-	_, _, err = tb.CheckToken("token", "user")
-	assert.Error(t, err)
-	assert.NoError(t, tb.ProcessUpdate(context.Background(), getUpdatesResp))
-	assert.Len(t, tb.requests.data, 1, "expired token was cleaned up")
-	tgID, site, err := tb.CheckToken("token", "user")
-	assert.NoError(t, err)
-	assert.Len(t, tb.requests.data, 0, "token is deleted after successful check")
-	assert.Equal(t, "313131313", tgID)
-	assert.Equal(t, "site", site)
-
-	tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
-	assert.Len(t, tb.requests.data, 1)
-	assert.EqualError(t, tb.ProcessUpdate(context.Background(), ""), "failed to decode provided telegram update: unexpected end of JSON input")
-	assert.Len(t, tb.requests.data, 0, "expired token should be cleaned up despite the error")
-}
-
-const sendMessageResp = `{
-  "ok": true,
-  "result": {
-     "message_id": 100,
-     "from": {
-        "id": 666666666,
-        "is_bot": true,
-        "first_name": "Test auth bot",
-        "username": "TestAuthBot"
-     },
-     "chat": {
-        "id": 313131313,
-        "first_name": "Joe",
-        "username": "joe123",
-        "type": "private"
-     },
-     "date": 1602430546,
-     "text": "123"
-  }
-}`
-
-func TestTelegram_SendText(t *testing.T) {
-	ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
-		assert.Equal(t, "123", r.URL.Query().Get("chat_id"))
-		assert.Equal(t, "hello there", r.URL.Query().Get("text"))
-		_, _ = w.Write([]byte(sendMessageResp))
-	})
-	defer ts.Close()
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID:    "remark_test",
-		Token:             "xxxsupersecretxxx",
-		UserNotifications: true,
-		apiPrefix:         ts.URL + "/",
-	})
-	assert.NoError(t, err)
-
-	err = tb.sendText(context.Background(), 123, "hello there")
-	assert.NoError(t, err)
-}
-
-const errorResp = `{"ok":false,"error_code":400,"description":"Very bad request"}`
-
-func TestTelegram_Error(t *testing.T) {
-	ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
-		w.WriteHeader(http.StatusBadRequest)
-		_, _ = w.Write([]byte(errorResp))
-	})
-	defer ts.Close()
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID:    "remark_test",
-		Token:             "xxxsupersecretxxx",
-		UserNotifications: true,
-		apiPrefix:         ts.URL + "/",
-	})
-	assert.NoError(t, err)
-
-	_, err = tb.getUpdates(context.Background())
-	assert.EqualError(t, err, "failed to fetch updates: unexpected telegram API status code 400, error: \"Very bad request\"")
-}
-
-func TestTelegram_TokenVerification(t *testing.T) {
-	ts := mockTelegramServer(func(w http.ResponseWriter, r *http.Request) {
-		if strings.Contains(r.URL.String(), "sendMessage") {
-			// respond normally to processUpdates attempt to send message back to user
-			_, _ = w.Write([]byte("{}"))
-			return
-		}
-		// responses to get updates calls to API
-		_, _ = w.Write([]byte(getUpdatesResp))
-	})
-	defer ts.Close()
-
-	tb, err := NewTelegram(TelegramParams{
-		AdminChannelID: "remark_test",
-		Token:          "good-token",
-		apiPrefix:      ts.URL + "/",
-	})
-	assert.NoError(t, err)
-	assert.NotNil(t, tb)
-	tb.AddToken("token", "user", "site", time.Now().Add(time.Minute))
-	assert.Len(t, tb.requests.data, 1)
-
-	// wrong token
-	tgID, site, err := tb.CheckToken("unknown token", "user")
-	assert.Empty(t, tgID)
-	assert.Empty(t, site)
-	assert.EqualError(t, err, "request is not found")
-
-	// right token and user, not verified yet
-	tgID, site, err = tb.CheckToken("token", "user")
-	assert.Empty(t, tgID)
-	assert.Empty(t, site)
-	assert.EqualError(t, err, "request is not verified yet")
-
-	// confirm request
-	authRequest, ok := tb.requests.data["token"]
-	assert.True(t, ok)
-	authRequest.confirmed = true
-	authRequest.telegramID = "telegramID"
-	tb.requests.data["token"] = authRequest
-
-	// wrong user
-	tgID, site, err = tb.CheckToken("token", "wrong user")
-	assert.Empty(t, tgID)
-	assert.Empty(t, site)
-	assert.EqualError(t, err, "user does not match original requester")
-
-	// successful check
-	tgID, site, err = tb.CheckToken("token", "user")
-	assert.NoError(t, err)
-	assert.Equal(t, "telegramID", tgID)
-	assert.Equal(t, "site", site)
-
-	// expired token
-	tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
-	tgID, site, err = tb.CheckToken("expired token", "user")
-	assert.Empty(t, tgID)
-	assert.Empty(t, site)
-	assert.EqualError(t, err, "request expired")
-	assert.Len(t, tb.requests.data, 0)
-
-	// expired token, cleaned up by the cleanup
-	tb.apiPollInterval = time.Millisecond * 15
-	tb.expiredCleanupInterval = time.Millisecond * 10
-	ctx, cancel := context.WithCancel(context.Background())
-	go tb.Run(ctx)
-	assert.Eventually(t, func() bool {
-		return tb.ProcessUpdate(ctx, "").Error() == "Run goroutine should not be used with ProcessUpdate"
-	}, time.Millisecond*100, time.Millisecond*10, "ProcessUpdate should not work same time as Run")
-	tb.AddToken("expired token", "user", "site", time.Now().Add(-time.Minute))
-	tb.requests.RLock()
-	assert.Len(t, tb.requests.data, 1)
-	tb.requests.RUnlock()
-	time.Sleep(tb.expiredCleanupInterval * 2)
-	tb.requests.RLock()
-	assert.Len(t, tb.requests.data, 0)
-	tb.requests.RUnlock()
-	cancel()
-	// give enough time for Run() to finish
-	time.Sleep(tb.expiredCleanupInterval)
-}
-
-const getMeResp = `{"ok": true,
-				"result": {
-					"first_name": "comments_test",
-					"id": 707381019,
-					"is_bot": true,
-					"username": "remark42_test_bot"
-				}}`
-
-func mockTelegramServer(h http.HandlerFunc) *httptest.Server {
-	if h != nil {
-		return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			if strings.Contains(r.URL.String(), "getMe") {
-				_, _ = w.Write([]byte(getMeResp))
-				return
-			}
-			h(w, r)
-		}))
-	}
-	router := chi.NewRouter()
-	router.Get("/good-token/getMe", func(w http.ResponseWriter, r *http.Request) {
-		_, _ = w.Write([]byte(getMeResp))
-	})
-	router.Get("/empty-json/getMe", func(w http.ResponseWriter, r *http.Request) {
-		_, _ = w.Write([]byte(`{}`))
-	})
-	router.Get("/non-json-resp/getMe", func(w http.ResponseWriter, r *http.Request) {
-		_, _ = w.Write([]byte(`not-a-json`))
-	})
-	router.Get("/404/getMe", func(w http.ResponseWriter, r *http.Request) {
-		w.WriteHeader(404)
-	})
-
-	router.Post("/good-token/sendMessage", func(w http.ResponseWriter, r *http.Request) {
-		_, _ = w.Write([]byte(`{"ok": true}`))
-	})
-
-	return httptest.NewServer(router)
 }

backend/app/notify/webhook.go

@@ -3,111 +3,92 @@ package notify
 import (
 	"bytes"
 	"context"
+	"encoding/json"
 	"fmt"
-	"io"
-	"net/http"
-	"strings"
 	"text/template"
+	"time"
 
 	log "github.com/go-pkgz/lgr"
-	"github.com/pkg/errors"
+	ntf "github.com/go-pkgz/notify"
 )
 
 const (
-	webhookDefaultTemplate = `{"text": "{{.Text}}"}`
+	webhookDefaultTemplate = `{"text": {{.Text | escapeJSONString}}}`
 )
 
-// WebhookClient defines an interface of client for webhook
-type WebhookClient interface {
-	Do(*http.Request) (*http.Response, error)
-}
-
 // WebhookParams contain settings for webhook notifications
 type WebhookParams struct {
-	WebhookURL string
+	URL      string
-	Template   string
+	Template string
-	Headers    []string
+	Headers  []string
+	Timeout  time.Duration
 }
 
 // Webhook implements notify.Destination for Webhook notifications
 type Webhook struct {
-	WebhookParams
+	*ntf.Webhook
-	webhookClient   WebhookClient
+
-	webhookTemplate *template.Template
+	url      string
+	template *template.Template
 }
 
 // NewWebhook makes Webhook
-func NewWebhook(client WebhookClient, params WebhookParams) (*Webhook, error) {
+func NewWebhook(params WebhookParams) (*Webhook, error) {
-	res := &Webhook{WebhookParams: params}
+	res := &Webhook{
-	if res.WebhookURL == "" {
+		Webhook: ntf.NewWebhook(ntf.WebhookParams{
-		return nil, errors.New("webhook URL is required for webhook notifications")
+			Timeout: params.Timeout,
+			Headers: params.Headers,
+		}),
+		url: params.URL,
 	}
 
-	if res.Template == "" {
+	if res.url == "" {
-		res.Template = webhookDefaultTemplate
+		return nil, fmt.Errorf("webhook URL is required for webhook notifications")
 	}
 
-	payloadTmpl, err := template.New("webhook").Parse(res.Template)
+	if params.Template == "" {
+		params.Template = webhookDefaultTemplate
+	}
+
+	payloadTmpl, err := template.New("webhook").Funcs(template.FuncMap{"escapeJSONString": escapeJSONString}).Parse(params.Template)
 	if err != nil {
-		return nil, errors.Wrap(err, "unable to parse webhook template")
+		return nil, fmt.Errorf("unable to parse webhook template: %w", err)
 	}
 
-	res.webhookClient = client
+	res.template = payloadTmpl
-	res.webhookTemplate = payloadTmpl
 
-	log.Printf("[DEBUG] create new webhook notifier for %s", res.WebhookURL)
+	log.Printf("[DEBUG] create new webhook notifier for %s", res.url)
 
 	return res, nil
 }
 
 // Send sends Webhook notification
-func (t *Webhook) Send(ctx context.Context, req Request) error {
+func (w *Webhook) Send(ctx context.Context, req Request) error {
-	var payload bytes.Buffer
-	err := t.webhookTemplate.Execute(&payload, req.Comment)
-	if err != nil {
-		return errors.Wrap(err, "unable to compile webhook template")
-	}
-
-	httpReq, err := http.NewRequestWithContext(ctx, "POST", t.WebhookURL, &payload)
-	if err != nil {
-		return errors.Wrap(err, "unable to create webhook request")
-	}
-
-	for _, h := range t.Headers {
-		elems := strings.Split(h, ":")
-		if len(elems) != 2 {
-			continue
-		}
-		httpReq.Header.Set(strings.TrimSpace(elems[0]), strings.TrimSpace(elems[1]))
-	}
-
-	resp, err := t.webhookClient.Do(httpReq)
-	if err != nil {
-		return errors.Wrap(err, "webhook request failed")
-	}
-
-	defer resp.Body.Close()
-
-	if resp.StatusCode != http.StatusOK {
-		errMsg := fmt.Sprintf("webhook request failed with non-OK status code: %d", resp.StatusCode)
-		respBody, e := io.ReadAll(resp.Body)
-		if e != nil {
-			return fmt.Errorf(errMsg)
-		}
-		return fmt.Errorf("%s, body: %s", errMsg, respBody)
-	}
-
 	log.Printf("[DEBUG] send webhook notification, comment id %s", req.Comment.ID)
+	var payload bytes.Buffer
+	err := w.template.Execute(&payload, req.Comment)
+	if err != nil {
+		return fmt.Errorf("unable to compile webhook template: %w", err)
+	}
 
-	return nil
+	return w.Webhook.Send(ctx, w.url, payload.String())
 }
 
 // SendVerification is not implemented for Webhook
-func (t *Webhook) SendVerification(_ context.Context, _ VerificationRequest) error {
+func (w *Webhook) SendVerification(_ context.Context, _ VerificationRequest) error {
 	return nil
 }
 
 // String describes the webhook instance
-func (t *Webhook) String() string {
+func (w *Webhook) String() string {
-	return fmt.Sprintf("webhook notification to %s", t.WebhookURL)
+	return fmt.Sprintf("%s to %s", w.Webhook.String(), w.url)
+}
+
+// escapeJSONString escapes string for JSON
+func escapeJSONString(s string) (string, error) {
+	b, err := json.Marshal(s)
+	if err != nil {
+		return "", err
+	}
+	return string(b), nil
 }

backend/app/notify/webhook_test.go

@@ -1,12 +1,12 @@
 package notify
 
 import (
-	"bytes"
 	"context"
-	"errors"
 	"io"
 	"net/http"
+	"net/http/httptest"
 	"testing"
+	"time"
 
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
@@ -14,72 +14,59 @@ import (
 	"github.com/umputun/remark42/backend/app/store"
 )
 
-type funcWebhookClient func(*http.Request) (*http.Response, error)
-
-func (c funcWebhookClient) Do(r *http.Request) (*http.Response, error) {
-	return c(r)
-}
-
-var okWebhookClient = funcWebhookClient(func(*http.Request) (*http.Response, error) {
-	return &http.Response{
-		StatusCode: http.StatusOK,
-		Body:       io.NopCloser(bytes.NewBufferString("ok")),
-	}, nil
-})
-
-type errReader struct {
-}
-
-func (errReader) Read(p []byte) (n int, err error) {
-	return 0, errors.New("test error")
-}
-
 func TestWebhook_NewWebhook(t *testing.T) {
-
+	wh, err := NewWebhook(WebhookParams{
-	wh, err := NewWebhook(okWebhookClient, WebhookParams{
+		URL:     "https://example.org/webhook",
-		WebhookURL: "https://example.org/webhook",
+		Headers: []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="},
-		Headers:    []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="},
 	})
 	assert.NoError(t, err)
 	assert.NotNil(t, wh)
 
-	assert.Equal(t, "https://example.org/webhook", wh.WebhookURL)
+	assert.Equal(t, "https://example.org/webhook", wh.url)
 	assert.Equal(t, []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="}, wh.Headers)
-	assert.Equal(t, `{"text": "{{.Text}}"}`, wh.Template)
+	assert.NotNil(t, wh.template)
 
-	wh, err = NewWebhook(okWebhookClient, WebhookParams{
+	wh, err = NewWebhook(WebhookParams{})
-		WebhookURL: "https://example.org/webhook",
-		Headers:    []string{"Authorization:Basic AXVubzpwQDU1dzByYM=="},
-		Template:   "{{.Text}}",
-	})
-	assert.NoError(t, err)
-	assert.NotNil(t, wh)
-	assert.Equal(t, "{{.Text}}", wh.Template)
-
-	wh, err = NewWebhook(okWebhookClient, WebhookParams{})
 	assert.Nil(t, wh)
 	assert.Error(t, err)
 	assert.Equal(t, "webhook URL is required for webhook notifications", err.Error())
 
-	wh, err = NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook", Template: "{{.Text"})
+	wh, err = NewWebhook(WebhookParams{URL: "https://example.org/webhook", Template: "{{.Text"})
 	assert.Nil(t, wh)
 	assert.Error(t, err)
 	assert.Contains(t, err.Error(), "unable to parse webhook template")
 }
 
+// https://github.com/umputun/remark42/issues/1791
+func TestWebhook_ReceiveValidJSON(t *testing.T) {
+	ts := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
+		assert.Equal(t, r.URL.Path, "/webhook-notify")
+		assert.Equal(t, "POST", r.Method)
+		body, err := io.ReadAll(r.Body)
+		assert.NoError(t, err)
+		t.Log("received body", string(body))
+		assert.JSONEq(t, `{"text": "<p>testme</p>\n"}`, string(body))
+	}))
+	defer ts.Close()
+
+	wh, err := NewWebhook(WebhookParams{
+		URL:     ts.URL + "/webhook-notify",
+		Headers: []string{"Content-Type:application/json,text/plain"},
+	})
+	assert.NoError(t, err)
+	assert.NotNil(t, wh)
+
+	f := store.NewCommentFormatter()
+	c := store.Comment{Text: f.FormatText("testme", false), ParentID: "1", ID: "999"}
+
+	err = wh.Send(context.Background(), Request{Comment: c})
+	assert.NoError(t, err)
+}
+
 func TestWebhook_Send(t *testing.T) {
-
+	wh, err := NewWebhook(WebhookParams{
-	wh, err := NewWebhook(funcWebhookClient(func(r *http.Request) (*http.Response, error) {
+		URL:     "bad-url",
-		assert.Len(t, r.Header, 1)
+		Headers: []string{"Content-Type:application/json,text/plain", ""},
-		assert.Equal(t, r.Header.Get("Content-Type"), "application/json,text/plain")
-
-		return &http.Response{
-			StatusCode: http.StatusOK,
-			Body:       io.NopCloser(bytes.NewBufferString("")),
-		}, nil
-	}), WebhookParams{
-		WebhookURL: "https://example.org/webhook",
-		Headers:    []string{"Content-Type:application/json,text/plain", ""},
 	})
 	assert.NoError(t, err)
 	assert.NotNil(t, wh)
@@ -87,74 +74,45 @@ func TestWebhook_Send(t *testing.T) {
 	c := store.Comment{Text: "some text", ParentID: "1", ID: "999"}
 	c.User.Name = "from"
 
-	err = wh.Send(context.TODO(), Request{Comment: c})
+	err = wh.Send(context.Background(), Request{Comment: c})
-	assert.NoError(t, err)
+	assert.Error(t, err)
 
-	wh, err = NewWebhook(okWebhookClient, WebhookParams{
+	wh, err = NewWebhook(WebhookParams{
-		WebhookURL: "https://example.org/webhook",
+		URL:      "https://example.org/webhook",
-		Template:   "{{.InvalidProperty}}",
+		Template: "{{.InvalidProperty}}",
 	})
 	assert.NoError(t, err)
-	err = wh.Send(context.TODO(), Request{Comment: c})
+	err = wh.Send(context.Background(), Request{Comment: c})
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "webhook template")
 
-	wh, err = NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook"})
+	wh, err = NewWebhook(WebhookParams{URL: "https://example.org/webhook"})
 	assert.NoError(t, err)
 	err = wh.Send(nil, Request{Comment: c}) // nolint
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "unable to create webhook request")
 
-	wh, err = NewWebhook(funcWebhookClient(func(*http.Request) (*http.Response, error) {
+	wh, err = NewWebhook(WebhookParams{URL: "https://not-existing-url.net"})
-		return nil, errors.New("request failed")
-	}), WebhookParams{WebhookURL: "https://not-existing-url.net"})
 	assert.NoError(t, err)
-	err = wh.Send(context.TODO(), Request{Comment: c})
+	err = wh.Send(context.Background(), Request{Comment: c})
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "webhook request failed")
-
-	wh, err = NewWebhook(funcWebhookClient(func(*http.Request) (*http.Response, error) {
-		return &http.Response{
-			StatusCode: http.StatusNotFound,
-			Body:       io.NopCloser(bytes.NewBufferString("not found")),
-		}, nil
-	}), WebhookParams{
-		WebhookURL: "http:/example.org/invalid-url",
-	})
-	assert.NoError(t, err)
-	err = wh.Send(context.TODO(), Request{Comment: c})
-	require.Error(t, err)
-	assert.Contains(t, err.Error(), "non-OK status code: 404, body: not found")
-
-	wh, err = NewWebhook(funcWebhookClient(func(*http.Request) (*http.Response, error) {
-		return &http.Response{
-			StatusCode: http.StatusNotFound,
-			Body:       io.NopCloser(errReader{}),
-		}, nil
-	}), WebhookParams{
-		WebhookURL: "http:/example.org/invalid-url",
-	})
-	assert.NoError(t, err)
-	err = wh.Send(context.TODO(), Request{Comment: c})
-	require.Error(t, err)
-	assert.Contains(t, err.Error(), "non-OK status code: 404")
-	assert.NotContains(t, err.Error(), "body")
 }
 
 func TestWebhook_SendVerification(t *testing.T) {
-	wh, err := NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook"})
+	wh, err := NewWebhook(WebhookParams{URL: "https://example.org/webhook"})
 	assert.NoError(t, err)
 	assert.NotNil(t, wh)
 
-	err = wh.SendVerification(context.TODO(), VerificationRequest{})
+	err = wh.SendVerification(context.Background(), VerificationRequest{})
 	assert.NoError(t, err)
 }
 
 func TestWebhook_String(t *testing.T) {
-	wh, err := NewWebhook(okWebhookClient, WebhookParams{WebhookURL: "https://example.org/webhook"})
+	wh, err := NewWebhook(WebhookParams{URL: "https://example.org/webhook", Timeout: time.Minute * 5})
 	assert.NoError(t, err)
 	assert.NotNil(t, wh)
 
 	str := wh.String()
-	assert.Equal(t, "webhook notification to https://example.org/webhook", str)
+	assert.Equal(t, "webhook notification with timeout 5m0s to https://example.org/webhook", str)
 }

backend/app/providers/telegram.go

@@ -11,12 +11,11 @@ import (
 	"time"
 
 	log "github.com/go-pkgz/lgr"
-
+	ntf "github.com/go-pkgz/notify"
-	"github.com/umputun/remark42/backend/app/notify"
 )
 
 type tgRequester interface {
-	Request(ctx context.Context, method string, b []byte, data interface{}) error
+	Request(ctx context.Context, method string, b []byte, data any) error
 }
 
 // TGUpdatesReceiver used to dispatch telegram updates to multiple receivers
@@ -28,8 +27,8 @@ type TGUpdatesReceiver interface {
 // DispatchTelegramUpdates dispatches telegram updates to provided list of receivers
 // Blocks caller
 func DispatchTelegramUpdates(ctx context.Context, requester tgRequester, receivers []TGUpdatesReceiver, period time.Duration) {
-	// Identifier of the first update to be requested.
+	// identifier of the first update to be requested.
-	// Should be equal to LastSeenUpdateID + 1
+	// should be equal to LastSeenUpdateID + 1
 	// See https://core.telegram.org/bots/api#getupdates
 	var updateOffset int
 
@@ -45,7 +44,7 @@ func DispatchTelegramUpdates(ctx context.Context, requester tgRequester, receive
 				url += fmt.Sprintf("&offset=%d", updateOffset)
 			}
 
-			var update notify.TelegramUpdate
+			var update ntf.TelegramUpdate
 
 			err := requester.Request(ctx, url, nil, &update)
 			if err != nil {

backend/app/providers/telegram_test.go

@@ -3,22 +3,24 @@ package providers
 import (
 	"context"
 	"encoding/json"
-	"errors"
+	"fmt"
 	"testing"
+	"testing/synctest"
 	"time"
 
+	ntf "github.com/go-pkgz/notify"
 	"github.com/stretchr/testify/assert"
-
-	"github.com/umputun/remark42/backend/app/notify"
 )
 
 func TestDispatchTelegramUpdates(t *testing.T) {
-	ctx, cancel := context.WithCancel(context.Background())
+	synctest.Test(t, func(t *testing.T) {
-	poolPeriod := time.Millisecond * 100
+		ctx, cancel := context.WithCancel(context.Background())
-	go DispatchTelegramUpdates(ctx, &mockTGRequester{t: t}, []TGUpdatesReceiver{&mockTGUpdatesReceiver{t: t}}, poolPeriod)
+		poolPeriod := time.Millisecond * 100
-	time.Sleep(poolPeriod * 3)
+		go DispatchTelegramUpdates(ctx, &mockTGRequester{t: t}, []TGUpdatesReceiver{&mockTGUpdatesReceiver{t: t}}, poolPeriod)
-	cancel()
+		time.Sleep(poolPeriod * 3)
-	time.Sleep(poolPeriod)
+		cancel()
+		synctest.Wait()
+	})
 }
 
 const getUpdatesResp = `{
@@ -40,13 +42,13 @@ type mockTGRequester struct {
 	t   *testing.T
 }
 
-func (m *mockTGRequester) Request(_ context.Context, _ string, _ []byte, data interface{}) error {
+func (m *mockTGRequester) Request(_ context.Context, _ string, _ []byte, data any) error {
 	if m.hit < 2 {
 		m.hit++
 		assert.NoError(m.t, json.Unmarshal([]byte(getUpdatesResp), data))
 		return nil
 	}
-	return errors.New("test error")
+	return fmt.Errorf("test error")
 }
 
 type mockTGUpdatesReceiver struct {
@@ -59,7 +61,7 @@ func (m *mockTGUpdatesReceiver) String() string {
 }
 
 func (m *mockTGUpdatesReceiver) ProcessUpdate(_ context.Context, textUpdate string) error {
-	var result notify.TelegramUpdate
+	var result ntf.TelegramUpdate
 	err := json.Unmarshal([]byte(textUpdate), &result)
 	assert.NoError(m.t, err)
 	if m.hit < 2 {
@@ -68,5 +70,5 @@ func (m *mockTGUpdatesReceiver) ProcessUpdate(_ context.Context, textUpdate stri
 		return nil
 	}
 	assert.Nil(m.t, result.Result)
-	return errors.New("test error")
+	return fmt.Errorf("test error")
 }

backend/app/rest/api/admin.go

@@ -1,15 +1,14 @@
 package api
 
 import (
-	"errors"
+	"fmt"
 	"net/http"
 	"path"
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/go-chi/render"
+	"github.com/go-pkgz/auth/v2"
-	"github.com/go-pkgz/auth"
+	cache "github.com/go-pkgz/lcw/v2"
-	cache "github.com/go-pkgz/lcw"
 	log "github.com/go-pkgz/lgr"
 	R "github.com/go-pkgz/rest"
 
@@ -29,22 +28,21 @@ type admin struct {
 
 type adminStore interface {
 	Delete(locator store.Locator, commentID string, mode store.DeleteMode) error
-	DeleteUser(siteID string, userID string, mode store.DeleteMode) error
+	DeleteUser(siteID, userID string, mode store.DeleteMode) error
-	DeleteUserDetail(siteID string, userID string, detail engine.UserDetail) error
+	DeleteUserDetail(siteID, userID string, detail engine.UserDetail) error
 	User(siteID, userID string, limit, skip int, user store.User) ([]store.Comment, error)
-	IsBlocked(siteID string, userID string) bool
+	IsBlocked(siteID, userID string) bool
-	SetBlock(siteID string, userID string, status bool, ttl time.Duration) error
+	SetBlock(siteID, userID string, status bool, ttl time.Duration) error
 	BlockedUsers(siteID string) ([]store.BlockedUser, error)
 	Info(locator store.Locator, readonlyAge int) (store.PostInfo, error)
 	SetTitle(locator store.Locator, commentID string) (comment store.Comment, err error)
-	SetVerified(siteID string, userID string, status bool) error
+	SetVerified(siteID, userID string, status bool) error
 	SetReadOnly(locator store.Locator, status bool) error
 	SetPin(locator store.Locator, commentID string, status bool) error
 }
 
 // DELETE /comment/{id}?site=siteID&url=post-url - removes comment
 func (a *admin) deleteCommentCtrl(w http.ResponseWriter, r *http.Request) {
-
 	id := chi.URLParam(r, "id")
 	locator := store.Locator{SiteID: r.URL.Query().Get("site"), URL: r.URL.Query().Get("url")}
 	log.Printf("[INFO] delete comment %s", id)
@@ -55,13 +53,11 @@ func (a *admin) deleteCommentCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	a.cache.Flush(cache.Flusher(locator.SiteID).Scopes(locator.SiteID, locator.URL, lastCommentsScope))
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, R.JSON{"id": id, "locator": locator})
-	render.JSON(w, r, R.JSON{"id": id, "locator": locator})
 }
 
 // DELETE /user/{userid}?site=side-id - delete all user comments for requested userid
 func (a *admin) deleteUserCtrl(w http.ResponseWriter, r *http.Request) {
-
 	userID := chi.URLParam(r, "userid")
 	siteID := r.URL.Query().Get("site")
 	log.Printf("[INFO] delete all user comments for %s, site %s", userID, siteID)
@@ -71,13 +67,11 @@ func (a *admin) deleteUserCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	a.cache.Flush(cache.Flusher(siteID).Scopes(userID, siteID, lastCommentsScope))
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, R.JSON{"user_id": userID, "site_id": siteID})
-	render.JSON(w, r, R.JSON{"user_id": userID, "site_id": siteID})
 }
 
 // GET /user/{userid}?site=side-id - get user info for requested userid
 func (a *admin) getUserInfoCtrl(w http.ResponseWriter, r *http.Request) {
-
 	userID := chi.URLParam(r, "userid")
 	siteID := r.URL.Query().Get("site")
 	log.Printf("[INFO] get user info for %s, site %s", userID, siteID)
@@ -87,14 +81,12 @@ func (a *admin) getUserInfoCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get user info", rest.ErrInternal)
 		return
 	}
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, ucomments[0].User)
-	render.JSON(w, r, ucomments[0].User)
 }
 
 // GET /deleteme?token=jwt - delete all user comments and details by user's request. Gets info about deleted used from provided token
 // request made GET to allow direct click from the email sent by user
 func (a *admin) deleteMeRequestCtrl(w http.ResponseWriter, r *http.Request) {
-
 	token := r.URL.Query().Get("token")
 
 	claims, err := a.authenticator.TokenService().Parse(token)
@@ -107,17 +99,25 @@ func (a *admin) deleteMeRequestCtrl(w http.ResponseWriter, r *http.Request) {
 
 	// deleteme set by deleteMeCtrl, this check just to make sure we not trying to delete with leaked token
 	if !claims.User.BoolAttr("delete_me") {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("forbidden"), "can't use provided token", rest.ErrNoAccess)
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("forbidden"), "can't use provided token", rest.ErrNoAccess)
 		return
 	}
 
-	if err = a.dataService.DeleteUserDetail(claims.Audience, claims.User.ID, engine.AllUserDetails); err != nil {
+	// audience is a slice but we set it to a single element, and situation when there is no audience or there are more than one is unexpected
+	if len(claims.Audience) != 1 {
+		rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("bad request"), "can't process token, claims.Audience expected to be a single element but it's not", rest.ErrActionRejected)
+		return
+	}
+
+	audience := claims.Audience[0]
+
+	if err = a.dataService.DeleteUserDetail(audience, claims.User.ID, engine.AllUserDetails); err != nil {
 		code := parseError(err, rest.ErrInternal)
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't delete user details for user", code)
 		return
 	}
 
-	if err = a.dataService.DeleteUser(claims.Audience, claims.User.ID, store.HardDelete); err != nil {
+	if err = a.dataService.DeleteUser(audience, claims.User.ID, store.HardDelete); err != nil {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't delete user", rest.ErrNoAccess)
 		return
 	}
@@ -130,9 +130,8 @@ func (a *admin) deleteMeRequestCtrl(w http.ResponseWriter, r *http.Request) {
 		}
 	}
 
-	a.cache.Flush(cache.Flusher(claims.Audience).Scopes(claims.Audience, claims.User.ID, lastCommentsScope))
+	a.cache.Flush(cache.Flusher(audience).Scopes(audience, claims.User.ID, lastCommentsScope))
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, R.JSON{"user_id": claims.User.ID, "site_id": claims.Audience})
-	render.JSON(w, r, R.JSON{"user_id": claims.User.ID, "site_id": claims.Audience})
 }
 
 // PUT /user/{userid}?site=side-id&block=1&ttl=7d - block or unblock user
@@ -160,7 +159,7 @@ func (a *admin) setBlockCtrl(w http.ResponseWriter, r *http.Request) {
 		}
 	}
 	a.cache.Flush(cache.Flusher(siteID).Scopes(userID, siteID, lastCommentsScope))
-	render.JSON(w, r, R.JSON{"user_id": userID, "site_id": siteID, "block": blockStatus})
+	R.RenderJSON(w, R.JSON{"user_id": userID, "site_id": siteID, "block": blockStatus})
 }
 
 // GET /blocked?site=siteID - list blocked users
@@ -171,7 +170,7 @@ func (a *admin) blockedUsersCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get blocked users", rest.ErrSiteNotFound)
 		return
 	}
-	render.JSON(w, r, users)
+	R.RenderJSON(w, users)
 }
 
 // PUT /readonly?site=siteID&url=post-url&ro=1 - set or reset read-only status for the post
@@ -187,7 +186,7 @@ func (a *admin) setReadOnlyCtrl(w http.ResponseWriter, r *http.Request) {
 	// don't allow to reset ro for posts turned to ro by ReadOnlyAge
 	if !roStatus {
 		if info, e := a.dataService.Info(locator, a.readOnlyAge); e == nil && isRoByAge(info) {
-			rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"),
+			rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"),
 				"read-only due the age", rest.ErrActionRejected)
 			return
 		}
@@ -198,7 +197,7 @@ func (a *admin) setReadOnlyCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	a.cache.Flush(cache.Flusher(locator.SiteID).Scopes(locator.URL, locator.SiteID))
-	render.JSON(w, r, R.JSON{"locator": locator, "read-only": roStatus})
+	R.RenderJSON(w, R.JSON{"locator": locator, "read-only": roStatus})
 }
 
 // PUT /title/{id}?site=siteID&url=post-url - set comment PostTitle to page's title
@@ -214,8 +213,7 @@ func (a *admin) setTitleCtrl(w http.ResponseWriter, r *http.Request) {
 	log.Printf("[INFO] set comment's title %s to %q", id, c.PostTitle)
 
 	a.cache.Flush(cache.Flusher(locator.SiteID).Scopes(locator.URL, lastCommentsScope))
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, R.JSON{"id": id, "locator": locator})
-	render.JSON(w, r, R.JSON{"id": id, "locator": locator})
 }
 
 // PUT /verify?site=siteID&url=post-url&ro=1 - set or reset read-only status for the post
@@ -229,7 +227,7 @@ func (a *admin) setVerifyCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	a.cache.Flush(cache.Flusher(siteID).Scopes(siteID, userID))
-	render.JSON(w, r, R.JSON{"user": userID, "verified": verifyStatus})
+	R.RenderJSON(w, R.JSON{"user": userID, "verified": verifyStatus})
 }
 
 // PUT /pin/{id}?site=siteID&url=post-url&pin=1
@@ -244,5 +242,5 @@ func (a *admin) setPinCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	a.cache.Flush(cache.Flusher(locator.SiteID).Scopes(locator.URL))
-	render.JSON(w, r, R.JSON{"id": commentID, "locator": locator, "pin": pinStatus})
+	R.RenderJSON(w, R.JSON{"id": commentID, "locator": locator, "pin": pinStatus})
 }

backend/app/rest/api/admin_test.go

@@ -13,10 +13,10 @@ import (
 	"testing"
 	"time"
 
-	"github.com/go-pkgz/auth/token"
+	"github.com/go-pkgz/auth/v2/token"
-	cache "github.com/go-pkgz/lcw"
+	cache "github.com/go-pkgz/lcw/v2"
 	R "github.com/go-pkgz/rest"
-	"github.com/golang-jwt/jwt"
+	"github.com/golang-jwt/jwt/v5"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
@@ -38,7 +38,7 @@ func TestAdmin_Delete(t *testing.T) {
 
 	// check last comments
 	res, code := get(t, ts.URL+"/api/v1/last/2?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := []store.Comment{}
 	err := json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -65,10 +65,10 @@ func TestAdmin_Delete(t *testing.T) {
 	resp, err = sendReq(t, req, adminUmputunToken)
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	body, code := getWithDevAuth(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	cr := store.Comment{}
 	err = json.Unmarshal([]byte(body), &cr)
 	assert.NoError(t, err)
@@ -78,7 +78,7 @@ func TestAdmin_Delete(t *testing.T) {
 	time.Sleep(250 * time.Millisecond)
 	// check last comments updated
 	res, code = get(t, ts.URL+"/api/v1/last/2?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = []store.Comment{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -86,8 +86,8 @@ func TestAdmin_Delete(t *testing.T) {
 
 	// check count updated
 	res, code = get(t, ts.URL+"/api/v1/count?site=remark42&url=https://radio-t.com/blah")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
-	b := map[string]interface{}{}
+	b := map[string]any{}
 	err = json.Unmarshal([]byte(res), &b)
 	assert.NoError(t, err)
 	t.Logf("%#v", b)
@@ -111,7 +111,7 @@ func TestAdmin_Title(t *testing.T) {
 	ts, srv, teardown := startupT(t)
 	defer teardown()
 
-	srv.DataService.TitleExtractor = service.NewTitleExtractor(http.Client{Timeout: time.Second})
+	srv.DataService.TitleExtractor = service.NewTitleExtractor(http.Client{Timeout: time.Second}, []string{"127.0.0.1"})
 	tss := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		if r.URL.String() == "/post1" {
 			_, err := w.Write([]byte("<html><title>post1 blah 123</title><body> 2222</body></html>"))
@@ -142,10 +142,10 @@ func TestAdmin_Title(t *testing.T) {
 	resp, err := sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	body, code := get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=%s/post1", ts.URL, id1, tss.URL))
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	cr := store.Comment{}
 	err = json.Unmarshal([]byte(body), &cr)
 	assert.NoError(t, err)
@@ -177,11 +177,11 @@ func TestAdmin_DeleteUser(t *testing.T) {
 	resp, err := sendReq(t, req, adminUmputunToken)
 	assert.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	// all 3 comments here, but for id2 they deleted
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	cmntWithInfo := commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &cmntWithInfo)
 	assert.NoError(t, err)
@@ -219,6 +219,7 @@ func TestAdmin_Pin(t *testing.T) {
 
 	pin := func(val int) int {
 		client := http.Client{}
+		defer client.CloseIdleConnections()
 		req, err := http.NewRequest(http.MethodPut,
 			fmt.Sprintf("%s/api/v1/admin/pin/%s?site=remark42&url=https://radio-t.com/blah&pin=%d", ts.URL, id1, val), http.NoBody)
 		assert.NoError(t, err)
@@ -231,19 +232,19 @@ func TestAdmin_Pin(t *testing.T) {
 	}
 
 	code := pin(1)
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 
 	body, code := get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	cr := store.Comment{}
 	err := json.Unmarshal([]byte(body), &cr)
 	assert.NoError(t, err)
 	assert.True(t, cr.Pin)
 
 	code = pin(-1)
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	body, code = get(t, fmt.Sprintf("%s/api/v1/id/%s?site=remark42&url=https://radio-t.com/blah", ts.URL, id1))
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	cr = store.Comment{}
 	err = json.Unmarshal([]byte(body), &cr)
 	assert.NoError(t, err)
@@ -286,7 +287,7 @@ func TestAdmin_Block(t *testing.T) {
 
 	// block permanently
 	code, body := block(1, "")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	j := R.JSON{}
 	err := json.Unmarshal(body, &j)
 	assert.NoError(t, err)
@@ -299,7 +300,7 @@ func TestAdmin_Block(t *testing.T) {
 
 	// get last to confirm one comment deleted
 	bodyStr, code := get(t, ts.URL+"/api/v1/last/10?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	pi := []store.PostInfo{}
 	assert.NoError(t, json.Unmarshal([]byte(bodyStr), &pi))
 	assert.Equal(t, 1, len(pi), "last status updated, one comment left")
@@ -317,7 +318,7 @@ func TestAdmin_Block(t *testing.T) {
 	assert.Equal(t, []store.PostInfo{{URL: "https://radio-t.com/blah", Count: 1}}, pi)
 
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -327,7 +328,7 @@ func TestAdmin_Block(t *testing.T) {
 
 	// unblock
 	code, body = block(-1, "")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal(body, &j)
 	assert.NoError(t, err)
 	assert.Equal(t, false, j["block"])
@@ -335,11 +336,11 @@ func TestAdmin_Block(t *testing.T) {
 	// block with ttl
 	makeTwoComments()
 	code, _ = block(1, "50ms")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 
 	// get as regular user
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -347,11 +348,11 @@ func TestAdmin_Block(t *testing.T) {
 	assert.Equal(t, "test test #1", comments.Comments[2].Text, "comment not removed and not cleared")
 	assert.False(t, comments.Comments[2].Deleted, "not deleted")
 
-	srv.pubRest.cache = cache.NewScache(cache.NewNopCache()) // TODO: with lru cache it won't be refreshed and invalidated for long
+	srv.pubRest.cache = cache.NewScache[[]byte](cache.NewNopCache[[]byte]()) // TODO: with lru cache it won't be refreshed and invalidated for long
 	// time
 	time.Sleep(50 * time.Millisecond)
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -385,7 +386,7 @@ func TestAdmin_BlockedList(t *testing.T) {
 	res, err := sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, res.Body.Close())
-	assert.Equal(t, 200, res.StatusCode)
+	assert.Equal(t, http.StatusOK, res.StatusCode)
 
 	// block user2
 	req, err = http.NewRequest(http.MethodPut,
@@ -394,13 +395,13 @@ func TestAdmin_BlockedList(t *testing.T) {
 	res, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, res.Body.Close())
-	assert.Equal(t, 200, res.StatusCode)
+	assert.Equal(t, http.StatusOK, res.StatusCode)
 
 	req, err = http.NewRequest("GET", ts.URL+"/api/v1/admin/blocked?site=remark42", http.NoBody)
 	require.NoError(t, err)
 	res, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
-	require.Equal(t, 200, res.StatusCode)
+	require.Equal(t, http.StatusOK, res.StatusCode)
 	users := []store.BlockedUser{}
 	err = json.NewDecoder(res.Body).Decode(&users)
 	assert.NoError(t, err)
@@ -417,7 +418,7 @@ func TestAdmin_BlockedList(t *testing.T) {
 	require.NoError(t, err)
 	res, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
-	require.Equal(t, 200, res.StatusCode)
+	require.Equal(t, http.StatusOK, res.StatusCode)
 	users = []store.BlockedUser{}
 	err = json.NewDecoder(res.Body).Decode(&users)
 	assert.NoError(t, err)
@@ -450,11 +451,11 @@ func TestAdmin_ReadOnly(t *testing.T) {
 	resp, err := sendReq(t, req, "") // non-admin user
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 401, resp.StatusCode)
+	assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
 	resp, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
 	assert.NoError(t, err)
 	assert.True(t, info.ReadOnly)
@@ -464,7 +465,7 @@ func TestAdmin_ReadOnly(t *testing.T) {
 		Locator: store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}}
 	b, err := json.Marshal(c)
 	assert.NoError(t, err, "can't marshal comment %+v", c)
-	req, err = http.NewRequest("POST", ts.URL+"/api/v1/comment", bytes.NewBuffer(b))
+	req, err = http.NewRequest("POST", ts.URL+"/api/v1/comment?site=remark42", bytes.NewBuffer(b))
 	require.NoError(t, err)
 	resp, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
@@ -478,7 +479,7 @@ func TestAdmin_ReadOnly(t *testing.T) {
 	resp, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
 	assert.NoError(t, err)
 	assert.False(t, info.ReadOnly)
@@ -488,7 +489,7 @@ func TestAdmin_ReadOnly(t *testing.T) {
 		Locator: store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}}
 	b, err = json.Marshal(c)
 	assert.NoError(t, err, "can't marshal comment %+v", c)
-	req, err = http.NewRequest("POST", ts.URL+"/api/v1/comment", bytes.NewBuffer(b))
+	req, err = http.NewRequest("POST", ts.URL+"/api/v1/comment?site="+c.Locator.SiteID, bytes.NewBuffer(b))
 	require.NoError(t, err)
 	resp, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
@@ -508,18 +509,29 @@ func TestAdmin_ReadOnlyNoComments(t *testing.T) {
 	resp, err := sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	_, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
 	assert.Error(t, err)
 
+	// test format "tree"
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&format=tree")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
 	assert.Equal(t, 0, len(comments.Comments), "should have 0 comments")
 	assert.True(t, comments.Info.ReadOnly)
 	t.Logf("%+v", comments)
+
+	// test format "plain"
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah")
+	assert.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	assert.NoError(t, err)
+	assert.Equal(t, 0, len(comments.Comments), "should have 0 comments")
+	assert.True(t, comments.Info.ReadOnly)
+	t.Logf("%+v", comments)
 }
 
 func TestAdmin_ReadOnlyWithAge(t *testing.T) {
@@ -544,7 +556,7 @@ func TestAdmin_ReadOnlyWithAge(t *testing.T) {
 	resp, err := sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
 	assert.NoError(t, err)
 	assert.True(t, info.ReadOnly)
@@ -556,11 +568,10 @@ func TestAdmin_ReadOnlyWithAge(t *testing.T) {
 	resp, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 403, resp.StatusCode)
+	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 	info, err = srv.DataService.Info(store.Locator{SiteID: "remark42", URL: "https://radio-t.com/blah"}, 0)
 	assert.NoError(t, err)
 	assert.True(t, info.ReadOnly)
-
 }
 func TestAdmin_Verify(t *testing.T) {
 	ts, srv, teardown := startupT(t)
@@ -586,12 +597,12 @@ func TestAdmin_Verify(t *testing.T) {
 	resp, err := sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	verified = srv.DataService.IsVerified("remark42", "user1")
 	assert.True(t, verified)
 
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -605,12 +616,12 @@ func TestAdmin_Verify(t *testing.T) {
 	resp, err = sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	verified = srv.DataService.IsVerified("remark42", "user1")
 	assert.False(t, verified)
 
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -632,7 +643,7 @@ func TestAdmin_ExportStream(t *testing.T) {
 	addComment(t, c2, ts)
 
 	body, code := getWithAdminAuth(t, ts.URL+"/api/v1/admin/export?site=remark42&mode=stream")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	assert.Equal(t, 3, strings.Count(body, "\n"))
 	assert.Equal(t, 2, strings.Count(body, "\"text\""))
 	t.Logf("%s", body)
@@ -656,7 +667,7 @@ func TestAdmin_ExportFile(t *testing.T) {
 	resp, err := sendReq(t, req, adminUmputunToken)
 	require.NoError(t, err)
 
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	assert.Equal(t, "application/gzip", resp.Header.Get("Content-Type"))
 
 	ungzReader, err := gzip.NewReader(resp.Body)
@@ -697,17 +708,17 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
 
 	claims := token.Claims{
 		SessionOnly: true,
-		StandardClaims: jwt.StandardClaims{
+		RegisteredClaims: jwt.RegisteredClaims{
-			Audience:  "remark42",
+			Audience:  jwt.ClaimStrings{"remark42"},
-			Id:        "1234567",
+			ID:        "1234567",
 			Issuer:    "remark42",
-			NotBefore: time.Now().Add(-1 * time.Minute).Unix(),
+			NotBefore: jwt.NewNumericDate(time.Now().Add(-1 * time.Minute)),
-			ExpiresAt: time.Now().Add(30 * time.Minute).Unix(),
+			ExpiresAt: jwt.NewNumericDate(time.Now().Add(30 * time.Minute)),
 		},
 		User: &token.User{
 			ID:      "user1",
 			Picture: "pic.image",
-			Attributes: map[string]interface{}{
+			Attributes: map[string]any{
 				"delete_me": true,
 			},
 		},
@@ -720,6 +731,7 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
 	assert.NoError(t, err)
 
 	client := http.Client{}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, tkn), http.NoBody)
 	assert.NoError(t, err)
 
@@ -727,7 +739,7 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
 	resp, err := client.Do(req)
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 
 	_, err = srv.DataService.User("remark42", "user1", 0, 0, store.User{})
 	assert.EqualError(t, err, "no comments for user user1 in store")
@@ -735,7 +747,6 @@ func TestAdmin_DeleteMeRequest(t *testing.T) {
 	email, err = srv.DataService.GetUserEmail("remark42", "user1")
 	assert.NoError(t, err)
 	assert.Empty(t, email, "user1 email was deleted")
-
 }
 
 func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
@@ -743,9 +754,9 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
 	defer teardown()
 
 	c1 := store.Comment{Text: "test test #1", Locator: store.Locator{SiteID: "remark42",
-		URL: "https://radio-t.com/blah"}, User: store.User{Name: "user1 name", ID: "user1"}}
+		URL: "https://radio-t.com/blah"}, User: store.User{Name: "user1 name", ID: "provider1_user1"}}
 	c2 := store.Comment{Text: "test test #2", ParentID: "p1", Locator: store.Locator{SiteID: "remark42",
-		URL: "https://radio-t.com/blah"}, User: store.User{Name: "user2", ID: "user2"}}
+		URL: "https://radio-t.com/blah"}, User: store.User{Name: "user2", ID: "provider1_user2"}}
 
 	_, err := srv.DataService.Create(c1)
 	assert.NoError(t, err)
@@ -754,27 +765,28 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
 
 	// try with bad token
 	client := http.Client{}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, "bad token"), http.NoBody)
 	assert.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
 	resp, err := client.Do(req)
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
-	assert.Equal(t, 400, resp.StatusCode)
+	assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
 
 	// try with bad auth
 	claims := token.Claims{
 		SessionOnly: true,
-		StandardClaims: jwt.StandardClaims{
+		RegisteredClaims: jwt.RegisteredClaims{
-			Audience:  "remark42",
+			Audience:  jwt.ClaimStrings{"remark42"},
-			Id:        "1234567",
+			ID:        "provider1_1234567",
 			Issuer:    "remark42",
-			NotBefore: time.Now().Add(-1 * time.Minute).Unix(),
+			NotBefore: jwt.NewNumericDate(time.Now().Add(-1 * time.Minute)),
-			ExpiresAt: time.Now().Add(30 * time.Minute).Unix(),
+			ExpiresAt: jwt.NewNumericDate(time.Now().Add(30 * time.Minute)),
 		},
 		User: &token.User{
-			ID: "user1",
+			ID: "provider1_user1",
-			Attributes: map[string]interface{}{
+			Attributes: map[string]any{
 				"delete_me": true,
 			},
 		},
@@ -788,12 +800,12 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
 	resp, err = client.Do(req)
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
-	assert.Equal(t, 403, resp.StatusCode)
+	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 
 	// try bad user
-	badClaims := claims
+	badClaimsUser := claims
-	badClaims.User.ID = "no-such-id"
+	badClaimsUser.User.ID = "no-such-id"
-	tkn, err = srv.Authenticator.TokenService().Token(badClaims)
+	tkn, err = srv.Authenticator.TokenService().Token(badClaimsUser)
 	assert.NoError(t, err)
 	req, err = http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, tkn), http.NoBody)
 	assert.NoError(t, err)
@@ -801,23 +813,42 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
 	resp, err = client.Do(req)
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
-	assert.Equal(t, 400, resp.StatusCode, resp.Status)
+	assert.Equal(t, http.StatusBadRequest, resp.StatusCode, resp.Status)
+	badClaimsUser.User.ID = "provider1_user1"
 
 	// try without deleteme flag
-	badClaims2 := claims
+	badClaimsWithoutDeleteMe := claims
-	badClaims2.User.SetBoolAttr("delete_me", false)
+	badClaimsWithoutDeleteMe.User.SetBoolAttr("delete_me", false)
-	tkn, err = srv.Authenticator.TokenService().Token(badClaims2)
+	tkn, err = srv.Authenticator.TokenService().Token(badClaimsWithoutDeleteMe)
 	assert.NoError(t, err)
 	req, err = http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, tkn), http.NoBody)
 	assert.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
 	resp, err = client.Do(req)
 	assert.NoError(t, err)
-	assert.Equal(t, 403, resp.StatusCode)
+	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 	b, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
-	assert.True(t, strings.Contains(string(b), "can't use provided token"))
+	assert.Contains(t, string(b), "can't use provided token")
+	badClaimsWithoutDeleteMe.User.SetBoolAttr("delete_me", true)
+
+	// try with wrong audience
+	badClaimsMultipleAudience := claims
+	badClaimsMultipleAudience.Audience = jwt.ClaimStrings{"remark42", "something else"}
+	tkn, err = srv.Authenticator.TokenService().Token(badClaimsMultipleAudience)
+	assert.NoError(t, err)
+	req, err = http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/admin/deleteme?token=%s", ts.URL, tkn), http.NoBody)
+	assert.NoError(t, err)
+	req.SetBasicAuth("admin", "password")
+	resp, err = client.Do(req)
+	assert.NoError(t, err)
+	assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
+	b, err = io.ReadAll(resp.Body)
+	assert.NoError(t, err)
+	assert.NoError(t, resp.Body.Close())
+	assert.Contains(t, string(b), "can't process token, claims.Audience expected to be a single element but it's not")
+	badClaimsMultipleAudience.Audience = jwt.ClaimStrings{"remark42"}
 }
 
 func TestAdmin_GetUserInfo(t *testing.T) {
@@ -836,7 +867,7 @@ func TestAdmin_GetUserInfo(t *testing.T) {
 
 	body, code := getWithAdminAuth(t, fmt.Sprintf("%s/api/v1/admin/user/user1?site=remark42&url=https://radio-t.com/blah",
 		ts.URL))
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	u := store.User{}
 	err = json.Unmarshal([]byte(body), &u)
 	assert.NoError(t, err)
@@ -844,8 +875,8 @@ func TestAdmin_GetUserInfo(t *testing.T) {
 		Admin: false, Blocked: false, Verified: false}, u)
 
 	_, code = get(t, fmt.Sprintf("%s/api/v1/admin/user/user1?site=remark42&url=https://radio-t.com/blah", ts.URL))
-	assert.Equal(t, 401, code, "no auth")
+	assert.Equal(t, http.StatusUnauthorized, code, "no auth")
 
 	_, code = getWithAdminAuth(t, fmt.Sprintf("%s/api/v1/admin/user/userX?site=remark42&url=https://radio-t.com/blah", ts.URL))
-	assert.Equal(t, 400, code, "no info about user")
+	assert.Equal(t, http.StatusBadRequest, code, "no info about user")
 }

backend/app/rest/api/migrator.go

@@ -1,21 +1,20 @@
 package api
 
 import (
+	"bytes"
 	"compress/gzip"
 	"context"
 	"fmt"
 	"io"
-	"io/ioutil"
 	"net/http"
 	"os"
+	"strconv"
 	"sync"
 	"time"
 
-	"github.com/go-chi/render"
+	cache "github.com/go-pkgz/lcw/v2"
-	cache "github.com/go-pkgz/lcw"
 	log "github.com/go-pkgz/lgr"
 	R "github.com/go-pkgz/rest"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/migrator"
 	"github.com/umputun/remark42/backend/app/rest"
@@ -44,11 +43,10 @@ type KeyStore interface {
 // POST /import?secret=key&site=site-id&provider=disqus|remark|wordpress
 // imports comments from post body.
 func (m *Migrator) importCtrl(w http.ResponseWriter, r *http.Request) {
-
 	siteID := r.URL.Query().Get("site")
 
 	if m.isBusy(siteID) {
-		rest.SendErrorJSON(w, r, http.StatusConflict, errors.New("already running"),
+		rest.SendErrorJSON(w, r, http.StatusConflict, fmt.Errorf("already running"),
 			"import rejected", rest.ErrActionRejected)
 		return
 	}
@@ -61,8 +59,7 @@ func (m *Migrator) importCtrl(w http.ResponseWriter, r *http.Request) {
 
 	go m.runImport(siteID, r.URL.Query().Get("provider"), tmpfile) // import runs in background and sets busy flag for site
 
-	render.Status(r, http.StatusAccepted)
+	_ = R.EncodeJSON(w, http.StatusAccepted, R.JSON{"status": "import request accepted"})
-	render.JSON(w, r, R.JSON{"status": "import request accepted"})
 }
 
 // POST /import/form?secret=key&site=site-id&provider=disqus|remark|wordpress
@@ -71,11 +68,12 @@ func (m *Migrator) importFormCtrl(w http.ResponseWriter, r *http.Request) {
 	siteID := r.URL.Query().Get("site")
 
 	if m.isBusy(siteID) {
-		rest.SendErrorJSON(w, r, http.StatusConflict, errors.New("already running"),
+		rest.SendErrorJSON(w, r, http.StatusConflict, fmt.Errorf("already running"),
 			"import rejected", rest.ErrActionRejected)
 		return
 	}
 
+	r.Body = http.MaxBytesReader(w, r.Body, 256*1024*1024) // hard cap on upload to prevent memory exhaustion
 	if err := r.ParseMultipartForm(20 * 1024 * 1024); err != nil { // 20M max memory, if bigger will make a file
 		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't parse multipart form", rest.ErrDecode)
 		return
@@ -96,8 +94,7 @@ func (m *Migrator) importFormCtrl(w http.ResponseWriter, r *http.Request) {
 
 	go m.runImport(siteID, r.URL.Query().Get("provider"), tmpfile) // import runs in background and sets busy flag for site
 
-	render.Status(r, http.StatusAccepted)
+	_ = R.EncodeJSON(w, http.StatusAccepted, R.JSON{"status": "import request accepted"})
-	render.JSON(w, r, R.JSON{"status": "import request accepted"})
 }
 
 // GET /wait?site=site-id
@@ -113,46 +110,49 @@ func (m *Migrator) waitCtrl(w http.ResponseWriter, r *http.Request) {
 
 	ctx, cancel := context.WithTimeout(context.Background(), timeOut)
 	defer cancel()
-	for {
+	for m.isBusy(siteID) {
-		if !m.isBusy(siteID) {
+
-			break
-		}
 		select {
 		case <-ctx.Done():
-			render.Status(r, http.StatusGatewayTimeout)
+			_ = R.EncodeJSON(w, http.StatusGatewayTimeout, R.JSON{"status": "timeout expired", "site_id": siteID})
-			render.JSON(w, r, R.JSON{"status": "timeout expired", "site_id": siteID})
 			return
 		case <-time.After(100 * time.Millisecond):
 		}
 	}
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, R.JSON{"status": "completed", "site_id": siteID})
-	render.JSON(w, r, R.JSON{"status": "completed", "site_id": siteID})
 }
 
 // GET /export?site=site-id&secret=12345&?mode=file|stream
 // exports all comments for siteID as gz file
 func (m *Migrator) exportCtrl(w http.ResponseWriter, r *http.Request) {
-
 	siteID := r.URL.Query().Get("site")
 
-	var writer io.Writer = w
 	if r.URL.Query().Get("mode") == "file" {
+		// buffer to memory to handle errors before committing to response
+		var buf bytes.Buffer
+		gzWriter := gzip.NewWriter(&buf)
+		if _, err := m.NativeExporter.Export(gzWriter, siteID); err != nil {
+			rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "export failed", rest.ErrInternal)
+			return
+		}
+		if err := gzWriter.Close(); err != nil {
+			rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "export failed", rest.ErrInternal)
+			return
+		}
+
 		exportFile := fmt.Sprintf("%s-%s.json.gz", siteID, time.Now().Format("20060102"))
 		w.Header().Set("Content-Type", "application/gzip")
 		w.Header().Set("Content-Disposition", "attachment;filename="+exportFile)
-		w.WriteHeader(http.StatusOK)
+		w.Header().Set("Content-Length", strconv.Itoa(buf.Len()))
-		gzWriter := gzip.NewWriter(w)
+		if _, err := io.Copy(w, &buf); err != nil {
-		defer func() {
+			log.Printf("[WARN] failed to write export response: %v", err)
-			if e := gzWriter.Close(); e != nil {
+		}
-				log.Printf("[WARN] can't close gzip writer, %s", e)
+		return
-			}
-		}()
-		writer = gzWriter
 	}
 
-	if _, err := m.NativeExporter.Export(writer, siteID); err != nil {
+	// stream mode - write directly to response
+	if _, err := m.NativeExporter.Export(w, siteID); err != nil {
 		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "export failed", rest.ErrInternal)
-		return
 	}
 }
 
@@ -167,7 +167,7 @@ func (m *Migrator) remapCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "remap failed, bad given rules", rest.ErrDecode)
 		return
 	}
-	defer r.Body.Close()
+	defer r.Body.Close() //nolint gosec // we don't care about response body
 
 	// start remap procedure with mapper
 	go func() {
@@ -175,13 +175,13 @@ func (m *Migrator) remapCtrl(w http.ResponseWriter, r *http.Request) {
 		defer m.setBusy(siteID, false)
 
 		// do export
-		fh, e := ioutil.TempFile("", "remark42_convert")
+		fh, e := os.CreateTemp("", "remark42_convert")
 		if e != nil {
 			log.Printf("[WARN] failed to make temp file %+v", e)
 			return
 		}
 		defer func() {
-			if e = os.Remove(fh.Name()); e != nil {
+			if e = os.Remove(fh.Name()); e != nil { //nolint:gosec // fh.Name() is from os.CreateTemp, server-controlled
 				log.Printf("[WARN] failed to remove temp file %+v", e)
 			}
 		}()
@@ -208,8 +208,7 @@ func (m *Migrator) remapCtrl(w http.ResponseWriter, r *http.Request) {
 		log.Printf("[DEBUG] convert request completed. site=%s, comments=%d", siteID, size)
 	}()
 
-	render.Status(r, http.StatusAccepted)
+	_ = R.EncodeJSON(w, http.StatusAccepted, R.JSON{"status": "convert request accepted"})
-	render.JSON(w, r, R.JSON{"status": "convert request accepted"})
 }
 
 // runImport reads from tmpfile and import for given siteID and provider
@@ -253,17 +252,17 @@ func (m *Migrator) runImport(siteID, provider, tmpfile string) {
 
 // saveTemp reads from reader and saves to temp file
 func (m *Migrator) saveTemp(r io.Reader) (string, error) {
-	tmpfile, err := ioutil.TempFile("", "remark42_import")
+	tmpfile, err := os.CreateTemp("", "remark42_import")
 	if err != nil {
-		return "", errors.Wrap(err, "can't make temp file")
+		return "", fmt.Errorf("can't make temp file: %w", err)
 	}
 
 	if _, err = io.Copy(tmpfile, r); err != nil {
-		return "", errors.Wrap(err, "can't copy to temp file")
+		return "", fmt.Errorf("can't copy to temp file: %w", err)
 	}
 
 	if err = tmpfile.Close(); err != nil {
-		return "", errors.Wrap(err, "can't close temp file")
+		return "", fmt.Errorf("can't close temp file: %w", err)
 	}
 
 	return tmpfile.Name(), nil

backend/app/rest/api/migrator_test.go

@@ -9,6 +9,7 @@ import (
 	"mime/multipart"
 	"net/http"
 	"net/http/httptest"
+	"os"
 	"strings"
 	"testing"
 	"time"
@@ -34,6 +35,7 @@ func TestMigrator_Import(t *testing.T) {
 "votes":{},"time":"2018-04-30T01:37:00.861387771-05:00"}`)
 
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -48,6 +50,22 @@ func TestMigrator_Import(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 
 	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://radio-t.com/blah1")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
 }
 
 func TestMigrator_ImportForm(t *testing.T) {
@@ -83,15 +101,32 @@ func TestMigrator_ImportForm(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 
 	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://radio-t.com/blah1")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
 }
 
 func TestMigrator_ImportFromWP(t *testing.T) {
 	ts, _, teardown := startupT(t)
 	defer teardown()
 
-	r := strings.NewReader(strings.Replace(xmlTestWP, "'", "`", -1))
+	r := strings.NewReader(strings.ReplaceAll(xmlTestWP, "'", "`"))
 
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=wordpress", r)
 	assert.NoError(t, err)
 	req.Header.Add("Content-Type", "application/xml; charset=utf-8")
@@ -106,6 +141,22 @@ func TestMigrator_ImportFromWP(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 
 	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://realmenweardress.es/2010/07/do-you-rp/")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 3, comments.Info.Count)
+	require.Equal(t, 3, len(comments.Comments))
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://realmenweardress.es/2010/07/do-you-rp/")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 3, comments.Info.Count)
+	require.Equal(t, 2, len(comments.Comments), "2 comments with 1 reply")
 }
 
 func TestMigrator_ImportFromCommento(t *testing.T) {
@@ -113,13 +164,14 @@ func TestMigrator_ImportFromCommento(t *testing.T) {
 	defer teardown()
 
 	r := strings.NewReader(`{"version":1,"comments":[{"commentHex":"7d77e39fcd813241d6281478cc8f21ab5f807d043c750bc1a936bc23b34fb854",
-"domain":"example.com","url":"https://example.com/blog/post/1","commenterHex":"a1ac58ed1146bd7fe3feff6a7276f73955c3bfd23cacee00e2e0a7a89b1a8c10",
+"domain":"example.com","url":"/blog/post/1","commenterHex":"a1ac58ed1146bd7fe3feff6a7276f73955c3bfd23cacee00e2e0a7a89b1a8c10",
 "markdown":"Example content","html":"","parentHex":"root","score":0,"state":"approved","creationDate":"2021-03-17T12:09:47.722181Z",
 "direction":0,"deleted":false}],"commenters":[{"commenterHex":"a1ac58ed1146bd7fe3feff6a7276f73955c3bfd23cacee00e2e0a7a89b1a8c10",
 "email":"somegreatmail@gmail.com","name":"User5276","link":"https://example.com/profile/257","photo":"https://secure.gravatar.com/avatar/8f279626d26175134b0d5c88648172f7",
 "provider":"sso:example.com","joinDate":"2021-03-19T19:27:25.954285Z","isModerator":false}]}`)
 
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=commento", r)
 	assert.NoError(t, err)
 	req.Header.Add("Content-Type", "application/json; charset=utf-8")
@@ -134,6 +186,63 @@ func TestMigrator_ImportFromCommento(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 
 	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://example.com/blog/post/1")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://example.com/blog/post/1")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
+}
+
+func TestMigrator_ImportFromCommentoJSON(t *testing.T) {
+	ts, _, teardown := startupT(t)
+	defer teardown()
+
+	r, err := os.Open("testdata/commento.json")
+	require.NoError(t, err)
+
+	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
+	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=commento", r)
+	assert.NoError(t, err)
+	req.Header.Add("Content-Type", "application/json; charset=utf-8")
+	req.SetBasicAuth("admin", "password")
+	resp, err := client.Do(req)
+	assert.NoError(t, err)
+	assert.Equal(t, http.StatusAccepted, resp.StatusCode)
+
+	b, err := io.ReadAll(resp.Body)
+	assert.NoError(t, err)
+	assert.Equal(t, "{\"status\":\"import request accepted\"}\n", string(b))
+	assert.NoError(t, resp.Body.Close())
+
+	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://example.com/example")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 7, comments.Info.Count)
+	require.Equal(t, 7, len(comments.Comments))
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://example.com/example")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 7, comments.Info.Count)
+	require.Equal(t, 5, len(comments.Comments), "five comments with two replies")
 }
 
 func TestMigrator_ImportRejected(t *testing.T) {
@@ -150,6 +259,7 @@ func TestMigrator_ImportRejected(t *testing.T) {
 "votes":{},"time":"2018-04-30T01:37:00.861387771-05:00"}`)
 
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native&secret=XYZ", r)
 	assert.NoError(t, err)
 	resp, err := client.Do(req)
@@ -166,12 +276,13 @@ func TestMigrator_ImportDouble(t *testing.T) {
 "picture":"/api/v1/avatar/remark.image","profile":"https://remark42.com","admin":true,
 "ip":"ae12fe3b5f129b5cc4cdd2b136b7b7947c4d2741"},"locator":{"site":"remark42","url":"https://radio-t.com/blah1"},"score":0,
 "votes":{},"time":"2018-04-30T01:37:00.849053725-05:00"}`
-	recs := []string{}
+	recs := make([]string, 0, 50)
-	for i := 0; i < 50; i++ {
+	for i := range 50 {
 		recs = append(recs, fmt.Sprintf(tmpl, i))
 	}
 	r := strings.NewReader(`{"version":1}` + strings.Join(recs, "\n")) // reader with 10k records
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -182,6 +293,7 @@ func TestMigrator_ImportDouble(t *testing.T) {
 	assert.Equal(t, http.StatusAccepted, resp.StatusCode)
 
 	client = &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err = http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -191,6 +303,20 @@ func TestMigrator_ImportDouble(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 	assert.Equal(t, http.StatusConflict, resp.StatusCode)
 	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 50, comments.Info.Count)
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 50, comments.Info.Count)
 }
 
 func TestMigrator_ImportWaitExpired(t *testing.T) {
@@ -203,11 +329,12 @@ func TestMigrator_ImportWaitExpired(t *testing.T) {
 "votes":{},"time":"2018-04-30T01:37:00.849053725-05:00"}`
 	nRecs := 50
 	recs := make([]string, 0, nRecs)
-	for i := 0; i < nRecs; i++ {
+	for i := range nRecs {
 		recs = append(recs, fmt.Sprintf(tmpl, i))
 	}
 	r := strings.NewReader(`{"version":1}` + strings.Join(recs, "\n")) // reader with `nRecs` records
 	client := &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -218,6 +345,7 @@ func TestMigrator_ImportWaitExpired(t *testing.T) {
 	assert.Equal(t, http.StatusAccepted, resp.StatusCode)
 
 	client = &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err = http.NewRequest("GET", ts.URL+"/api/v1/admin/wait?site=remark42&timeout=5ms", http.NoBody)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -228,6 +356,14 @@ func TestMigrator_ImportWaitExpired(t *testing.T) {
 	assert.Equal(t, http.StatusGatewayTimeout, resp.StatusCode)
 
 	waitForMigrationCompletion(t, ts)
+
+	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://example.com/example")
+	require.Equal(t, http.StatusOK, code)
+	comments := commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 0, comments.Info.Count)
+	require.Equal(t, 0, len(comments.Comments))
 }
 
 func TestMigrator_Export(t *testing.T) {
@@ -245,6 +381,7 @@ func TestMigrator_Export(t *testing.T) {
 
 	// import comments first
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/import?site=remark42&provider=native", r)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -254,13 +391,23 @@ func TestMigrator_Export(t *testing.T) {
 	require.Equal(t, http.StatusAccepted, resp.StatusCode)
 	waitForMigrationCompletion(t, ts)
 
+	// export wrong site, should result in error
+	req, err = http.NewRequest("GET", ts.URL+"/api/v1/admin/export?mode=file&site=test", http.NoBody)
+	require.NoError(t, err)
+	req.SetBasicAuth("admin", "password")
+	resp, err = client.Do(req)
+	require.NoError(t, err)
+	resp.Body.Close()
+	require.Equal(t, http.StatusInternalServerError, resp.StatusCode)
+	require.Equal(t, "application/json", resp.Header.Get("Content-Type"))
+
 	// check file mode
 	req, err = http.NewRequest("GET", ts.URL+"/api/v1/admin/export?mode=file&site=remark42", http.NoBody)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
 	resp, err = client.Do(req)
 	require.NoError(t, err)
-	require.Equal(t, 200, resp.StatusCode)
+	require.Equal(t, http.StatusOK, resp.StatusCode)
 	require.Equal(t, "application/gzip", resp.Header.Get("Content-Type"))
 
 	ungzReader, err := gzip.NewReader(resp.Body)
@@ -278,7 +425,7 @@ func TestMigrator_Export(t *testing.T) {
 	req.SetBasicAuth("admin", "password")
 	resp, err = client.Do(req)
 	require.NoError(t, err)
-	require.Equal(t, 200, resp.StatusCode)
+	require.Equal(t, http.StatusOK, resp.StatusCode)
 	require.Equal(t, "text/plain; charset=utf-8", resp.Header.Get("Content-Type"))
 
 	body, err := io.ReadAll(resp.Body)
@@ -325,19 +472,21 @@ func TestMigrator_Remap(t *testing.T) {
 
 	// check that comments created as expected
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo/")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	comments := commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	require.NoError(t, err)
 	require.Equal(t, 2, comments.Info.Count)
+	require.Equal(t, 2, len(comments.Comments))
 	require.False(t, comments.Info.ReadOnly)
 
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo-another/")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	require.NoError(t, err)
 	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
 	require.True(t, comments.Info.ReadOnly)
 
 	// we want remap urls to another domain - www.remark42.com
@@ -350,35 +499,57 @@ func TestMigrator_Remap(t *testing.T) {
 
 	// after remap finished we should find comments from new urls
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://www.remark42.com/demo/")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	require.NoError(t, err)
 	require.Equal(t, 2, comments.Info.Count)
+	require.Equal(t, 2, len(comments.Comments))
+	require.False(t, comments.Info.ReadOnly)
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://www.remark42.com/demo/")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 2, comments.Info.Count)
+	require.Equal(t, 2, len(comments.Comments))
 	require.False(t, comments.Info.ReadOnly)
 
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://www.remark42.com/demo-another/")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	require.NoError(t, err)
 	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
+	require.True(t, comments.Info.ReadOnly)
+
+	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&format=tree&url=https://www.remark42.com/demo-another/")
+	require.Equal(t, http.StatusOK, code)
+	comments = commentsWithInfo{}
+	err = json.Unmarshal([]byte(res), &comments)
+	require.NoError(t, err)
+	require.Equal(t, 1, comments.Info.Count)
+	require.Equal(t, 1, len(comments.Comments))
 	require.True(t, comments.Info.ReadOnly)
 
 	// should find nothing from previous url
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo/")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	require.NoError(t, err)
 	require.Equal(t, 0, comments.Info.Count)
+	require.Equal(t, 0, len(comments.Comments))
 
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://remark42.com/demo-another/")
-	require.Equal(t, 200, code)
+	require.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	require.NoError(t, err)
 	require.Equal(t, 0, comments.Info.Count)
+	require.Equal(t, 0, len(comments.Comments))
 }
 
 func TestMigrator_RemapReject(t *testing.T) {
@@ -387,6 +558,7 @@ func TestMigrator_RemapReject(t *testing.T) {
 
 	// without admin credentials
 	client := &http.Client{Timeout: 1 * time.Second}
+	defer client.CloseIdleConnections()
 	rules := strings.NewReader(`https://remark42.com/* https://www.remark42.com/*`)
 	req, err := http.NewRequest("POST", ts.URL+"/api/v1/admin/remap?site=remark42", rules)
 	require.NoError(t, err)
@@ -398,12 +570,13 @@ func TestMigrator_RemapReject(t *testing.T) {
 
 func waitForMigrationCompletion(t *testing.T, ts *httptest.Server) {
 	client := &http.Client{Timeout: 10 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("GET", ts.URL+"/api/v1/admin/wait?site=remark42", http.NoBody)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
 	resp, err := client.Do(req)
 	require.NoError(t, err)
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	b, err := io.ReadAll(resp.Body)
 	require.NoError(t, err)
 	defer resp.Body.Close()

backend/app/rest/api/rest.go

@@ -3,26 +3,28 @@ package api
 import (
 	"bytes"
 	"context"
+	"embed"
 	"encoding/json"
 	"fmt"
+	"io/fs"
 	"net/http"
+	"net/mail"
+	"os"
+	"regexp"
 	"strings"
 	"sync"
 	"time"
 
-	"github.com/didip/tollbooth/v6"
+	"github.com/didip/tollbooth/v8"
-	"github.com/didip/tollbooth_chi"
+	"github.com/didip/tollbooth/v8/limiter"
 	"github.com/go-chi/chi/v5"
 	"github.com/go-chi/chi/v5/middleware"
 	"github.com/go-chi/cors"
-	"github.com/go-chi/render"
+	"github.com/go-pkgz/auth/v2"
-	"github.com/go-pkgz/auth"
+	"github.com/go-pkgz/lcw/v2"
-	"github.com/go-pkgz/lcw"
 	log "github.com/go-pkgz/lgr"
 	R "github.com/go-pkgz/rest"
 	"github.com/go-pkgz/rest/logger"
-	"github.com/pkg/errors"
-	"github.com/rakyll/statik/fs"
 
 	"github.com/umputun/remark42/backend/app/notify"
 	"github.com/umputun/remark42/backend/app/rest"
@@ -30,7 +32,6 @@ import (
 	"github.com/umputun/remark42/backend/app/store"
 	"github.com/umputun/remark42/backend/app/store/image"
 	"github.com/umputun/remark42/backend/app/store/service"
-	"github.com/umputun/remark42/backend/app/templates"
 )
 
 // Rest is a rest access server
@@ -49,6 +50,7 @@ type Rest struct {
 
 	AnonVote        bool
 	WebRoot         string
+	WebFS           embed.FS
 	RemarkURL       string
 	ReadOnlyAge     int
 	SharedSecret    string
@@ -56,25 +58,29 @@ type Rest struct {
 		Low      int
 		Critical int
 	}
-	UpdateLimiter       float64
+	UpdateLimiter              float64
-	EmailNotifications  bool
+	EmailNotifications         bool
-	TelegramBotUsername string
+	TelegramNotifications      bool
-	EmojiEnabled        bool
+	EmojiEnabled               bool
-	SimpleView          bool
+	SimpleView                 bool
-	ProxyCORS           bool
+	ProxyCORS                  bool
-	SendJWTHeader       bool
+	SendJWTHeader              bool
-	AllowedAncestors    []string // sets Content-Security-Policy "frame-ancestors ..."
+	AllowedAncestors           []string // sets Content-Security-Policy "frame-ancestors ..."
-	SubscribersOnly     bool
+	SubscribersOnly            bool
+	DisableSignature           bool // prevent signature from being added to headers
+	DisableFancyTextFormatting bool // disables SmartyPants in the comment text rendering of the posted comments
+	ExternalImageProxy         bool
 
 	SSLConfig   SSLConfig
 	httpsServer *http.Server
 	httpServer  *http.Server
 	lock        sync.Mutex
 
-	pubRest   public
+	pubRest          public
-	privRest  private
+	privRest         private
-	adminRest admin
+	adminRest        admin
-	rssRest   rss
+	rssRest          rss
+	openRouteLimiter float64
 }
 
 // LoadingCache defines interface for caching
@@ -85,17 +91,21 @@ type LoadingCache interface {
 }
 
 const hardBodyLimit = 1024 * 64 // limit size of body
-
+const openRouteLimiter = 10     // limit for open routes
 const lastCommentsScope = "last"
 
 type commentsWithInfo struct {
 	Comments []store.Comment `json:"comments"`
-	Info     store.PostInfo  `json:"info,omitempty"`
+	Info     store.PostInfo  `json:"info"`
+}
+
+type treeWithInfo struct {
+	*service.Tree
+	Info store.PostInfo `json:"info"`
 }
 
 // Run the lister and request's router, activate rest server
 func (s *Rest) Run(address string, port int) {
-
 	if address == "*" {
 		address = ""
 	}
@@ -189,9 +199,17 @@ func (s *Rest) makeHTTPServer(address string, port int, router http.Handler) *ht
 }
 
 func (s *Rest) routes() chi.Router {
+	if s.openRouteLimiter == 0 {
+		// set the default open route limiter. Just a safety measure as it should be set by Run method anyway
+		s.openRouteLimiter = openRouteLimiter
+	}
 	router := chi.NewRouter()
 	router.Use(middleware.Throttle(1000), middleware.RealIP, R.Recoverer(log.Default()))
-	router.Use(R.AppInfo("remark42", "umputun", s.Version), R.Ping)
+	router.Use(securityHeadersMiddleware(s.ExternalImageProxy, s.AllowedAncestors))
+	if !s.DisableSignature {
+		router.Use(R.AppInfo("remark42", "umputun", s.Version))
+	}
+	router.Use(R.Ping)
 
 	s.pubRest, s.privRest, s.adminRest, s.rssRest = s.controllerGroups() // assign controllers for groups
 
@@ -209,11 +227,6 @@ func (s *Rest) routes() chi.Router {
 		router.Use(corsMiddleware.Handler)
 	}
 
-	if len(s.AllowedAncestors) > 0 {
-		log.Printf("[INFO] allowed from %+v only", s.AllowedAncestors)
-		router.Use(frameAncestors(s.AllowedAncestors))
-	}
-
 	ipFn := func(ip string) string { return store.HashValue(ip, s.SharedSecret)[:12] } // logger uses it for anonymization
 	logInfoWithBody := logger.New(logger.Log(log.Default()), logger.WithBody, logger.IPfn(ipFn), logger.Prefix("[INFO]")).Handler
 
@@ -221,13 +234,14 @@ func (s *Rest) routes() chi.Router {
 
 	router.Group(func(r chi.Router) {
 		r.Use(middleware.Timeout(5 * time.Second))
-		r.Use(logInfoWithBody, tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)), middleware.NoCache)
+		r.Use(logInfoWithBody, rateLimiter(2), middleware.NoCache)
+		r.Use(validEmailAuth()) // reject suspicious email logins
 		r.Mount("/auth", authHandler)
 	})
 
 	router.Group(func(r chi.Router) {
 		r.Use(middleware.Timeout(5 * time.Second))
-		r.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(100, nil)))
+		r.Use(rateLimiter(100))
 		r.Mount("/avatar", avatarHandler)
 	})
 
@@ -235,17 +249,17 @@ func (s *Rest) routes() chi.Router {
 
 	// api routes
 	router.Route("/api/v1", func(rapi chi.Router) {
-
+		rapi.Use(apiCSPMiddleware)
 		rapi.Group(func(rava chi.Router) {
 			rava.Use(middleware.Timeout(5 * time.Second))
-			rava.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(100, nil)))
+			rava.Use(rateLimiter(100))
 			rava.Mount("/avatar", avatarHandler)
 		})
 
 		// open routes
 		rapi.Group(func(ropen chi.Router) {
 			ropen.Use(middleware.Timeout(30 * time.Second))
-			ropen.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
+			ropen.Use(rateLimiter(s.openRouteLimiter))
 			ropen.Use(authMiddleware.Trace, middleware.NoCache, logInfoWithBody)
 			ropen.Get("/config", s.configCtrl)
 			ropen.Get("/find", s.pubRest.findCommentsCtrl)
@@ -256,28 +270,33 @@ func (s *Rest) routes() chi.Router {
 			ropen.Post("/counts", s.pubRest.countMultiCtrl)
 			ropen.Get("/list", s.pubRest.listCtrl)
 			ropen.Get("/info", s.pubRest.infoCtrl)
-			ropen.Get("/img", s.ImageProxy.Handler)
 
 			ropen.Route("/rss", func(rrss chi.Router) {
 				rrss.Get("/post", s.rssRest.postCommentsCtrl)
 				rrss.Get("/site", s.rssRest.siteCommentsCtrl)
 				rrss.Get("/reply", s.rssRest.repliesCtrl)
 			})
-
 		})
 
-		// open routes, cached
+		// open routes, cached. /img lives here (not in the NoCache group above) because
+		// middleware.NoCache strips If-None-Match from incoming requests, which would
+		// defeat the proxy handler's 304 short-circuit. The handler sets a 30-day
+		// max-age on validated success responses (with a versioned etag for cache
+		// invalidation on revalidation); error responses get Cache-Control: no-store
+		// so transient failures aren't pinned in the cache.
 		rapi.Group(func(ropen chi.Router) {
 			ropen.Use(middleware.Timeout(30 * time.Second))
-			ropen.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
+			ropen.Use(rateLimiter(10))
 			ropen.Use(authMiddleware.Trace, logInfoWithBody)
+			ropen.Get("/img", s.ImageProxy.Handler)
 			ropen.Get("/picture/{user}/{id}", s.pubRest.loadPictureCtrl)
+			ropen.Get("/qr/telegram", s.pubRest.telegramQrCtrl)
 		})
 
 		// protected routes, require auth
 		rapi.Group(func(rauth chi.Router) {
 			rauth.Use(middleware.Timeout(30 * time.Second))
-			rauth.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
+			rauth.Use(rateLimiter(10))
 			rauth.Use(authMiddleware.Auth, matchSiteID, middleware.NoCache, logInfoWithBody)
 			rauth.Get("/user", s.privRest.userInfoCtrl)
 			rauth.Get("/userdata", s.privRest.userAllDataCtrl)
@@ -286,7 +305,7 @@ func (s *Rest) routes() chi.Router {
 		// admin routes, require auth and admin users only
 		rapi.Route("/admin", func(radmin chi.Router) {
 			radmin.Use(middleware.Timeout(30 * time.Second))
-			radmin.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
+			radmin.Use(rateLimiter(10))
 			radmin.Use(authMiddleware.Auth, authMiddleware.AdminOnly, matchSiteID)
 			radmin.Use(middleware.NoCache, logInfoWithBody)
 
@@ -312,7 +331,7 @@ func (s *Rest) routes() chi.Router {
 		// protected routes, throttled to 10/s by default, controlled by external UpdateLimiter param
 		rapi.Group(func(rauth chi.Router) {
 			rauth.Use(middleware.Timeout(10 * time.Second))
-			rauth.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(s.updateLimiter(), nil)))
+			rauth.Use(rateLimiter(s.updateLimiter()))
 			rauth.Use(authMiddleware.Auth, matchSiteID, subscribersOnly(s.SubscribersOnly))
 			rauth.Use(middleware.NoCache, logInfoWithBody)
 
@@ -332,52 +351,48 @@ func (s *Rest) routes() chi.Router {
 		// protected routes, anonymous rejected
 		rapi.Group(func(rauth chi.Router) {
 			rauth.Use(middleware.Timeout(10 * time.Second))
-			rauth.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(s.updateLimiter(), nil)))
+			rauth.Use(rateLimiter(s.updateLimiter()))
 			rauth.Use(authMiddleware.Auth, rejectAnonUser, matchSiteID)
 			rauth.Use(logger.New(logger.Log(log.Default()), logger.Prefix("[DEBUG]"), logger.IPfn(ipFn)).Handler)
 			rauth.Post("/picture", s.privRest.savePictureCtrl)
 		})
-
 	})
 
 	// open routes on root level
 	router.Group(func(rroot chi.Router) {
 		rroot.Use(middleware.Timeout(10 * time.Second))
-		rroot.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(50, nil)))
+		rroot.Use(rateLimiter(50))
-		rroot.Get("/index.html", s.pubRest.getStartedCtrl)
 		rroot.Get("/robots.txt", s.pubRest.robotsCtrl)
 		rroot.Get("/email/unsubscribe.html", s.privRest.emailUnsubscribeCtrl)
 		rroot.Post("/email/unsubscribe.html", s.privRest.emailUnsubscribeCtrl)
 	})
 
-	// file server for static content from /web
+	// file server for static content from s.WebRoot on path /web
-	addFileServer(router, "/web", http.Dir(s.WebRoot), s.Version)
+	addFileServer(router, s.WebFS, s.WebRoot, s.Version)
 	return router
 }
 
 func (s *Rest) controllerGroups() (public, private, admin, rss) {
-
 	pubGrp := public{
 		dataService:      s.DataService,
 		cache:            s.Cache,
 		imageService:     s.ImageService,
 		commentFormatter: s.CommentFormatter,
 		readOnlyAge:      s.ReadOnlyAge,
-		webRoot:          s.WebRoot,
 	}
 
 	privGrp := private{
-		dataService:      s.DataService,
+		dataService:                s.DataService,
-		cache:            s.Cache,
+		cache:                      s.Cache,
-		imageService:     s.ImageService,
+		imageService:               s.ImageService,
-		commentFormatter: s.CommentFormatter,
+		commentFormatter:           s.CommentFormatter,
-		readOnlyAge:      s.ReadOnlyAge,
+		readOnlyAge:                s.ReadOnlyAge,
-		authenticator:    s.Authenticator,
+		authenticator:              s.Authenticator,
-		notifyService:    s.NotifyService,
+		notifyService:              s.NotifyService,
-		telegramService:  s.TelegramService,
+		telegramService:            s.TelegramService,
-		remarkURL:        s.RemarkURL,
+		remarkURL:                  s.RemarkURL,
-		anonVote:         s.AnonVote,
+		anonVote:                   s.AnonVote,
-		templates:        templates.NewFS(),
+		disableFancyTextFormatting: s.DisableFancyTextFormatting,
 	}
 
 	admGrp := admin{
@@ -413,44 +428,46 @@ func (s *Rest) configCtrl(w http.ResponseWriter, r *http.Request) {
 	emails, _ := s.DataService.AdminStore.Email(siteID)
 
 	cnf := struct {
-		Version             string   `json:"version"`
+		Version               string   `json:"version"`
-		EditDuration        int      `json:"edit_duration"`
+		EditDuration          int      `json:"edit_duration"`
-		AdminEdit           bool     `json:"admin_edit"`
+		AdminEdit             bool     `json:"admin_edit"`
-		MaxCommentSize      int      `json:"max_comment_size"`
+		MinCommentSize        int      `json:"min_comment_size"`
-		Admins              []string `json:"admins"`
+		MaxCommentSize        int      `json:"max_comment_size"`
-		AdminEmail          string   `json:"admin_email"`
+		Admins                []string `json:"admins"`
-		Auth                []string `json:"auth_providers"`
+		AdminEmail            string   `json:"admin_email"`
-		AnonVote            bool     `json:"anon_vote"`
+		Auth                  []string `json:"auth_providers"`
-		LowScore            int      `json:"low_score"`
+		AnonVote              bool     `json:"anon_vote"`
-		CriticalScore       int      `json:"critical_score"`
+		LowScore              int      `json:"low_score"`
-		PositiveScore       bool     `json:"positive_score"`
+		CriticalScore         int      `json:"critical_score"`
-		ReadOnlyAge         int      `json:"readonly_age"`
+		PositiveScore         bool     `json:"positive_score"`
-		MaxImageSize        int      `json:"max_image_size"`
+		ReadOnlyAge           int      `json:"readonly_age"`
-		EmailNotifications  bool     `json:"email_notifications"`
+		MaxImageSize          int      `json:"max_image_size"`
-		TelegramBotUsername string   `json:"telegram_bot_username"`
+		EmailNotifications    bool     `json:"email_notifications"`
-		EmojiEnabled        bool     `json:"emoji_enabled"`
+		TelegramNotifications bool     `json:"telegram_notifications"`
-		SimpleView          bool     `json:"simple_view"`
+		EmojiEnabled          bool     `json:"emoji_enabled"`
-		SendJWTHeader       bool     `json:"send_jwt_header"`
+		SimpleView            bool     `json:"simple_view"`
-		SubscribersOnly     bool     `json:"subscribers_only"`
+		SendJWTHeader         bool     `json:"send_jwt_header"`
+		SubscribersOnly       bool     `json:"subscribers_only"`
 	}{
-		Version:             s.Version,
+		Version:               s.Version,
-		EditDuration:        int(s.DataService.EditDuration.Seconds()),
+		EditDuration:          int(s.DataService.EditDuration.Seconds()),
-		AdminEdit:           s.DataService.AdminEdits,
+		AdminEdit:             s.DataService.AdminEdits,
-		MaxCommentSize:      s.DataService.MaxCommentSize,
+		MinCommentSize:        s.DataService.MinCommentSize,
-		Admins:              admins,
+		MaxCommentSize:        s.DataService.MaxCommentSize,
-		AdminEmail:          emails,
+		Admins:                admins,
-		LowScore:            s.ScoreThresholds.Low,
+		AdminEmail:            emails,
-		CriticalScore:       s.ScoreThresholds.Critical,
+		LowScore:              s.ScoreThresholds.Low,
-		PositiveScore:       s.DataService.PositiveScore,
+		CriticalScore:         s.ScoreThresholds.Critical,
-		ReadOnlyAge:         s.ReadOnlyAge,
+		PositiveScore:         s.DataService.PositiveScore,
-		MaxImageSize:        s.ImageService.MaxSize,
+		ReadOnlyAge:           s.ReadOnlyAge,
-		EmailNotifications:  s.EmailNotifications,
+		MaxImageSize:          s.ImageService.MaxSize,
-		TelegramBotUsername: s.TelegramBotUsername,
+		EmailNotifications:    s.EmailNotifications,
-		EmojiEnabled:        s.EmojiEnabled,
+		TelegramNotifications: s.TelegramNotifications,
-		AnonVote:            s.AnonVote,
+		EmojiEnabled:          s.EmojiEnabled,
-		SimpleView:          s.SimpleView,
+		AnonVote:              s.AnonVote,
-		SendJWTHeader:       s.SendJWTHeader,
+		SimpleView:            s.SimpleView,
-		SubscribersOnly:     s.SubscribersOnly,
+		SendJWTHeader:         s.SendJWTHeader,
+		SubscribersOnly:       s.SubscribersOnly,
 	}
 
 	cnf.Auth = []string{}
@@ -461,39 +478,31 @@ func (s *Rest) configCtrl(w http.ResponseWriter, r *http.Request) {
 	if cnf.Admins == nil { // prevent json serialization to nil
 		cnf.Admins = []string{}
 	}
-	render.Status(r, http.StatusOK)
+	R.RenderJSON(w, cnf)
-	render.JSON(w, r, cnf)
 }
 
-// serves static files from /web or embedded by statik
+// serves static files from the webRoot directory or files embedded into the compiled binary if that directory is absent
-func addFileServer(r chi.Router, path string, root http.FileSystem, version string) {
+func addFileServer(r chi.Router, embedFS embed.FS, webRoot, version string) {
-
 	var webFS http.Handler
 
-	statikFS, err := fs.New()
+	if _, err := os.Stat(webRoot); err == nil {
-	if err != nil {
+		log.Printf("[INFO] run file server from %s from the disk", webRoot)
-		log.Printf("[DEBUG] no embedded assets loaded, %s", err)
+		webFS = http.FileServer(http.Dir(webRoot))
-		log.Printf("[INFO] run file server for %s, path %s", root, path)
-		webFS = http.FileServer(root)
 	} else {
-		log.Printf("[INFO] run file server for %s, embedded", root)
+		log.Printf("[INFO] run file server, embedded")
-		webFS = http.FileServer(statikFS)
+		var contentFS, _ = fs.Sub(embedFS, "web")
+		webFS = http.FileServer(http.FS(contentFS))
 	}
 
-	origPath := path
+	webFS = http.StripPrefix("/web", webFS)
-	webFS = http.StripPrefix(path, webFS)
+	r.Get("/web", http.RedirectHandler("/web/", http.StatusMovedPermanently).ServeHTTP)
-	if path != "/" && path[len(path)-1] != '/' {
-		r.Get(path, http.RedirectHandler(path+"/", http.StatusMovedPermanently).ServeHTTP)
-		path += "/"
-	}
-	path += "*"
 
-	r.With(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(20, nil)),
+	r.With(rateLimiter(20),
 		middleware.Timeout(10*time.Second),
 		cacheControl(time.Hour, version),
-	).Get(path, func(w http.ResponseWriter, r *http.Request) {
+	).Get("/web/*", func(w http.ResponseWriter, r *http.Request) {
 		// don't show dirs, just serve files
-		if strings.HasSuffix(r.URL.Path, "/") && len(r.URL.Path) > 1 && r.URL.Path != (origPath+"/") {
+		if strings.HasSuffix(r.URL.Path, "/") && len(r.URL.Path) > 1 && r.URL.Path != ("/web/") {
 			http.NotFound(w, r)
 			return
 		}
@@ -501,12 +510,12 @@ func addFileServer(r chi.Router, path string, root http.FileSystem, version stri
 	})
 }
 
-func encodeJSONWithHTML(v interface{}) ([]byte, error) {
+func encodeJSONWithHTML(v any) ([]byte, error) {
 	buf := &bytes.Buffer{}
 	enc := json.NewEncoder(buf)
 	enc.SetEscapeHTML(false)
 	if err := enc.Encode(v); err != nil {
-		return nil, errors.Wrap(err, "json encoding failed")
+		return nil, fmt.Errorf("json encoding failed: %w", err)
 	}
 	return buf.Bytes(), nil
 }
@@ -581,7 +590,9 @@ func matchSiteID(next http.Handler) http.Handler {
 		}
 
 		siteID := r.URL.Query().Get("site")
-		if siteID != "" && user.SiteID != siteID {
+		// require an explicit site so the user.SiteID check below cannot be bypassed
+		// by simply omitting the query parameter
+		if siteID == "" || user.SiteID != siteID {
 			http.Error(w, "Access denied", http.StatusForbidden)
 			return
 		}
@@ -592,7 +603,6 @@ func matchSiteID(next http.Handler) http.Handler {
 
 // cacheControl is a middleware setting cache expiration. Using url+version as etag
 func cacheControl(expiration time.Duration, version string) func(http.Handler) http.Handler {
-
 	etag := func(r *http.Request, version string) string {
 		s := version + ":" + r.URL.String()
 		return store.EncodeID(s)
@@ -616,19 +626,52 @@ func cacheControl(expiration time.Duration, version string) func(http.Handler) h
 	}
 }
 
-// frameAncestors is a middleware setting Content-Security-Policy "frame-ancestors host1 host2 ..."
+// apiCSPMiddleware overrides the global Content-Security-Policy on /api/v1 routes
-// prevents loading of comments widgets from any other origins. In case if the list of allowed empty, ignored.
+// with a strict, default-deny policy. The global CSP (securityHeadersMiddleware) keeps
-func frameAncestors(hosts []string) func(http.Handler) http.Handler {
+// 'self' 'unsafe-inline' for script-src/style-src because the widget HTML pages
-	return func(h http.Handler) http.Handler {
+// (/web/*.html) need inline bootstrap blocks. API responses serve JSON, XML/RSS, or
-		fn := func(w http.ResponseWriter, r *http.Request) {
+// images — none of those should ever execute scripts when rendered, so they get the
-			if len(hosts) == 0 {
+// strictest policy available as defense-in-depth against future trust-boundary bugs.
-				h.ServeHTTP(w, r)
+//
-				return
+// Image-serving handlers (/api/v1/img, /api/v1/picture/{user}/{id}) re-apply the same
+// rest.StrictImageCSP value at the handler level and additionally set Content-Disposition:
+// inline; filename="image" (framing the response as a file rather than a renderable
+// document) and X-Content-Type-Options: nosniff. The CSP re-apply is intentional belt-and-
+// braces: if a future route refactor bypasses this middleware, the image handlers still
+// emit the policy.
+func apiCSPMiddleware(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Security-Policy", rest.StrictImageCSP)
+		next.ServeHTTP(w, r)
+	})
+}
+
+// securityHeadersMiddleware sets security-related headers:
+//   - Content-Security-Policy: controls which resources the browser is allowed to load
+//   - Permissions-Policy: disables browser features (camera, mic, etc.) not needed by a comment widget
+//   - X-Content-Type-Options: prevents browsers from MIME-sniffing responses away from the declared type,
+//     stopping e.g. a user-uploaded image from being reinterpreted as executable HTML/JS
+//   - Referrer-Policy: controls how much URL information leaks in the Referer header on cross-origin
+//     requests; "strict-origin-when-cross-origin" sends only the origin (no path) to other domains
+//     and nothing at all on HTTPS→HTTP downgrades
+func securityHeadersMiddleware(imageProxyEnabled bool, allowedAncestors []string) func(http.Handler) http.Handler {
+	return func(next http.Handler) http.Handler {
+		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			imgSrc := "*"
+			if imageProxyEnabled {
+				imgSrc = "'self'"
 			}
-			w.Header().Set("Content-Security-Policy", "frame-ancestors "+strings.Join(hosts, " ")+";")
+			frameAncestors := "*"
-			h.ServeHTTP(w, r)
+			if len(allowedAncestors) > 0 {
-		}
+				frameAncestors = strings.Join(allowedAncestors, " ")
-		return http.HandlerFunc(fn)
+			}
+			// font-src is set to 'none' (no @font-face / no base64 fonts in the bundle).
+			w.Header().Set("Content-Security-Policy", fmt.Sprintf("default-src 'none'; base-uri 'none'; form-action 'none'; connect-src 'self'; frame-src 'self' mailto:; img-src %s; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; font-src 'none'; object-src 'none'; frame-ancestors %s;", imgSrc, frameAncestors))
+			w.Header().Set("Permissions-Policy", "accelerometer=(), autoplay=(), camera=(), cross-origin-isolated=(), display-capture=(), encrypted-media=(), fullscreen=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), screen-wake-lock=(), sync-xhr=(), usb=(), xr-spatial-tracking=(), clipboard-read=(), clipboard-write=(), gamepad=(), hid=(), idle-detection=(), interest-cohort=(), serial=(), unload=(), window-management=()")
+			w.Header().Set("X-Content-Type-Options", "nosniff")
+			w.Header().Set("Referrer-Policy", "strict-origin-when-cross-origin")
+			next.ServeHTTP(w, r)
+		})
 	}
 }
 
@@ -653,6 +696,52 @@ func subscribersOnly(enable bool) func(http.Handler) http.Handler {
 	}
 }
 
+// validEmailAuth is a middleware for auth endpoints for email method.
+// it rejects login request if user, site or email are suspicious
+func validEmailAuth() func(http.Handler) http.Handler {
+
+	reUser := regexp.MustCompile(`^[\p{L}\d\s_]{4,64}$`) // matches ui side validation, adding min/max limitation
+	reSite := regexp.MustCompile(`^[a-zA-Z\d\s_.-]{1,64}$`)
+
+	return func(h http.Handler) http.Handler {
+		fn := func(w http.ResponseWriter, r *http.Request) {
+
+			if r.URL.Path != "/auth/email/login" {
+				// not email login, skip the check
+				h.ServeHTTP(w, r)
+				return
+			}
+
+			if u := r.URL.Query().Get("user"); u != "" {
+				if !reUser.MatchString(u) {
+					log.Printf("[WARN] suspicious user rejected: %s", u)
+					http.Error(w, "Access denied", http.StatusForbidden)
+					return
+				}
+			}
+
+			if a := r.URL.Query().Get("address"); a != "" {
+				if _, err := mail.ParseAddress(a); err != nil {
+					log.Printf("[WARN] suspicious address rejected: %s", a)
+					http.Error(w, "Access denied", http.StatusForbidden)
+					return
+				}
+			}
+
+			if s := r.URL.Query().Get("site"); s != "" {
+				if !reSite.MatchString(s) {
+					log.Printf("[WARN] suspicious site rejected: %s", s)
+					http.Error(w, "Access denied", http.StatusForbidden)
+					return
+				}
+			}
+
+			h.ServeHTTP(w, r)
+		}
+		return http.HandlerFunc(fn)
+	}
+}
+
 func parseError(err error, defaultCode int) (code int) {
 	code = defaultCode
 
@@ -672,8 +761,20 @@ func parseError(err error, defaultCode int) (code int) {
 		code = rest.ErrCommentEditExpired
 	case strings.HasPrefix(err.Error(), "parent comment with reply can't be edited"):
 		code = rest.ErrCommentEditChanged
-
 	}
 
 	return code
 }
+
+// rateLimiter creates a rate limiting middleware with proper IP lookup configuration.
+// tollbooth v8 requires explicit IP lookup method to be set.
+// uses RemoteAddr which is set by chi's middleware.RealIP to the real client IP
+// from X-Forwarded-For, X-Real-IP, or True-Client-IP headers.
+func rateLimiter(maxReq float64) func(http.Handler) http.Handler {
+	lmt := tollbooth.NewLimiter(maxReq, nil)
+	lmt.SetIPLookup(limiter.IPLookup{
+		Name:           "RemoteAddr",
+		IndexFromRight: 0,
+	})
+	return tollbooth.HTTPMiddleware(lmt)
+}

backend/app/rest/api/rest_private.go

@@ -6,23 +6,23 @@ import (
 	"crypto/rand"
 	"crypto/sha1" //nolint:gosec //not used for security
 	"encoding/json"
+	"errors"
 	"fmt"
 	"html/template"
 	"io"
+	"net"
 	"net/http"
 	"strings"
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/go-chi/render"
+	"github.com/go-pkgz/auth/v2"
-	"github.com/go-pkgz/auth"
+	"github.com/go-pkgz/auth/v2/token"
-	"github.com/go-pkgz/auth/token"
+	cache "github.com/go-pkgz/lcw/v2"
-	cache "github.com/go-pkgz/lcw"
 	log "github.com/go-pkgz/lgr"
 	R "github.com/go-pkgz/rest"
-	"github.com/golang-jwt/jwt"
+	"github.com/golang-jwt/jwt/v5"
 	"github.com/hashicorp/go-multierror"
-	"github.com/pkg/errors"
 
 	"github.com/umputun/remark42/backend/app/notify"
 	"github.com/umputun/remark42/backend/app/rest"
@@ -34,17 +34,17 @@ import (
 )
 
 type private struct {
-	dataService      privStore
+	dataService                privStore
-	cache            LoadingCache
+	cache                      LoadingCache
-	readOnlyAge      int
+	readOnlyAge                int
-	commentFormatter *store.CommentFormatter
+	commentFormatter           *store.CommentFormatter
-	imageService     *image.Service
+	imageService               *image.Service
-	notifyService    *notify.Service
+	notifyService              *notify.Service
-	authenticator    *auth.Service
+	authenticator              *auth.Service
-	telegramService  telegramService
+	telegramService            telegramService
-	remarkURL        string
+	remarkURL                  string
-	anonVote         bool
+	anonVote                   bool
-	templates        templates.FileReader
+	disableFancyTextFormatting bool // disables SmartyPants in the comment text rendering of the posted comments
 }
 
 // telegramService is a subset of Telegram service used for setting up user telegram notifications
@@ -60,15 +60,15 @@ type privStore interface {
 	Vote(req service.VoteReq) (comment store.Comment, err error)
 	Get(locator store.Locator, commentID string, user store.User) (store.Comment, error)
 	User(siteID, userID string, limit, skip int, user store.User) ([]store.Comment, error)
-	GetUserEmail(siteID string, userID string) (string, error)
+	GetUserEmail(siteID, userID string) (string, error)
-	SetUserEmail(siteID string, userID string, value string) (string, error)
+	SetUserEmail(siteID, userID, value string) (string, error)
-	GetUserTelegram(siteID string, userID string) (string, error)
+	GetUserTelegram(siteID, userID string) (string, error)
-	SetUserTelegram(siteID string, userID string, value string) (string, error)
+	SetUserTelegram(siteID, userID, value string) (string, error)
-	DeleteUserDetail(siteID string, userID string, detail engine.UserDetail) error
+	DeleteUserDetail(siteID, userID string, detail engine.UserDetail) error
 	ValidateComment(c *store.Comment) error
-	IsVerified(siteID string, userID string) bool
+	IsVerified(siteID, userID string) bool
 	IsReadOnly(locator store.Locator) bool
-	IsBlocked(siteID string, userID string) bool
+	IsBlocked(siteID, userID string) bool
 	Info(locator store.Locator, readonlyAge int) (store.PostInfo, error)
 }
 
@@ -77,7 +77,7 @@ func (s *private) previewCommentCtrl(w http.ResponseWriter, r *http.Request) {
 	user := rest.MustGetUserInfo(r)
 
 	comment := store.Comment{}
-	if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &comment); err != nil {
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, hardBodyLimit)).Decode(&comment); err != nil {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't bind comment", rest.ErrDecode)
 		return
 	}
@@ -89,26 +89,24 @@ func (s *private) previewCommentCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	comment = s.commentFormatter.Format(comment)
+	comment = s.commentFormatter.Format(comment, s.disableFancyTextFormatting)
 	comment.Sanitize()
 
-	// check if images are valid
+	// check if images are valid, omit proxied images as they are lazy-loaded
-	for _, id := range s.imageService.ExtractPictures(comment.Text) {
+	for _, id := range s.imageService.ExtractNonProxiedPictures(comment.Text) {
 		err := s.imageService.ResetCleanupTimer(id)
 		if err != nil {
-			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't renew staged picture cleanup timer", rest.ErrImgNotFound)
+			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't load picture from the comment", rest.ErrImgNotFound)
 			return
 		}
 	}
-
+	rest.HTMLResponse(w, http.StatusOK, comment.Text)
-	render.HTML(w, r, comment.Text)
 }
 
 // POST /comment - adds comment, resets all immutable fields
 func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
-
 	comment := store.Comment{}
-	if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &comment); err != nil {
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, hardBodyLimit)).Decode(&comment); err != nil {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't bind comment", rest.ErrDecode)
 		return
 	}
@@ -123,17 +121,17 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
 
 	comment.PrepareUntrusted() // clean all fields user not supposed to set
 	comment.User = user
-	comment.User.IP = strings.Split(r.RemoteAddr, ":")[0]
+	comment.User.IP = extractIP(r.RemoteAddr)
 
 	comment.Orig = comment.Text // original comment text, prior to md render
 	if err := s.dataService.ValidateComment(&comment); err != nil {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "invalid comment", rest.ErrCommentValidation)
 		return
 	}
-	comment = s.commentFormatter.Format(comment)
+	comment = s.commentFormatter.Format(comment, s.disableFancyTextFormatting)
 
-	// check if images are valid
+	// check if images are valid, omit proxied images as they are lazy-loaded
-	for _, id := range s.imageService.ExtractPictures(comment.Text) {
+	for _, id := range s.imageService.ExtractNonProxiedPictures(comment.Text) {
 		_, err := s.imageService.Load(id)
 		if err != nil {
 			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't load picture from the comment", rest.ErrImgNotFound)
@@ -143,17 +141,17 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
 
 	// check if user blocked
 	if s.dataService.IsBlocked(comment.Locator.SiteID, comment.User.ID) {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "user blocked", rest.ErrUserBlocked)
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "user blocked", rest.ErrUserBlocked)
 		return
 	}
 
 	if s.isReadOnly(comment.Locator) {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "old post, read-only", rest.ErrReadOnly)
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "old post, read-only", rest.ErrReadOnly)
 		return
 	}
 
 	id, err := s.dataService.Create(comment)
-	if err == service.ErrRestrictedWordsFound {
+	if errors.Is(err, service.ErrRestrictedWordsFound) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "invalid comment", rest.ErrCommentRestrictWords)
 		return
 	}
@@ -175,23 +173,21 @@ func (s *private) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
 		s.notifyService.Submit(notify.Request{Comment: finalComment})
 	}
 
-	log.Printf("[DEBUG] created commend %+v", finalComment)
+	log.Printf("[DEBUG] created comment %+v", finalComment)
 
-	render.Status(r, http.StatusCreated)
+	_ = R.EncodeJSON(w, http.StatusCreated, &finalComment)
-	render.JSON(w, r, &finalComment)
 }
 
 // PUT /comment/{id}?site=siteID&url=post-url - update comment
 func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
-
 	edit := struct {
 		Text    string
 		Summary string
 		Delete  bool
 	}{}
 
-	if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, hardBodyLimit), &edit); err != nil {
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, hardBodyLimit)).Decode(&edit); err != nil {
-		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't bind comment", rest.ErrDecode)
+		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't read comment details from body", rest.ErrDecode)
 		return
 	}
 
@@ -209,13 +205,13 @@ func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
 	}
 
 	if currComment.User.ID != user.ID {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"),
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"),
 			"can not edit comments for other users", rest.ErrNoAccess)
 		return
 	}
 
 	editReq := service.EditRequest{
-		Text:    s.commentFormatter.FormatText(edit.Text),
+		Text:    s.commentFormatter.FormatText(edit.Text, s.disableFancyTextFormatting),
 		Orig:    edit.Text,
 		Summary: edit.Summary,
 		Delete:  edit.Delete,
@@ -223,7 +219,7 @@ func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
 	}
 
 	res, err := s.dataService.EditComment(locator, id, editReq)
-	if err == service.ErrRestrictedWordsFound {
+	if errors.Is(err, service.ErrRestrictedWordsFound) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "invalid comment", rest.ErrCommentValidation)
 		return
 	}
@@ -235,7 +231,7 @@ func (s *private) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
 	}
 
 	s.cache.Flush(cache.Flusher(locator.SiteID).Scopes(locator.SiteID, locator.URL, lastCommentsScope, user.ID))
-	render.JSON(w, r, res)
+	R.RenderJSON(w, res)
 }
 
 // GET /user?site=siteID - returns user info
@@ -248,12 +244,12 @@ func (s *private) userInfoCtrl(w http.ResponseWriter, r *http.Request) {
 		if err != nil {
 			log.Printf("[WARN] can't read email for %s, %v", user.ID, err)
 		}
-		if len(email) > 0 {
+		if email != "" {
 			user.EmailSubscription = true
 		}
 	}
 
-	render.JSON(w, r, user)
+	R.RenderJSON(w, user)
 }
 
 // PUT /vote/{id}?site=siteID&url=post-url&vote=1 - vote for/against comment
@@ -270,13 +266,13 @@ func (s *private) voteCtrl(w http.ResponseWriter, r *http.Request) {
 	vote := r.URL.Query().Get("vote") == "1"
 
 	if s.isReadOnly(locator) {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "old post, read-only", rest.ErrReadOnly)
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "old post, read-only", rest.ErrReadOnly)
 		return
 	}
 
 	// check if user blocked
 	if s.dataService.IsBlocked(locator.SiteID, user.ID) {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("rejected"), "user blocked", rest.ErrUserBlocked)
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("rejected"), "user blocked", rest.ErrUserBlocked)
 		return
 	}
 
@@ -284,7 +280,7 @@ func (s *private) voteCtrl(w http.ResponseWriter, r *http.Request) {
 		Locator:   locator,
 		CommentID: id,
 		UserID:    user.ID,
-		UserIP:    strings.Split(r.RemoteAddr, ":")[0],
+		UserIP:    extractIP(r.RemoteAddr),
 		Val:       vote,
 	}
 	comment, err := s.dataService.Vote(req)
@@ -294,7 +290,7 @@ func (s *private) voteCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	s.cache.Flush(cache.Flusher(locator.SiteID).Scopes(locator.URL, comment.User.ID))
-	render.JSON(w, r, R.JSON{"id": comment.ID, "score": comment.Score})
+	R.RenderJSON(w, R.JSON{"id": comment.ID, "score": comment.Score})
 }
 
 // getEmailCtrl gets email address for authenticated user.
@@ -307,36 +303,67 @@ func (s *private) getEmailCtrl(w http.ResponseWriter, r *http.Request) {
 		log.Printf("[WARN] can't read email for %s, %v", user.ID, err)
 	}
 
-	render.JSON(w, r, R.JSON{"user": user, "address": address})
+	R.RenderJSON(w, R.JSON{"user": user, "address": address})
 }
 
 // sendEmailConfirmationCtrl gets address and siteID from query, makes confirmation token and sends it to user.
-// GET /email/subscribe?site=siteID&address=someone@example.com
+// In case user is logged in with the same email, and auto_confirm is true, confirm it right away.
+// In case of quick confirmation, "updated" is set to true, otherwise - to false.
+// POST /email/subscribe with site and address in json body
+//
 //nolint:dupl // too hard to deduplicate that logic, as then it's tricky to use SendErrorJSON
 func (s *private) sendEmailConfirmationCtrl(w http.ResponseWriter, r *http.Request) {
 	user := rest.MustGetUserInfo(r)
-	address := r.URL.Query().Get("address")
+
-	siteID := r.URL.Query().Get("site")
+	subscribe := struct {
-	if address == "" {
+		Site        string
+		Address     string
+		autoConfirm bool
+	}{autoConfirm: true}
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, hardBodyLimit)).Decode(&subscribe); err != nil {
+		if err != io.EOF {
+			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't parse request body", rest.ErrDecode)
+			return
+		}
+		// old behavior fallback, reading from the query params. Auto confirm is false in this case.
+		subscribe.Address = r.URL.Query().Get("address")
+		subscribe.Site = r.URL.Query().Get("site")
+		subscribe.autoConfirm = false
+	}
+
+	if subscribe.Address == "" {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest,
-			errors.New("missing parameter"), "address parameter is required", rest.ErrInternal)
+			fmt.Errorf("missing parameter"), "address parameter is required", rest.ErrInternal)
 		return
 	}
-	existingAddress, err := s.dataService.GetUserEmail(siteID, user.ID)
+	existingAddress, getErr := s.dataService.GetUserEmail(subscribe.Site, user.ID)
-	if err != nil {
+	if getErr != nil {
-		log.Printf("[WARN] can't read email for %s, %v", user.ID, err)
+		log.Printf("[WARN] can't read email for %s, %v", user.ID, getErr)
 	}
-	if address == existingAddress {
+	if subscribe.Address == existingAddress {
 		rest.SendErrorJSON(w, r, http.StatusConflict,
-			errors.New("already verified"), "email address is already verified for this user", rest.ErrInternal)
+			fmt.Errorf("already verified"), "email address is already verified for this user", rest.ErrInternal)
 		return
 	}
+
+	// in case the user logged in with the same email as they try to subscribe with, confirm it right away
+	// this behavior is different from the previous one and is hidden behind the autoConfirm flag,
+	// which is true for the new API, and false for the old one
+	//
+	// nolint:gosec // this is not used for security purposes
+	if subscribe.autoConfirm &&
+		strings.HasPrefix(user.ID, "email_") &&
+		strings.TrimPrefix(user.ID, "email_") == token.HashID(sha1.New(), subscribe.Address) {
+		s.setEmail(w, r, user.ID, subscribe.Site, subscribe.Address)
+		return
+	}
+
 	claims := token.Claims{
-		Handshake: &token.Handshake{ID: user.ID + "::" + address},
+		Handshake: &token.Handshake{ID: user.ID + "::" + subscribe.Address},
-		StandardClaims: jwt.StandardClaims{
+		RegisteredClaims: jwt.RegisteredClaims{
-			Audience:  r.URL.Query().Get("site"),
+			Audience:  jwt.ClaimStrings{r.URL.Query().Get("site")},
-			ExpiresAt: time.Now().Add(30 * time.Minute).Unix(),
+			ExpiresAt: jwt.NewNumericDate(time.Now().Add(30 * time.Minute)),
-			NotBefore: time.Now().Add(-1 * time.Minute).Unix(),
+			NotBefore: jwt.NewNumericDate(time.Now().Add(-1 * time.Minute)),
 			Issuer:    "remark42",
 		},
 	}
@@ -349,14 +376,14 @@ func (s *private) sendEmailConfirmationCtrl(w http.ResponseWriter, r *http.Reque
 
 	s.notifyService.SubmitVerification(
 		notify.VerificationRequest{
-			SiteID: siteID,
+			SiteID: subscribe.Site,
 			User:   user.Name,
-			Email:  address,
+			Email:  subscribe.Address,
 			Token:  tkn,
 		},
 	)
 
-	render.JSON(w, r, R.JSON{"user": user, "address": address})
+	R.RenderJSON(w, R.JSON{"user": user, "address": subscribe.Address, "updated": false})
 }
 
 // telegramSubscribeCtrl generates and verifies telegram notification request
@@ -366,7 +393,7 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
 
 	if s.telegramService == nil {
 		rest.SendErrorJSON(w, r, http.StatusInternalServerError,
-			errors.New("not enabled"), "telegram notifications are not enabled", rest.ErrActionRejected)
+			fmt.Errorf("not enabled"), "telegram notifications are not enabled", rest.ErrActionRejected)
 		return
 	}
 
@@ -375,16 +402,16 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
 		// GET /telegram/subscribe?site=siteID (No token supplied)
 		siteID := r.URL.Query().Get("site")
 		if siteID == "" {
-			rest.SendErrorJSON(w, r, http.StatusBadRequest, errors.New("missing parameter"), "site parameter is required", rest.ErrInternal)
+			rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "site parameter is required", rest.ErrInternal)
 			return
 		}
 		// we don't care as much if we can't retrieve the current value of that field for the user, so ignore the error
 		if existingAddress, _ := s.dataService.GetUserTelegram(siteID, user.ID); existingAddress != "" {
 			rest.SendErrorJSON(w, r, http.StatusConflict,
-				errors.New("already subscribed"), "telegram subscription is already set for this user, delete if first to re-subscribe", rest.ErrActionRejected)
+				fmt.Errorf("already subscribed"), "telegram subscription is already set for this user, delete if first to re-subscribe", rest.ErrActionRejected)
 			return
 		}
-		// Generate and send token
+		// generate and send token
 		tkn, err := randToken()
 		if err != nil {
 			rest.SendErrorJSON(w, r, http.StatusForbidden, err, "failed to generate verification token", rest.ErrInternal)
@@ -394,7 +421,7 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
 
 		s.telegramService.AddToken(tkn, user.ID, siteID, expires)
 
-		render.JSON(w, r, R.JSON{"token": tkn, "bot": s.telegramService.GetBotUsername()})
+		R.RenderJSON(w, R.JSON{"token": tkn, "bot": s.telegramService.GetBotUsername()})
 
 		return
 	}
@@ -403,7 +430,7 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
 	var address, siteID string
 	address, siteID, err := s.telegramService.CheckToken(queryToken, user.ID)
 	if err != nil {
-		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't set telegram for user", rest.ErrInternal)
+		rest.SendErrorJSON(w, r, http.StatusNotFound, err, "request is not verified yet", rest.ErrInternal)
 		return
 	}
 
@@ -416,49 +443,64 @@ func (s *private) telegramSubscribeCtrl(w http.ResponseWriter, r *http.Request)
 		return
 	}
 
-	render.JSON(w, r, R.JSON{"updated": true, "address": val})
+	R.RenderJSON(w, R.JSON{"updated": true, "address": val})
 }
 
 // setConfirmedEmailCtrl uses provided token parameter (generated by sendEmailConfirmationCtrl) to set email and add it to user token
-// PUT /email/confirm?site=siteID&tkn=jwt
+// POST /email/confirm with site and token in json body
 func (s *private) setConfirmedEmailCtrl(w http.ResponseWriter, r *http.Request) {
-	tkn := r.URL.Query().Get("tkn")
+	user := rest.MustGetUserInfo(r)
-	if tkn == "" {
+
-		rest.SendErrorJSON(w, r, http.StatusBadRequest, errors.New("missing parameter"), "token parameter is required", rest.ErrInternal)
+	confirm := struct {
+		Site  string
+		Token string
+	}{}
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, hardBodyLimit)).Decode(&confirm); err != nil {
+		if err != io.EOF {
+			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't parse request body", rest.ErrDecode)
+			return
+		}
+		// old behavior fallback, reading from the query params
+		confirm.Token = r.URL.Query().Get("tkn")
+		confirm.Site = r.URL.Query().Get("site")
+	}
+
+	if confirm.Token == "" {
+		rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "token parameter is required", rest.ErrInternal)
 		return
 	}
-	user := rest.MustGetUserInfo(r)
+	confClaims, err := s.authenticator.TokenService().Parse(confirm.Token)
-	siteID := r.URL.Query().Get("site")
-
-	confClaims, err := s.authenticator.TokenService().Parse(tkn)
 	if err != nil {
 		rest.SendErrorJSON(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal)
 		return
 	}
 
 	if s.authenticator.TokenService().IsExpired(confClaims) {
-		rest.SendErrorJSON(w, r, http.StatusForbidden, errors.New("expired"), "failed to verify confirmation token", rest.ErrInternal)
+		rest.SendErrorJSON(w, r, http.StatusForbidden, fmt.Errorf("expired"), "failed to verify confirmation token", rest.ErrInternal)
 		return
 	}
 
-	// Handshake.ID is user.ID + "::" + address
+	// handshake.ID is user.ID + "::" + address
 	elems := strings.Split(confClaims.Handshake.ID, "::")
 	if len(elems) != 2 || elems[0] != user.ID {
-		rest.SendErrorJSON(w, r, http.StatusBadRequest, errors.New(confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal)
+		rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("%s", confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal)
 		return
 	}
 	address := elems[1]
+	s.setEmail(w, r, user.ID, confirm.Site, address)
+}
 
-	log.Printf("[DEBUG] set email for user %s", user.ID)
+func (s *private) setEmail(w http.ResponseWriter, r *http.Request, userID, siteID, address string) {
+	log.Printf("[DEBUG] set email for user %s", userID)
 
-	val, err := s.dataService.SetUserEmail(siteID, user.ID, address)
+	val, err := s.dataService.SetUserEmail(siteID, userID, address)
 	if err != nil {
 		code := parseError(err, rest.ErrInternal)
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't set email for user", code)
 		return
 	}
 
-	// update User.Email from the token
+	// update User.Email field
 	claims, _, err := s.authenticator.TokenService().Get(r)
 	if err != nil {
 		rest.SendErrorJSON(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal)
@@ -469,36 +511,33 @@ func (s *private) setConfirmedEmailCtrl(w http.ResponseWriter, r *http.Request)
 		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to set token", rest.ErrInternal)
 		return
 	}
-	render.JSON(w, r, R.JSON{"updated": true, "address": val})
+	R.RenderJSON(w, R.JSON{"updated": true, "address": val})
 }
 
 // POST/GET /email/unsubscribe.html?site=siteID&tkn=jwt - unsubscribe the user in token from email notifications
 func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
 	tkn := r.URL.Query().Get("tkn")
 	if tkn == "" {
-		rest.SendErrorHTML(w, r, http.StatusBadRequest,
+		rest.SendErrorHTML(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "token parameter is required", rest.ErrInternal)
-			errors.New("missing parameter"), "token parameter is required", rest.ErrInternal, s.templates)
 		return
 	}
 	siteID := r.URL.Query().Get("site")
 
 	confClaims, err := s.authenticator.TokenService().Parse(tkn)
 	if err != nil {
-		rest.SendErrorHTML(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal, s.templates)
+		rest.SendErrorHTML(w, r, http.StatusForbidden, err, "failed to verify confirmation token", rest.ErrInternal)
 		return
 	}
 
 	if s.authenticator.TokenService().IsExpired(confClaims) {
-		rest.SendErrorHTML(w, r, http.StatusForbidden,
+		rest.SendErrorHTML(w, r, http.StatusForbidden, fmt.Errorf("expired"), "failed to verify confirmation token", rest.ErrInternal)
-			errors.New("expired"), "failed to verify confirmation token", rest.ErrInternal, s.templates)
 		return
 	}
 
-	// Handshake.ID is user.ID + "::" + address
+	// handshake.ID is user.ID + "::" + address
 	elems := strings.Split(confClaims.Handshake.ID, "::")
 	if len(elems) != 2 {
-		rest.SendErrorHTML(w, r, http.StatusBadRequest,
+		rest.SendErrorHTML(w, r, http.StatusBadRequest, fmt.Errorf("%s", confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal)
-			errors.New(confClaims.Handshake.ID), "invalid handshake token", rest.ErrInternal, s.templates)
 		return
 	}
 	userID := elems[0]
@@ -511,14 +550,11 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
 		log.Printf("[WARN] can't read email for %s, %v", userID, err)
 	}
 	if existingAddress == "" {
-		rest.SendErrorHTML(w, r, http.StatusConflict,
+		rest.SendErrorHTML(w, r, http.StatusConflict, fmt.Errorf("user is not subscribed"), "user does not have active email subscription", rest.ErrInternal)
-			errors.New("user is not subscribed"), "user does not have active email subscription", rest.ErrInternal, s.templates)
 		return
 	}
 	if address != existingAddress {
-		rest.SendErrorHTML(w, r, http.StatusBadRequest,
+		rest.SendErrorHTML(w, r, http.StatusBadRequest, fmt.Errorf("wrong email unsubscription"), "email address in request does not match known for this user", rest.ErrInternal)
-			errors.New("wrong email unsubscription"), "email address in request does not match known for this user",
-			rest.ErrInternal, s.templates)
 		return
 	}
 
@@ -526,7 +562,7 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
 
 	if err = s.dataService.DeleteUserDetail(siteID, userID, engine.UserEmail); err != nil {
 		code := parseError(err, rest.ErrInternal)
-		rest.SendErrorHTML(w, r, http.StatusBadRequest, err, "can't delete email for user", code, s.templates)
+		rest.SendErrorHTML(w, r, http.StatusBadRequest, err, "can't delete email for user", code)
 		return
 	}
 	// clean User.Email from the token, if user has the token
@@ -537,29 +573,29 @@ func (s *private) emailUnsubscribeCtrl(w http.ResponseWriter, r *http.Request) {
 	if claims.User != nil && claims.User.Email != "" {
 		claims.User.Email = ""
 		if _, err = s.authenticator.TokenService().Set(w, claims); err != nil {
-			rest.SendErrorHTML(w, r, http.StatusInternalServerError, err, "failed to set token", rest.ErrInternal, s.templates)
+			rest.SendErrorHTML(w, r, http.StatusInternalServerError, err, "failed to set token", rest.ErrInternal)
 			return
 		}
 	}
 
 	// MustExecute behaves like template.Execute, but panics if an error occurs.
-	MustExecute := func(tmpl *template.Template, wr io.Writer, data interface{}) {
+	MustExecute := func(tmpl *template.Template, wr io.Writer, data any) {
 		if err := tmpl.Execute(wr, data); err != nil {
 			panic(err)
 		}
 	}
 	MustRead := func(path string) string {
-		file, err := s.templates.ReadFile(path)
+		file, err := templates.Read(path)
 		if err != nil {
 			panic(err)
 		}
 		return string(file)
 	}
-	tmplstr := MustRead("unsubscribe.html.tmpl")
+	tmplstr := MustRead("email_unsubscribe.html.tmpl")
 	tmpl := template.Must(template.New("unsubscribe").Parse(tmplstr))
 	msg := bytes.Buffer{}
 	MustExecute(tmpl, &msg, nil)
-	render.HTML(w, r, msg.String())
+	rest.HTMLResponse(w, http.StatusOK, msg.String())
 }
 
 // DELETE /email?site=siteID - removes user's email
@@ -586,7 +622,7 @@ func (s *private) deleteEmailCtrl(w http.ResponseWriter, r *http.Request) {
 			return
 		}
 	}
-	render.JSON(w, r, R.JSON{"deleted": true})
+	R.RenderJSON(w, R.JSON{"deleted": true})
 }
 
 // DELETE /telegram?site=siteID - removes user's telegram
@@ -600,7 +636,7 @@ func (s *private) deleteTelegramCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't delete telegram for user", code)
 		return
 	}
-	render.JSON(w, r, R.JSON{"deleted": true})
+	R.RenderJSON(w, R.JSON{"deleted": true})
 }
 
 // GET /userdata?site=siteID - exports all data about the user as a json with user info and list of all comments
@@ -634,7 +670,7 @@ func (s *private) userAllDataCtrl(w http.ResponseWriter, r *http.Request) {
 	merr = multierror.Append(merr, write([]byte(`, "comments":`))) // send comments prefix
 
 	// get comments in 100 in each paginated request
-	for i := 0; i < 100; i++ {
+	for i := range 100 {
 		comments, errUser := s.dataService.User(siteID, user.ID, 100, i*100, rest.GetUserOrEmpty(r))
 		if errUser != nil {
 			rest.SendErrorJSON(w, r, http.StatusInternalServerError, errUser, "can't get user comments", rest.ErrInternal)
@@ -657,7 +693,6 @@ func (s *private) userAllDataCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusInternalServerError, merr, "can't write user info", rest.ErrInternal)
 		return
 	}
-
 }
 
 // POST /deleteme?site_id=site - requesting delete of all user info
@@ -667,16 +702,16 @@ func (s *private) deleteMeCtrl(w http.ResponseWriter, r *http.Request) {
 	siteID := r.URL.Query().Get("site")
 
 	claims := token.Claims{
-		StandardClaims: jwt.StandardClaims{
+		RegisteredClaims: jwt.RegisteredClaims{
-			Audience:  siteID,
+			Audience:  jwt.ClaimStrings{siteID},
 			Issuer:    "remark42",
-			ExpiresAt: time.Now().AddDate(0, 3, 0).Unix(),
+			ExpiresAt: jwt.NewNumericDate(time.Now().AddDate(0, 3, 0)),
-			NotBefore: time.Now().Add(-1 * time.Minute).Unix(),
+			NotBefore: jwt.NewNumericDate(time.Now().Add(-1 * time.Minute)),
 		},
 		User: &token.User{
 			ID:   user.ID,
 			Name: user.Name,
-			Attributes: map[string]interface{}{
+			Attributes: map[string]any{
 				"delete_me": true, // prevents this token from being used for login
 			},
 		},
@@ -689,14 +724,18 @@ func (s *private) deleteMeCtrl(w http.ResponseWriter, r *http.Request) {
 	}
 
 	link := fmt.Sprintf("%s/web/deleteme.html?token=%s", s.remarkURL, tokenStr)
-	render.JSON(w, r, R.JSON{"site": siteID, "user_id": user.ID, "token": tokenStr, "link": link})
+	R.RenderJSON(w, R.JSON{"site": siteID, "user_id": user.ID, "token": tokenStr, "link": link})
 }
 
 // POST /image - save image with form request
 func (s *private) savePictureCtrl(w http.ResponseWriter, r *http.Request) {
 	user := rest.MustGetUserInfo(r)
 
-	if err := r.ParseMultipartForm(5 * 1024 * 1024); err != nil { // 5M max memory, if bigger will make a file
+	r.Body = http.MaxBytesReader(w, r.Body, 32*1024*1024) // hard cap on upload to prevent memory exhaustion
+	// gosec G120: r.Body is already bounded by MaxBytesReader on the line above (32 MB),
+	// so ParseMultipartForm cannot read more than that regardless of the in-memory threshold.
+	// The 5 MB argument is the soft threshold above which the form is spilled to disk.
+	if err := r.ParseMultipartForm(5 * 1024 * 1024); err != nil { //nolint:gosec // bounded by MaxBytesReader above
 		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't parse multipart form", rest.ErrDecode)
 		return
 	}
@@ -714,7 +753,7 @@ func (s *private) savePictureCtrl(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	render.JSON(w, r, R.JSON{"id": id})
+	R.RenderJSON(w, R.JSON{"id": id})
 }
 
 func (s *private) isReadOnly(locator store.Locator) bool {
@@ -730,11 +769,21 @@ func (s *private) isReadOnly(locator store.Locator) bool {
 func randToken() (string, error) {
 	b := make([]byte, 32)
 	if _, err := rand.Read(b); err != nil {
-		return "", errors.Wrap(err, "can't get random")
+		return "", fmt.Errorf("can't get random: %w", err)
 	}
 	s := sha1.New() //nolint:gosec // not used for security
 	if _, err := s.Write(b); err != nil {
-		return "", errors.Wrap(err, "can't write randoms to sha1")
+		return "", fmt.Errorf("can't write randoms to sha1: %w", err)
 	}
 	return fmt.Sprintf("%x", s.Sum(nil)), nil
 }
+
+// extractIP returns the IP portion of the remote address, handling both IPv4 and IPv6 formats.
+// supports "ip:port", "[ip]:port", and bare "ip" formats.
+func extractIP(remoteAddr string) string {
+	ip, _, err := net.SplitHostPort(remoteAddr)
+	if err != nil {
+		return remoteAddr // already a bare IP (no port)
+	}
+	return ip
+}

backend/app/rest/api/rest_public.go

@@ -4,20 +4,21 @@ import (
 	"bytes"
 	"crypto/sha1" // nolint
 	"encoding/base64"
+	"encoding/json"
+	"fmt"
 	"io"
 	"net/http"
-	"os"
-	"path"
 	"strconv"
 	"strings"
 	"time"
+	"unicode"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/go-chi/render"
+	cache "github.com/go-pkgz/lcw/v2"
-	cache "github.com/go-pkgz/lcw"
 	log "github.com/go-pkgz/lgr"
 	R "github.com/go-pkgz/rest"
-	"github.com/pkg/errors"
+	"github.com/google/uuid"
+	"github.com/skip2/go-qrcode"
 
 	"github.com/umputun/remark42/backend/app/rest"
 	"github.com/umputun/remark42/backend/app/store"
@@ -31,7 +32,6 @@ type public struct {
 	readOnlyAge      int
 	commentFormatter *store.CommentFormatter
 	imageService     *image.Service
-	webRoot          string
 }
 
 type pubStore interface {
@@ -42,7 +42,7 @@ type pubStore interface {
 	User(siteID, userID string, limit, skip int, user store.User) ([]store.Comment, error)
 	UserCount(siteID, userID string) (int, error)
 	Count(locator store.Locator) (int, error)
-	List(siteID string, limit int, skip int) ([]store.PostInfo, error)
+	List(siteID string, limit, skip int) ([]store.PostInfo, error)
 	Info(locator store.Locator, readonlyAge int) (store.PostInfo, error)
 
 	ValidateComment(c *store.Comment) error
@@ -50,8 +50,18 @@ type pubStore interface {
 	Counts(siteID string, postIDs []string) ([]store.PostInfo, error)
 }
 
-// GET /find?site=siteID&url=post-url&format=[tree|plain]&sort=[+/-time|+/-score|+/-controversy]&view=[user|all]&since=unix_ts_msec
+// GET /find?site=siteID&url=post-url&format=[tree|plain]&sort=[+/-time|+/-score|+/-controversy]&view=[user|all]&since=unix_ts_msec&limit=100&offset_id={id}
-// find comments for given post. Returns in tree or plain formats, sorted
+// find comments for given post. Returns in tree or plain formats, sorted.
+//
+// When `url` parameter is not set (e.g. request is for site-wide comments), does not return deleted comments.
+//
+// When `limit` is set, first {limit} comments are returned. When `offset_id` is set, comments are returned starting
+// after the comment with the given id.
+// format="tree" limits comments by top-level comments and all their replies,
+// and never returns parent comment with only part of replies.
+//
+// `count` in the response refers to total number of non-deleted comments,
+// `count_left` to amount of comments left to be returned _including deleted_.
 func (s *public) findCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 	locator := store.Locator{SiteID: r.URL.Query().Get("site"), URL: r.URL.Query().Get("url")}
 	sort := r.URL.Query().Get("sort")
@@ -70,7 +80,24 @@ func (s *public) findCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 		since = time.Time{} // since doesn't make sense for tree
 	}
 
-	log.Printf("[DEBUG] get comments for %+v, sort %s, format %s, since %v", locator, sort, format, since)
+	limitParam := r.URL.Query().Get("limit")
+	var limit int
+	if limitParam != "" {
+		if limit, err = strconv.Atoi(limitParam); err != nil {
+			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "bad limit value", rest.ErrCommentNotFound)
+			return
+		}
+	}
+
+	offsetID := r.URL.Query().Get("offset_id")
+	if offsetID != "" {
+		if _, err = uuid.Parse(offsetID); err != nil {
+			rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "bad offset_id value", rest.ErrCommentNotFound)
+			return
+		}
+	}
+
+	log.Printf("[DEBUG] get comments for %+v, sort %s, format %s, since %v, limit %d, offset %s", locator, sort, format, since, limit, offsetID)
 
 	key := cache.NewKey(locator.SiteID).ID(URLKeyWithUser(r)).Scopes(locator.SiteID, locator.URL)
 	data, err := s.cache.Get(key, func() ([]byte, error) {
@@ -79,22 +106,44 @@ func (s *public) findCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 			comments = []store.Comment{} // error should clear comments and continue for post info
 		}
 		comments = s.applyView(comments, view)
+
+		var commentsInfo store.PostInfo
+		if info, ee := s.dataService.Info(locator, s.readOnlyAge); ee == nil {
+			commentsInfo = info
+		}
+
+		if !since.IsZero() { // if since is set, number of comments can be different from total in the DB
+			commentsInfo.Count = 0
+			for _, c := range comments {
+				if !c.Deleted {
+					commentsInfo.Count++
+				}
+			}
+		}
+
+		// post might be readonly without any comments, Info call will fail then and ReadOnly flag should be checked separately
+		if !commentsInfo.ReadOnly && locator.URL != "" && s.dataService.IsReadOnly(locator) {
+			commentsInfo.ReadOnly = true
+		}
+
 		var b []byte
 		switch format {
 		case "tree":
-			tree := service.MakeTree(comments, sort, s.readOnlyAge)
+			withInfo := treeWithInfo{Tree: service.MakeTree(comments, sort, limit, offsetID), Info: commentsInfo}
-			if tree.Nodes == nil { // eliminate json nil serialization
+			withInfo.Info.CountLeft = withInfo.CountLeft()
-				tree.Nodes = []*service.Node{}
+			withInfo.Info.LastComment = withInfo.LastComment()
+			if withInfo.Nodes == nil { // eliminate json nil serialization
+				withInfo.Nodes = []*service.Node{}
 			}
-			if s.dataService.IsReadOnly(locator) {
+			b, e = encodeJSONWithHTML(withInfo)
-				tree.Info.ReadOnly = true
-			}
-			b, e = encodeJSONWithHTML(tree)
 		default:
-			withInfo := commentsWithInfo{Comments: comments}
+			if limit > 0 || offsetID != "" {
-			if info, ee := s.dataService.Info(locator, s.readOnlyAge); ee == nil {
+				comments, commentsInfo.CountLeft = limitComments(comments, limit, offsetID)
-				withInfo.Info = info
 			}
+			if limit > 0 && len(comments) > 0 {
+				commentsInfo.LastComment = comments[len(comments)-1].ID
+			}
+			withInfo := commentsWithInfo{Comments: comments, Info: commentsInfo}
 			b, e = encodeJSONWithHTML(withInfo)
 		}
 		return b, e
@@ -173,7 +222,6 @@ func (s *public) lastCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 
 // GET /id/{id}?site=siteID&url=post-url - gets a comment by id
 func (s *public) commentByIDCtrl(w http.ResponseWriter, r *http.Request) {
-
 	id := chi.URLParam(r, "id")
 	siteID := r.URL.Query().Get("site")
 	url := r.URL.Query().Get("url")
@@ -185,7 +233,6 @@ func (s *public) commentByIDCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get comment by id", rest.ErrCommentNotFound)
 		return
 	}
-	render.Status(r, http.StatusOK)
 
 	if err = R.RenderJSONWithHTML(w, r, comment); err != nil {
 		log.Printf("[WARN] can't render last comments for url=%s, id=%s", url, id)
@@ -194,7 +241,6 @@ func (s *public) commentByIDCtrl(w http.ResponseWriter, r *http.Request) {
 
 // GET /comments?site=siteID&user=id&limit=123&skip=10 - returns comments for given userID
 func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
-
 	userID := r.URL.Query().Get("user")
 	siteID := r.URL.Query().Get("site")
 
@@ -209,8 +255,8 @@ func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 	limit, skip := getNumWithDef("limit"), getNumWithDef("skip")
 
 	resp := struct {
-		Comments []store.Comment `json:"comments,omitempty"`
+		Comments []store.Comment `json:"comments"`
-		Count    int             `json:"count,omitempty"`
+		Count    int             `json:"count"`
 	}{}
 
 	log.Printf("[DEBUG] get comments for userID %s, %s", userID, siteID)
@@ -219,6 +265,10 @@ func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 	data, err := s.cache.Get(key, func() ([]byte, error) {
 		comments, e := s.dataService.User(siteID, userID, limit, skip, rest.GetUserOrEmpty(r))
 		if e != nil {
+			if strings.Contains(e.Error(), "no comments for user") { // store returns this error when no comments found
+				resp.Comments, resp.Count = []store.Comment{}, 0
+				return encodeJSONWithHTML(resp)
+			}
 			return nil, e
 		}
 		comments = filterComments(comments, func(c store.Comment) bool { return !c.Deleted })
@@ -234,7 +284,6 @@ func (s *public) findUserCommentsCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get comment by user id", rest.ErrCommentNotFound)
 		return
 	}
-
 	if err = R.RenderJSONFromBytes(w, r, data); err != nil {
 		log.Printf("[WARN] can't render found comments for user %s", userID)
 	}
@@ -248,7 +297,7 @@ func (s *public) countCtrl(w http.ResponseWriter, r *http.Request) {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get count", rest.ErrPostNotFound)
 		return
 	}
-	render.JSON(w, r, R.JSON{"count": count, "locator": locator})
+	R.RenderJSON(w, R.JSON{"count": count, "locator": locator})
 }
 
 // POST /counts?site=siteID - get number of comments for posts from post body
@@ -256,7 +305,7 @@ func (s *public) countMultiCtrl(w http.ResponseWriter, r *http.Request) {
 	const countBodyLimit int64 = 1024 * 128 // count request can be big for some site because it lists all urls
 	siteID := r.URL.Query().Get("site")
 	posts := []string{}
-	if err := render.DecodeJSON(http.MaxBytesReader(w, r.Body, countBodyLimit), &posts); err != nil {
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, countBodyLimit)).Decode(&posts); err != nil {
 		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get list of posts from request", rest.ErrSiteNotFound)
 		return
 	}
@@ -287,7 +336,6 @@ func (s *public) countMultiCtrl(w http.ResponseWriter, r *http.Request) {
 
 // GET /list?site=siteID&limit=50&skip=10 - list posts with comments
 func (s *public) listCtrl(w http.ResponseWriter, r *http.Request) {
-
 	siteID := r.URL.Query().Get("site")
 	limit, skip := 0, 0
 
@@ -317,26 +365,81 @@ func (s *public) listCtrl(w http.ResponseWriter, r *http.Request) {
 	}
 }
 
+// safePictureSegment reports whether seg is acceptable as a path segment in
+// the picture URL (no traversal markers, no path separators, no control
+// characters). Picture IDs are server-generated hashes plus a known
+// extension, so any value carrying these characters is hostile and must be
+// rejected before reaching the store. Rejecting controls (CR, LF, TAB, NUL,
+// etc.) also closes a log-injection vector since the rejected segment is
+// echoed into the access log.
+func safePictureSegment(seg string) bool {
+	if seg == "" || seg == "." {
+		return false
+	}
+	if strings.ContainsAny(seg, "/\\") {
+		return false
+	}
+	if strings.Contains(seg, "..") { // also covers seg == ".."
+		return false
+	}
+	for _, r := range seg {
+		if unicode.IsControl(r) {
+			return false
+		}
+	}
+	return true
+}
+
+// sendPictureError writes a no-store Cache-Control header and delegates to rest.SendErrorJSON.
+// Used by every rejection branch in loadPictureCtrl so error responses never inherit the
+// 7-day client cache of the success path.
+func sendPictureError(w http.ResponseWriter, r *http.Request, status int, err error, details string, code int) {
+	w.Header().Set("Cache-Control", "no-store")
+	rest.SendErrorJSON(w, r, status, err, details, code)
+}
+
 // GET /picture/{user}/{id} - get picture
 func (s *public) loadPictureCtrl(w http.ResponseWriter, r *http.Request) {
-	id := chi.URLParam(r, "user") + "/" + chi.URLParam(r, "id")
+	rest.SetImageDefenseHeaders(w)
-	img, err := s.imageService.Load(id)
+
-	if err != nil {
+	user, imgID := chi.URLParam(r, "user"), chi.URLParam(r, "id")
-		rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't get image "+id, rest.ErrAssetNotFound)
+	if user == "" || imgID == "" || !safePictureSegment(user) || !safePictureSegment(imgID) {
+		log.Printf("[WARN] rejected picture request with unsafe id segments user=%q id=%q", user, imgID)
+		sendPictureError(w, r, http.StatusBadRequest, fmt.Errorf("invalid picture id"), "invalid picture id", rest.ErrAssetNotFound)
 		return
 	}
-	// enforce client-side caching
+	id := user + "/" + imgID
+	img, err := s.imageService.Load(id)
+	if err != nil {
+		log.Printf("[WARN] can't load image %s: %v", id, err)
+		sendPictureError(w, r, http.StatusBadRequest, fmt.Errorf("image not found"), "can't get image", rest.ErrAssetNotFound)
+		return
+	}
+
+	contentType, err := rest.SafeImgContentType(img)
+	if err != nil {
+		log.Printf("[WARN] rejecting non-image picture %s: %v", id, err)
+		sendPictureError(w, r, http.StatusUnsupportedMediaType, err, "invalid image content", rest.ErrAssetNotFound)
+		return
+	}
+
+	// /picture/ does not need a security-version etag prefix — the upload flow
+	// validates input format (readAndValidateImage) and the serve path re-validates
+	// the stored bytes via rest.SafeImgContentType. Bytes within the resize dimension
+	// limits ARE preserved verbatim by resize, so the browser defense relies on the
+	// response headers (validated Content-Type + nosniff + strict CSP +
+	// Content-Disposition: inline), not on byte normalization. Picture IDs are limited
+	// to safePictureSegment (alphanumeric xid-generated guids), so the comma split
+	// inside rest.EtagMatches cannot collide; if the ID format ever changes, revisit.
 	etag := `"` + id + `"`
 	w.Header().Set("Etag", etag)
 	w.Header().Set("Cache-Control", "max-age=604800") // 7 days
-	if match := r.Header.Get("If-None-Match"); match != "" {
+	if match := r.Header.Get("If-None-Match"); match != "" && rest.EtagMatches(match, etag) {
-		if strings.Contains(match, etag) {
+		w.WriteHeader(http.StatusNotModified)
-			w.WriteHeader(http.StatusNotModified)
+		return
-			return
-		}
 	}
 
-	w.Header().Set("Content-Type", s.imageService.ImgContentType(img))
+	w.Header().Set("Content-Type", contentType)
 	w.Header().Set("Content-Length", strconv.Itoa(len(img)))
 	w.WriteHeader(http.StatusOK)
 	if _, err = io.Copy(w, bytes.NewReader(img)); err != nil {
@@ -344,24 +447,49 @@ func (s *public) loadPictureCtrl(w http.ResponseWriter, r *http.Request) {
 	}
 }
 
-// GET /index.html - respond to /index.html with the content of getstarted.html under /web root
-func (s *public) getStartedCtrl(w http.ResponseWriter, r *http.Request) {
-	data, err := os.ReadFile(path.Join(s.webRoot, "getstarted.html"))
-	if err != nil {
-		w.WriteHeader(http.StatusNotFound)
-		return
-	}
-	render.HTML(w, r, string(data))
-}
-
 // GET /robots.txt
-func (s *public) robotsCtrl(w http.ResponseWriter, r *http.Request) {
+func (s *public) robotsCtrl(w http.ResponseWriter, _ *http.Request) {
 	allowed := []string{"/find", "/last", "/id", "/count", "/counts", "/list", "/config", "/user",
 		"/img", "/avatar", "/picture"}
 	for i := range allowed {
 		allowed[i] = "Allow: /api/v1" + allowed[i]
 	}
-	render.PlainText(w, r, "User-agent: *\nDisallow: /auth/\nDisallow: /api/\n"+strings.Join(allowed, "\n")+"\n")
+	responseText := fmt.Sprintf("User-agent: *\nDisallow: /auth/\nDisallow: /api/\n%s\n", strings.Join(allowed, "\n"))
+	rest.PlainTextResponse(w, http.StatusOK, responseText)
+}
+
+// GET /qr/telegram - generates QR for provided URL, used for Telegram auth and notifications subscription. The first
+// step of both is the user opening a link to Telegram and writing bot a message, and the user might try to log in
+// from a computer but have Telegram installed only on the mobile device.
+// Provided text should start with https://t.me/.
+func (s *public) telegramQrCtrl(w http.ResponseWriter, r *http.Request) {
+	text := r.URL.Query().Get("url")
+	if text == "" {
+		rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("missing parameter"), "text parameter is required", rest.ErrInternal)
+		return
+	}
+
+	if !strings.HasPrefix(text, "https://t.me/") {
+		rest.SendErrorJSON(w, r, http.StatusBadRequest, fmt.Errorf("wrong parameter"), "text parameter should start with https://t.me/", rest.ErrInternal)
+		return
+	}
+
+	q, err := qrcode.New(text, qrcode.Medium)
+	if err != nil {
+		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't generate QR", rest.ErrInternal)
+		return
+	}
+	q.DisableBorder = true
+	png, err := q.PNG(256)
+	if err != nil {
+		rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't generate QR", rest.ErrInternal)
+		return
+	}
+
+	w.Header().Set("Content-Type", "image/png")
+	if _, err = w.Write(png); err != nil { //nolint:gosec // png bytes from go-qrcode, not HTML
+		log.Printf("[WARN] can't render qr, %v", err)
+	}
 }
 
 func (s *public) applyView(comments []store.Comment, view string) []store.Comment {
@@ -388,9 +516,31 @@ func (s *public) parseSince(r *http.Request) (time.Time, error) {
 	if since := r.URL.Query().Get("since"); since != "" {
 		unixTS, e := strconv.ParseInt(since, 10, 64)
 		if e != nil {
-			return time.Time{}, errors.Wrap(e, "can't translate since parameter")
+			return time.Time{}, fmt.Errorf("can't translate since parameter: %w", e)
 		}
 		sinceTS = time.Unix(unixTS/1000, 1000000*(unixTS%1000)) // since param in msec timestamp
 	}
 	return sinceTS, nil
 }
+
+// limitComments returns limited list of comments and count of comments left after limit.
+// If offsetID is provided, the list will be sliced starting from the comment with this ID.
+// If offsetID is not found, the full list will be returned.
+// It's used for only "
+func limitComments(c []store.Comment, limit int, offsetID string) (comments []store.Comment, countLeft int) {
+	if offsetID != "" {
+		for i, comment := range c {
+			if comment.ID == offsetID {
+				c = c[i+1:]
+				break
+			}
+		}
+	}
+
+	if limit > 0 && len(c) > limit {
+		countLeft = len(c) - limit
+		c = c[:limit]
+	}
+
+	return c, countLeft
+}

backend/app/rest/api/rest_public_test.go

@@ -1,20 +1,28 @@
 package api
 
 import (
+	"bytes"
 	"encoding/json"
 	"fmt"
 	"io"
+	"mime/multipart"
 	"net/http"
+	"net/http/httptest"
+	"os"
+	"strconv"
 	"strings"
 	"testing"
 	"time"
 
-	cache "github.com/go-pkgz/lcw"
+	"github.com/go-chi/chi/v5"
+	cache "github.com/go-pkgz/lcw/v2"
 	R "github.com/go-pkgz/rest"
+	"github.com/google/uuid"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
 	"github.com/umputun/remark42/backend/app/store"
+	"github.com/umputun/remark42/backend/app/store/image"
 	"github.com/umputun/remark42/backend/app/store/service"
 )
 
@@ -22,9 +30,32 @@ func TestRest_Ping(t *testing.T) {
 	ts, _, teardown := startupT(t)
 	defer teardown()
 
-	res, code := get(t, ts.URL+"/api/v1/ping")
+	resp, err := http.Get(ts.URL + "/api/v1/ping")
-	assert.Equal(t, "pong", res)
+	require.NoError(t, err)
-	assert.Equal(t, 200, code)
+	defer resp.Body.Close()
+	body, err := io.ReadAll(resp.Body)
+	require.NoError(t, err)
+
+	assert.Equal(t, "pong", string(body))
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	assert.Equal(t, "remark42", resp.Header.Get("App-Name"))
+}
+
+func TestRest_PingNoSignature(t *testing.T) {
+	ts, _, teardown := startupT(t, func(srv *Rest) {
+		srv.DisableSignature = true
+	})
+	defer teardown()
+
+	resp, err := http.Get(ts.URL + "/api/v1/ping")
+	require.NoError(t, err)
+	defer resp.Body.Close()
+	body, err := io.ReadAll(resp.Body)
+	require.NoError(t, err)
+
+	assert.Equal(t, "pong", string(body))
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	assert.Equal(t, "", resp.Header.Get("App-Name"))
 }
 
 func TestRest_Preview(t *testing.T) {
@@ -42,7 +73,7 @@ func TestRest_Preview(t *testing.T) {
 	resp, err = post(t, ts.URL+"/api/v1/preview", "bad")
 	assert.NoError(t, err)
 	assert.NoError(t, resp.Body.Close())
-	assert.Equal(t, 400, resp.StatusCode)
+	assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
 
 	resp, err = post(t, ts.URL+"/api/v1/preview", fmt.Sprintf(`{"text": "![non-existent.jpg](%s/api/v1/picture/dev_user/bad_picture)", "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`, srv.RemarkURL))
 	assert.NoError(t, err)
@@ -52,14 +83,31 @@ func TestRest_Preview(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 	assert.Contains(t,
 		string(b),
-		"{\"code\":20,\"details\":\"can't renew staged picture cleanup timer\","+
+		`{"code":20,"details":"can't load picture from the comment",`+
-			"\"error\":\"can't get image stats for dev_user/bad_picture: stat",
+			`"error":"can't get image stats for dev_user/bad_picture: stat`,
 	)
 	assert.Contains(t,
 		string(b),
 		"/pics-remark42/staging/dev_user/62/bad_picture: no such file or directory\"}\n",
 	)
 
+	// test quotes with and without smartypants
+	resp, err = post(t, ts.URL+"/api/v1/preview", `{"text": "\"quoted\" text", "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`)
+	assert.NoError(t, err)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	b, err = io.ReadAll(resp.Body)
+	assert.NoError(t, err)
+	assert.NoError(t, resp.Body.Close())
+	assert.Equal(t, "<p>«quoted» text</p>\n", string(b))
+
+	srv.privRest.disableFancyTextFormatting = true
+	resp, err = post(t, ts.URL+"/api/v1/preview", `{"text": "\"quoted\" text", "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`)
+	assert.NoError(t, err)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	b, err = io.ReadAll(resp.Body)
+	assert.NoError(t, err)
+	assert.NoError(t, resp.Body.Close())
+	assert.Equal(t, "<p>&#34;quoted&#34; text</p>\n", string(b))
 }
 
 func TestRest_PreviewWithWrongImage(t *testing.T) {
@@ -74,8 +122,8 @@ func TestRest_PreviewWithWrongImage(t *testing.T) {
 	assert.NoError(t, resp.Body.Close())
 	assert.Contains(t,
 		string(b),
-		"{\"code\":20,\"details\":\"can't renew staged picture cleanup timer\","+
+		`{"code":20,"details":"can't load picture from the comment",`+
-			"\"error\":\"can't get image stats for dev_user/bad_picture: stat ",
+			`"error":"can't get image stats for dev_user/bad_picture: stat `,
 	)
 	assert.Contains(t,
 		string(b),
@@ -97,9 +145,9 @@ srv, ts := prep(t)
 }
 BKT
 `
-	text = strings.Replace(text, "BKT", "```", -1)
+	text = strings.ReplaceAll(text, "BKT", "```")
 	j := fmt.Sprintf(`{"text": %q, "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`, text)
-	j = strings.Replace(j, "\n", "\\n", -1)
+	j = strings.ReplaceAll(j, "\n", "\\n")
 
 	resp, err := post(t, ts.URL+"/api/v1/preview", j)
 	assert.NoError(t, err)
@@ -108,11 +156,11 @@ BKT
 	assert.NoError(t, err)
 	assert.Equal(t,
 		`<h1>h1</h1>
-<pre class="chroma">func TestRest_Preview(t *testing.T) {
+<pre class="chroma"><code><span class="line"><span class="cl"><span class="k">func</span> <span class="n">TestRest_Preview</span><span class="p">(</span><span class="n">t</span> <span class="o">*</span><span class="n">testing</span><span class="o">.</span><span class="n">T</span><span class="p">)</span> <span class="p">{</span>
-srv, ts := prep(t)
+</span></span><span class="line"><span class="cl"><span class="n">srv</span><span class="p">,</span> <span class="n">ts</span> <span class="p">:</span><span class="o">=</span> <span class="n">prep</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
-  require.NotNil(t, srv)
+</span></span><span class="line"><span class="cl">  <span class="n">require</span><span class="o">.</span><span class="n">NotNil</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">srv</span><span class="p">)</span>
-}
+</span></span><span class="line"><span class="cl"><span class="p">}</span>
-</pre>`,
+</span></span></code></pre>`,
 		string(b))
 	assert.NoError(t, resp.Body.Close())
 }
@@ -125,17 +173,17 @@ func TestRest_PreviewCode(t *testing.T) {
 func main(aa string) int {return 0}
 BKT
 `
-	text = strings.Replace(text, "BKT", "```", -1)
+	text = strings.ReplaceAll(text, "BKT", "```")
 	j := fmt.Sprintf(`{"text": %q, "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`, text)
-	j = strings.Replace(j, "\n", "\\n", -1)
+	j = strings.ReplaceAll(j, "\n", "\\n")
 
 	resp, err := post(t, ts.URL+"/api/v1/preview", j)
 	assert.NoError(t, err)
 	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	b, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
-	assert.Equal(t, `<pre class="chroma"><span class="kd">func</span> <span class="nf">main</span><span class="p">(</span><span class="nx">aa</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">int</span> <span class="p">{</span><span class="k">return</span> <span class="mi">0</span><span class="p">}</span>
+	assert.Equal(t, `<pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="nx">aa</span><span class="w"> </span><span class="kt">string</span><span class="p">)</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="p">{</span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">}</span><span class="w">
-</pre>`, string(b))
+</span></span></span></code></pre>`, string(b))
 	assert.NoError(t, resp.Body.Close())
 }
 
@@ -144,7 +192,7 @@ func TestRest_Find(t *testing.T) {
 	defer teardown()
 
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := commentsWithInfo{}
 	err := json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -162,7 +210,7 @@ func TestRest_Find(t *testing.T) {
 
 	// get sorted by +time
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=+time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -178,7 +226,7 @@ func TestRest_Find(t *testing.T) {
 
 	// get sorted by -time
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=-time")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
 	require.Equal(t, 2, len(comments.Comments), "should have 2 comments")
@@ -186,9 +234,9 @@ func TestRest_Find(t *testing.T) {
 	assert.Equal(t, id2, comments.Comments[0].ID)
 
 	// get in tree mode
-	tree := service.Tree{}
+	tree := treeWithInfo{}
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&format=tree")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &tree)
 	assert.NoError(t, err)
 	require.Equal(t, 1, len(tree.Nodes))
@@ -212,17 +260,17 @@ func TestRest_FindAge(t *testing.T) {
 	_, err = srv.DataService.Create(c2)
 	require.NoError(t, err)
 
-	tree := service.Tree{}
+	tree := treeWithInfo{}
 
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&format=tree")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &tree)
 	assert.NoError(t, err)
 	assert.Equal(t, "https://radio-t.com/blah1", tree.Info.URL)
 	assert.False(t, tree.Info.ReadOnly, "post is fresh")
 
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah2&format=tree")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &tree)
 	assert.NoError(t, err)
 	assert.Equal(t, "https://radio-t.com/blah2", tree.Info.URL)
@@ -246,6 +294,7 @@ func TestRest_FindReadOnly(t *testing.T) {
 
 	// set post to read-only
 	client := http.Client{}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest(http.MethodPut,
 		fmt.Sprintf("%s/api/v1/admin/readonly?site=remark42&url=https://radio-t.com/blah1&ro=1", ts.URL), http.NoBody)
 	assert.NoError(t, err)
@@ -254,17 +303,17 @@ func TestRest_FindReadOnly(t *testing.T) {
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
 
-	tree := service.Tree{}
+	tree := treeWithInfo{}
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&format=tree")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &tree)
 	require.NoError(t, err)
 	assert.Equal(t, "https://radio-t.com/blah1", tree.Info.URL)
 	assert.True(t, tree.Info.ReadOnly, "post is ro")
 
-	tree = service.Tree{}
+	tree = treeWithInfo{}
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah2&format=tree")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &tree)
 	require.NoError(t, err)
 	assert.Equal(t, "https://radio-t.com/blah2", tree.Info.URL)
@@ -276,7 +325,7 @@ func TestRest_FindUserView(t *testing.T) {
 	defer teardown()
 
 	res, code := get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&view=user")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := commentsWithInfo{}
 	err := json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -294,15 +343,15 @@ func TestRest_FindUserView(t *testing.T) {
 
 	// get sorted by +time with view=user
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=+time&view=user")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
 	require.Equal(t, 2, len(comments.Comments), "should have 2 comments")
 	assert.Equal(t, id1, comments.Comments[0].ID)
 	assert.Equal(t, id2, comments.Comments[1].ID)
-	assert.Equal(t, "dev", comments.Comments[0].User.ID)
+	assert.Equal(t, "provider1_dev", comments.Comments[0].User.ID)
-	assert.Equal(t, "dev", comments.Comments[1].User.ID)
+	assert.Equal(t, "provider1_dev", comments.Comments[1].User.ID)
 	assert.Equal(t, "", comments.Comments[0].Text)
 	assert.Equal(t, "", comments.Comments[1].Text)
 
@@ -311,7 +360,7 @@ func TestRest_FindUserView(t *testing.T) {
 	srv.Cache.Flush(cache.FlusherRequest{})
 
 	res, code = get(t, ts.URL+"/api/v1/find?site=remark42&url=https://radio-t.com/blah1&sort=+time&view=user")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = commentsWithInfo{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -324,7 +373,7 @@ func TestRest_Last(t *testing.T) {
 	defer teardown()
 
 	res, code := get(t, ts.URL+"/api/v1/last/2?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	assert.Equal(t, "[]\n", res, "empty last should return empty list")
 
 	c1 := store.Comment{Text: "test test #1", ParentID: "p1",
@@ -341,7 +390,7 @@ func TestRest_Last(t *testing.T) {
 	id2 := addComment(t, c2, ts)
 
 	res, code = get(t, ts.URL+"/api/v1/last/2?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments := []store.Comment{}
 	err := json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -350,7 +399,7 @@ func TestRest_Last(t *testing.T) {
 	assert.Equal(t, id2, comments[0].ID)
 
 	res, code = get(t, fmt.Sprintf("%s/api/v1/last/2?site=remark42&since=%d", ts.URL, ts1))
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = []store.Comment{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -359,7 +408,7 @@ func TestRest_Last(t *testing.T) {
 	assert.Equal(t, id2, comments[0].ID)
 
 	res, code = get(t, fmt.Sprintf("%s/api/v1/last/2?site=remark42&since=%d", ts.URL, ts2))
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	comments = []store.Comment{}
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
@@ -367,13 +416,13 @@ func TestRest_Last(t *testing.T) {
 	assert.Equal(t, id2, comments[0].ID)
 
 	res, code = get(t, ts.URL+"/api/v1/last/5?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
 	assert.Equal(t, 3, len(comments), "should have 3 comments")
 
 	res, code = get(t, ts.URL+"/api/v1/last/X?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
 	assert.Equal(t, 3, len(comments), "should have 3 comments")
@@ -382,13 +431,13 @@ func TestRest_Last(t *testing.T) {
 	assert.NoError(t, err)
 	srv.Cache.Flush(cache.FlusherRequest{})
 	res, code = get(t, ts.URL+"/api/v1/last/5?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(res), &comments)
 	assert.NoError(t, err)
 	assert.Equal(t, 2, len(comments), "should have 2 comments")
 
 	_, code = get(t, ts.URL+"/api/v1/last/2?site=remark42-BLAH")
-	assert.Equal(t, 500, code)
+	assert.Equal(t, http.StatusInternalServerError, code)
 }
 
 func TestRest_FindUserComments(t *testing.T) {
@@ -412,12 +461,12 @@ func TestRest_FindUserComments(t *testing.T) {
 	err := srv.DataService.Delete(c2.Locator, id, store.SoftDelete)
 	assert.NoError(t, err)
 
-	_, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=blah")
+	comments, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=blah")
-	assert.Equal(t, 400, code, "noting for user blah")
+	assert.Equal(t, http.StatusOK, code, "noting for user blah")
-
+	assert.Equal(t, `{"comments":[],"count":0}`+"\n", comments)
 	{
-		res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=dev")
+		res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=provider1_dev")
-		assert.Equal(t, 200, code)
+		assert.Equal(t, http.StatusOK, code)
 
 		resp := struct {
 			Comments []store.Comment
@@ -435,8 +484,8 @@ func TestRest_FindUserComments(t *testing.T) {
 	}
 
 	{
-		res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=dev&skip=1&limit=2")
+		res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=provider1_dev&skip=1&limit=2")
-		assert.Equal(t, 200, code)
+		assert.Equal(t, http.StatusOK, code)
 
 		resp := struct {
 			Comments []store.Comment
@@ -453,16 +502,298 @@ func TestRest_FindUserComments(t *testing.T) {
 	}
 }
 
+func TestRest_FindUserComments_CWE_918(t *testing.T) {
+	ts, srv, teardown := startupT(t)
+	srv.DataService.TitleExtractor = service.NewTitleExtractor(http.Client{Timeout: time.Second}, []string{"radio-t.com"}) // required for extracting the title, bad URL test
+	defer srv.DataService.TitleExtractor.Close()
+	defer teardown()
+
+	backendRequestedArbitraryServer := false
+	arbitraryServer := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
+		t.Logf("request received: %+v", r)
+		backendRequestedArbitraryServer = true
+	}))
+	defer arbitraryServer.Close()
+
+	arbitraryURLComment := store.Comment{Text: "arbitrary URL request test",
+		Locator: store.Locator{SiteID: "remark42", URL: arbitraryServer.URL}}
+
+	assert.False(t, backendRequestedArbitraryServer)
+	addComment(t, arbitraryURLComment, ts)
+	assert.False(t, backendRequestedArbitraryServer,
+		"no request is expected to the test server as it's not in the list of the allowed domains for the title extractor")
+
+	res, code := get(t, ts.URL+"/api/v1/comments?site=remark42&user=provider1_dev")
+	assert.Equal(t, http.StatusOK, code)
+
+	resp := struct {
+		Comments []store.Comment
+		Count    int
+	}{}
+
+	err := json.Unmarshal([]byte(res), &resp)
+	assert.NoError(t, err)
+	require.Equal(t, 1, len(resp.Comments), "should have 2 comments")
+
+	assert.Equal(t, "", resp.Comments[0].PostTitle, "empty from the first post")
+	assert.Equal(t, arbitraryServer.URL, resp.Comments[0].Locator.URL, "arbitrary URL provided by the request")
+}
+
+func TestPublic_FindCommentsCtrl_ConsistentCount(t *testing.T) {
+	// test that comment counting is consistent between tree and plain formats
+	ts, srv, teardown := startupT(t)
+	defer teardown()
+
+	commentLocator := store.Locator{URL: "test-url", SiteID: "remark42"}
+
+	// vote for comment multiple times
+	setScore := func(locator store.Locator, id string, val int) {
+		abs := func(x int) int {
+			if x < 0 {
+				return -x
+			}
+			return x
+		}
+		for i := 0; i < abs(val); i++ {
+			_, err := srv.DataService.Vote(service.VoteReq{
+				Locator:   locator,
+				CommentID: id,
+				// unique user ID is needed for correct counting of controversial votes
+				UserID: "user" + strconv.Itoa(val) + strconv.Itoa(i),
+				Val:    val > 0,
+			})
+			require.NoError(t, err)
+		}
+	}
+
+	// adding initial comments (8 to test-url and 1 to another-url) and voting, and delete two of comments to the first post.
+	// with sleep so that at least few millisecond pass between each comment
+	// and later we would be able to use that in "since" filter with millisecond precision
+	ids := make([]string, 9)
+	timestamps := make([]time.Time, 9)
+	c1 := store.Comment{Text: "top-level comment 1", Locator: commentLocator}
+	ids[0], timestamps[0] = addCommentGetCreatedTime(t, c1, ts)
+	// #3 by score
+	setScore(commentLocator, ids[0], 1)
+	time.Sleep(time.Millisecond * 5)
+
+	c2 := store.Comment{Text: "top-level comment 2", Locator: commentLocator}
+	ids[1], timestamps[1] = addCommentGetCreatedTime(t, c2, ts)
+	// #2 by score
+	setScore(commentLocator, ids[1], 2)
+	time.Sleep(time.Millisecond * 5)
+
+	c3 := store.Comment{Text: "second-level comment 1", ParentID: ids[0], Locator: commentLocator}
+	ids[2], timestamps[2] = addCommentGetCreatedTime(t, c3, ts)
+	// #1 by score
+	setScore(commentLocator, ids[2], 10)
+	time.Sleep(time.Millisecond * 5)
+
+	c4 := store.Comment{Text: "third-level comment 1", ParentID: ids[2], Locator: commentLocator}
+	ids[3], timestamps[3] = addCommentGetCreatedTime(t, c4, ts)
+	// #5 by score, #1 by controversy
+	setScore(commentLocator, ids[3], 4)
+	setScore(commentLocator, ids[3], -4)
+	time.Sleep(time.Millisecond * 5)
+
+	c5 := store.Comment{Text: "second-level comment 2", ParentID: ids[1], Locator: commentLocator}
+	ids[4], timestamps[4] = addCommentGetCreatedTime(t, c5, ts)
+	// #5 by score, #2 by controversy
+	setScore(commentLocator, ids[4], 2)
+	setScore(commentLocator, ids[4], -3)
+	time.Sleep(time.Millisecond * 5)
+
+	c6 := store.Comment{Text: "deleted third-level comment 2", ParentID: ids[4], Locator: commentLocator}
+	ids[5], timestamps[5] = addCommentGetCreatedTime(t, c6, ts)
+	// deleted later so not visible in site-wide requests
+	setScore(commentLocator, ids[5], 10)
+	setScore(commentLocator, ids[5], -10)
+	time.Sleep(time.Millisecond * 5)
+
+	c7 := store.Comment{Text: "top-level comment 3", Locator: commentLocator}
+	ids[6], timestamps[6] = addCommentGetCreatedTime(t, c7, ts)
+	// #6 by score, #4 by controversy
+	setScore(commentLocator, ids[6], -3)
+	setScore(commentLocator, ids[6], 1)
+	time.Sleep(time.Millisecond * 5)
+
+	c8 := store.Comment{Text: "deleted second-level comment 3", ParentID: ids[6], Locator: commentLocator}
+	ids[7], timestamps[7] = addCommentGetCreatedTime(t, c8, ts)
+	// deleted later so not visible in site-wide requests
+	setScore(commentLocator, ids[7], -20)
+
+	c9 := store.Comment{Text: "comment to post 2", Locator: store.Locator{URL: "another-url", SiteID: "remark42"}}
+	ids[8], timestamps[8] = addCommentGetCreatedTime(t, c9, ts)
+	// #7 by score
+	setScore(store.Locator{URL: "another-url", SiteID: "remark42"}, ids[8], -25)
+
+	// delete two comments bringing the total from 9 to 6
+	err := srv.DataService.Delete(commentLocator, ids[7], store.SoftDelete)
+	assert.NoError(t, err)
+	err = srv.DataService.Delete(commentLocator, ids[5], store.HardDelete)
+	assert.NoError(t, err)
+	srv.Cache.Flush(cache.FlusherRequest{})
+
+	commentLocator.URL = "readonly-test"
+	// set post without comments to read-only
+	assert.NoError(t, srv.DataService.SetReadOnly(commentLocator, true))
+
+	sinceTenSecondsAgo := strconv.FormatInt(time.Now().Add(-time.Second*10).UnixNano()/1000000, 10)
+	sinceTS := make([]string, 9)
+	formattedTS := make([]string, 9)
+	for i, created := range timestamps {
+		sinceTS[i] = strconv.FormatInt(created.UnixNano()/1000000, 10)
+		formattedTS[i] = created.Format(time.RFC3339Nano)
+	}
+	t.Logf("last timestamp: %v", timestamps[7])
+
+	testCases := []struct {
+		params       string
+		expectedBody string
+	}{
+		// test parameters url, format, since, sort
+		{"", fmt.Sprintf(`"info":{"count":7,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[8])},
+		{"url=test-url", fmt.Sprintf(`"info":{"url":"test-url","count":6,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[7])},
+		{"format=plain", fmt.Sprintf(`"info":{"count":7,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[8])},
+		{"format=plain&url=test-url", fmt.Sprintf(`"info":{"url":"test-url","count":6,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[7])},
+		{"since=" + sinceTenSecondsAgo, fmt.Sprintf(`"info":{"count":7,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[8])},
+		{"url=test-url&since=" + sinceTenSecondsAgo, fmt.Sprintf(`"info":{"url":"test-url","count":6,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[7])},
+		{"since=" + sinceTS[0], fmt.Sprintf(`"info":{"count":7,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[8])},
+		{"url=test-url&since=" + sinceTS[0], fmt.Sprintf(`"info":{"url":"test-url","count":6,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[7])},
+		{"since=" + sinceTS[1], fmt.Sprintf(`"info":{"count":6,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[8])},
+		{"url=test-url&since=" + sinceTS[1], fmt.Sprintf(`"info":{"url":"test-url","count":5,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[7])},
+		{"since=" + sinceTS[4], fmt.Sprintf(`"info":{"count":3,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[8])},
+		{"url=test-url&since=" + sinceTS[4], fmt.Sprintf(`"info":{"url":"test-url","count":2,"count_left":0,"first_time":%q,"last_time":%q}`, formattedTS[0], formattedTS[7])},
+		{"format=tree", `"info":{"count":7`},
+		{"format=tree&url=test-url", `"info":{"url":"test-url","count":6`},
+		{"format=tree&sort=+time", `"info":{"count":7`},
+		{"format=tree&url=test-url&sort=+time", `"info":{"url":"test-url","count":6`},
+		{"format=tree&sort=-score", `"info":{"count":7`},
+		{"format=tree&url=test-url&sort=-score", `"info":{"url":"test-url","count":6`},
+		{"sort=+time", fmt.Sprintf(`"score":-25,"vote":0,"time":%q}],"info":{"count":7`, formattedTS[8])},
+		{"sort=-time", fmt.Sprintf(`"score":1,"vote":0,"time":%q}],"info":{"count":7`, formattedTS[0])},
+		{"sort=+score", fmt.Sprintf(`"score":10,"vote":0,"time":%q}],"info":{"count":7`, formattedTS[2])},
+		{"sort=+score&url=test-url", fmt.Sprintf(`"score":10,"vote":0,"time":%q}],"info":{"url":"test-url","count":6`, formattedTS[2])},
+		{"sort=-score", fmt.Sprintf(`"score":-25,"vote":0,"time":%q}],"info":{"count":7`, formattedTS[8])},
+		{"sort=-score&url=test-url", fmt.Sprintf(`"score":-2,"vote":0,"controversy":1.5874010519681994,"time":%q}],"info":{"url":"test-url","count":6`, formattedTS[6])},
+		{"sort=-time&since=" + sinceTS[4], fmt.Sprintf(`"score":-1,"vote":0,"controversy":2.924017738212866,"time":%q}],"info":{"count":3`, formattedTS[4])},
+		{"sort=-score&since=" + sinceTS[3], fmt.Sprintf(`"score":-25,"vote":0,"time":%q}],"info":{"count":4`, formattedTS[8])},
+		{"sort=-score&url=test-url&since=" + sinceTS[3], fmt.Sprintf(`"score":-2,"vote":0,"controversy":1.5874010519681994,"time":%q}],"info":{"url":"test-url","count":3`, formattedTS[6])},
+		{"sort=+controversy&url=test-url&since=" + sinceTS[5], fmt.Sprintf(`"score":-2,"vote":0,"controversy":1.5874010519681994,"time":%q}],"info":{"url":"test-url","count":1`, formattedTS[6])},
+		// three comments of which last one deleted and doesn't have controversy so returned last
+		{"sort=-controversy&url=test-url&since=" + sinceTS[5], fmt.Sprintf(`"score":0,"vote":0,"time":%q,"delete":true}],"info":{"url":"test-url","count":1`, formattedTS[7])},
+		// test readonly status for the post without comments
+		{"url=readonly-test", `"info":{"count":0,"count_left":0,"read_only":true`},
+		{"format=tree&url=readonly-test", `"info":{"count":0,"count_left":0,"read_only":true`},
+
+		// test parameters limit, offset_id for format=plain
+		{"limit=bad", `{"code":1,"details":"bad limit value","error":"strconv.Atoi: parsing \"bad\": invalid syntax"}`},
+		{"offset_id=bad", `{"code":1,"details":"bad offset_id value","error":"invalid UUID length: 3"}`},
+		{"limit=2", `"info":{"count":7,"count_left":5,"last_comment":"` + ids[1]},
+		{"limit=6", `"info":{"count":7,"count_left":1,"last_comment":"` + ids[6]},
+		{"limit=7", `"info":{"count":7,"count_left":0,"last_comment":"` + ids[8]},
+		{"limit=2&url=test-url", `"info":{"url":"test-url","count":6,"count_left":6,"last_comment":"` + ids[1]},
+		{"limit=6&url=test-url", `"info":{"url":"test-url","count":6,"count_left":2,"last_comment":"` + ids[5]},
+		{"limit=7&url=test-url", `"info":{"url":"test-url","count":6,"count_left":1,"last_comment":"` + ids[6]},
+		{fmt.Sprintf("limit=2&offset_id=%s", ids[2]), `"info":{"count":7,"count_left":2,"last_comment":"` + ids[4]},
+		{fmt.Sprintf("limit=2&offset_id=%s", ids[3]), `"info":{"count":7,"count_left":1,"last_comment":"` + ids[6]},
+		{fmt.Sprintf("limit=2&offset_id=%s", ids[4]), `"info":{"count":7,"count_left":0`},
+		{fmt.Sprintf("limit=1&offset_id=%s", ids[6]), `"info":{"count":7,"count_left":0`},
+		{fmt.Sprintf("limit=2&offset_id=%s", ids[8]), `"info":{"count":7,"count_left":0`},
+		{fmt.Sprintf("limit=2&url=test-url&offset_id=%s", ids[2]), `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[4]},
+		{fmt.Sprintf("limit=2&url=test-url&offset_id=%s", ids[3]), `"info":{"url":"test-url","count":6,"count_left":2,"last_comment":"` + ids[5]},
+		{fmt.Sprintf("limit=2&url=test-url&offset_id=%s", ids[4]), `"info":{"url":"test-url","count":6,"count_left":1,"last_comment":"` + ids[6]},
+		{fmt.Sprintf("limit=1&url=test-url&offset_id=%s", ids[6]), `"info":{"url":"test-url","count":6,"count_left":0,"last_comment":"` + ids[7]},
+		{fmt.Sprintf("limit=2&url=test-url&offset_id=%s", ids[8]), `"info":{"url":"test-url","count":6,"count_left":6,`},
+		// deleted comment, offset is ignored in site-wide request but not for particular URL
+		{fmt.Sprintf("limit=2&offset_id=%s", ids[5]), `"info":{"count":7,"count_left":5,"last_comment":"` + ids[1]},
+		{fmt.Sprintf("limit=2&url=test-url&offset_id=%s", ids[5]), `"info":{"url":"test-url","count":6,"count_left":0,"last_comment":"` + ids[7]},
+		// non-existing comment, offset is ignored, deleted comments included into request with "url"
+		{fmt.Sprintf("limit=1&offset_id=%s", uuid.New().String()), `"info":{"count":7,"count_left":6,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("limit=1&url=test-url&offset_id=%s", uuid.New().String()), `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[0]},
+		// since is ignored for tree format, so we test it only for plain
+		{"limit=6&since=" + sinceTenSecondsAgo, `"info":{"count":7,"count_left":1,"last_comment":"` + ids[6]},
+		{"limit=1&since=" + sinceTS[4], `"info":{"count":3,"count_left":2,"last_comment":"` + ids[4]},
+		{"limit=6&url=test-url&since=" + sinceTenSecondsAgo, `"info":{"url":"test-url","count":6,"count_left":2,"last_comment":"` + ids[5]},
+		{"limit=1&url=test-url&since=" + sinceTS[4], `"info":{"url":"test-url","count":2,"count_left":3,"last_comment":"` + ids[4]},
+		// start with deleted comment timestamp
+		{"limit=1&since=" + sinceTS[5], `"info":{"count":2,"count_left":1,"last_comment":"` + ids[6]},
+		{"limit=1&since=" + sinceTS[6], `"info":{"count":2,"count_left":1,"last_comment":"` + ids[6]},
+		{"limit=1&url=test-url&since=" + sinceTS[5], `"info":{"url":"test-url","count":1,"count_left":2,"last_comment":"` + ids[5]},
+		{"limit=1&url=test-url&since=" + sinceTS[6], `"info":{"url":"test-url","count":1,"count_left":1,"last_comment":"` + ids[6]},
+		// test sort
+		{"limit=1&sort=+time&url=test-url", `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[0]},
+		{"limit=1&sort=-time&url=test-url", `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[7]},
+		{"limit=1&sort=+score&url=test-url", `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[6]},
+		{"limit=1&sort=-score&url=test-url", `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[2]},
+		{"limit=1&sort=+controversy&url=test-url", `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[0]},
+		{"limit=1&sort=-controversy&url=test-url", `"info":{"url":"test-url","count":6,"count_left":7,"last_comment":"` + ids[3]},
+
+		// test parameters limit, offset_id for format=tree
+		{"format=tree&limit=bad", `{"code":1,"details":"bad limit value","error":"strconv.Atoi: parsing \"bad\": invalid syntax"}`},
+		{"format=tree&offset_id=bad", `{"code":1,"details":"bad offset_id value","error":"invalid UUID length: 3"}`},
+		{"format=tree&limit=2", `"info":{"count":7,"count_left":4,"last_comment":"` + ids[0]},
+		{"format=tree&limit=6", `"info":{"count":7,"count_left":2,"last_comment":"` + ids[1]},
+		{"format=tree&limit=7", `"info":{"count":7,"count_left":1,"last_comment":"` + ids[6]},
+		{"format=tree&url=test-url&limit=2", `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{"format=tree&url=test-url&limit=6", `"info":{"url":"test-url","count":6,"count_left":1,"last_comment":"` + ids[1]},
+		{"format=tree&url=test-url&limit=7", `"info":{"url":"test-url","count":6,"count_left":0,"last_comment":"` + ids[6]},
+		// start after first top-level comment
+		{fmt.Sprintf("format=tree&limit=2&offset_id=%s", ids[0]), `"info":{"count":7,"count_left":2,"last_comment":"` + ids[1]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=2&offset_id=%s", ids[0]), `"info":{"url":"test-url","count":6,"count_left":1,"last_comment":"` + ids[1]},
+		// start after second top-level comment
+		{fmt.Sprintf("format=tree&limit=2&offset_id=%s", ids[1]), `"info":{"count":7,"count_left":1,"last_comment":"` + ids[6]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=2&offset_id=%s", ids[1]), `"info":{"url":"test-url","count":6,"count_left":0,"last_comment":"` + ids[6]},
+		// start after third top-level comment, so expect comment to post 2, or no comments on post 1 if "url" is set
+		{fmt.Sprintf("format=tree&limit=1&offset_id=%s", ids[6]), `"info":{"count":7,"count_left":0,"last_comment":"` + ids[8]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=1&offset_id=%s", ids[6]), `"info":{"url":"test-url","count":6,"count_left":0`},
+		// non-root comment IDs or non-existing IDs are ignored
+		{fmt.Sprintf("format=tree&limit=2&offset_id=%s", ids[2]), `"info":{"count":7,"count_left":4,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&limit=2&offset_id=%s", ids[3]), `"info":{"count":7,"count_left":4,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&limit=2&offset_id=%s", ids[4]), `"info":{"count":7,"count_left":4,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&limit=2&offset_id=%s", ids[7]), `"info":{"count":7,"count_left":4,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&limit=1&offset_id=%s", uuid.New().String()), `"info":{"count":7,"count_left":4,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=2&offset_id=%s", ids[2]), `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=2&offset_id=%s", ids[3]), `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=2&offset_id=%s", ids[4]), `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=2&offset_id=%s", ids[7]), `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{fmt.Sprintf("format=tree&url=test-url&limit=1&offset_id=%s", uuid.New().String()), `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		// test sort
+		{"format=tree&limit=1&sort=+time&url=test-url", `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{"format=tree&limit=1&sort=-time&url=test-url", `"info":{"url":"test-url","count":6,"count_left":5,"last_comment":"` + ids[6]},
+		{"format=tree&limit=1&sort=+score&url=test-url", `"info":{"url":"test-url","count":6,"count_left":5,"last_comment":"` + ids[6]},
+		{"format=tree&limit=1&sort=-score&url=test-url", `"info":{"url":"test-url","count":6,"count_left":4,"last_comment":"` + ids[1]},
+		{"format=tree&limit=1&sort=+controversy&url=test-url", `"info":{"url":"test-url","count":6,"count_left":3,"last_comment":"` + ids[0]},
+		{"format=tree&limit=1&sort=-controversy&url=test-url", `"info":{"url":"test-url","count":6,"count_left":5,"last_comment":"` + ids[6]},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.params, func(t *testing.T) {
+			url := fmt.Sprintf(ts.URL+"/api/v1/find?site=remark42&%s", tc.params)
+			body, code := get(t, url)
+			expectedStatus := http.StatusOK
+			if strings.Contains(tc.params, "=bad") {
+				expectedStatus = http.StatusBadRequest
+			}
+			assert.Equal(t, expectedStatus, code)
+			assert.Contains(t, body, tc.expectedBody)
+			t.Log(body)
+			// prevent hit limiter from engaging
+			time.Sleep(80 * time.Millisecond)
+		})
+	}
+}
+
 func TestRest_UserInfo(t *testing.T) {
 	ts, _, teardown := startupT(t)
 	defer teardown()
 
 	body, code := getWithDevAuth(t, ts.URL+"/api/v1/user?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	user := store.User{}
 	err := json.Unmarshal([]byte(body), &user)
 	assert.NoError(t, err)
-	assert.Equal(t, store.User{Name: "developer one", ID: "dev", Picture: "http://example.com/pic.png",
+	assert.Equal(t, store.User{Name: "developer one", ID: "provider1_dev", Picture: "http://example.com/pic.png",
 		IP: "127.0.0.1", SiteID: "remark42"}, user)
 }
 
@@ -482,20 +813,20 @@ func TestRest_Count(t *testing.T) {
 	addComment(t, c2, ts)
 
 	body, code := get(t, ts.URL+"/api/v1/count?site=remark42&url=https://radio-t.com/blah1")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	j := R.JSON{}
 	err := json.Unmarshal([]byte(body), &j)
 	assert.NoError(t, err)
 	assert.Equal(t, 3.0, j["count"])
 
 	body, code = get(t, ts.URL+"/api/v1/count?site=remark42&url=https://radio-t.com/blah2")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	err = json.Unmarshal([]byte(body), &j)
 	assert.NoError(t, err)
 	assert.Equal(t, 2.0, j["count"])
 
 	_, code = get(t, ts.URL+"/api/v1/count?site=remark42-BLAH&url=https://radio-t.com/blah1XXX")
-	assert.Equal(t, 400, code)
+	assert.Equal(t, http.StatusBadRequest, code)
 }
 
 func TestRest_Counts(t *testing.T) {
@@ -529,7 +860,7 @@ func TestRest_Counts(t *testing.T) {
 
 	resp, err = post(t, ts.URL+"/api/v1/counts?site=radio-XXX", `{}`)
 	require.NoError(t, err)
-	assert.Equal(t, 400, resp.StatusCode)
+	assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
 	assert.NoError(t, resp.Body.Close())
 }
 
@@ -549,7 +880,7 @@ func TestRest_List(t *testing.T) {
 	addComment(t, c2, ts)
 
 	body, code := get(t, ts.URL+"/api/v1/list?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	pi := []store.PostInfo{}
 	err := json.Unmarshal([]byte(body), &pi)
 	assert.NoError(t, err)
@@ -559,7 +890,7 @@ func TestRest_List(t *testing.T) {
 	assert.Equal(t, 3, pi[1].Count)
 
 	_, code = get(t, ts.URL+"/api/v1/list?site=remark42-BLAH")
-	assert.Equal(t, 400, code)
+	assert.Equal(t, http.StatusBadRequest, code)
 }
 
 func TestRest_ListWithSkipAndLimit(t *testing.T) {
@@ -582,7 +913,7 @@ func TestRest_ListWithSkipAndLimit(t *testing.T) {
 	addComment(t, c3, ts)
 
 	body, code := get(t, ts.URL+"/api/v1/list?site=remark42&skip=1&limit=2")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	pi := []store.PostInfo{}
 	err := json.Unmarshal([]byte(body), &pi)
 	assert.NoError(t, err)
@@ -598,12 +929,12 @@ func TestRest_Config(t *testing.T) {
 	defer teardown()
 
 	body, code := get(t, ts.URL+"/api/v1/config?site=remark42")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	j := R.JSON{}
 	err := json.Unmarshal([]byte(body), &j)
 	assert.NoError(t, err)
 	assert.Equal(t, 300.0, j["edit_duration"])
-	assert.EqualValues(t, []interface{}{"a1", "a2"}, j["admins"])
+	assert.EqualValues(t, []any{"a1", "a2"}, j["admins"])
 	assert.Equal(t, "admin@remark-42.com", j["admin_email"])
 	assert.Equal(t, 4000.0, j["max_comment_size"])
 	assert.Equal(t, -5.0, j["low_score"])
@@ -615,6 +946,44 @@ func TestRest_Config(t *testing.T) {
 	assert.Equal(t, false, j["admin_edit"].(bool))
 }
 
+func TestRest_QR(t *testing.T) {
+	ts, _, teardown := startupT(t)
+	defer teardown()
+
+	// missing parameter
+	body, code := get(t, ts.URL+"/api/v1/qr/telegram")
+	assert.Equal(t, http.StatusBadRequest, code)
+	assert.Equal(t, "{\"code\":0,\"details\":\"text parameter is required\",\"error\":\"missing parameter\"}\n", body)
+
+	// too long request to build the qr
+	body, code = get(t, ts.URL+"/api/v1/qr/telegram?url=https://t.me/"+strings.Repeat("string", 1000))
+	assert.Equal(t, http.StatusInternalServerError, code)
+	assert.Equal(t, "{\"code\":0,\"details\":\"can't generate QR\",\"error\":\"content too long to encode\"}\n", body)
+
+	// wrong request
+	body, code = get(t, ts.URL+"/api/v1/qr/telegram?url=nonsense")
+	assert.Equal(t, http.StatusBadRequest, code)
+	assert.Equal(t, "{\"code\":0,\"details\":\"text parameter should start with https://t.me/\",\"error\":\"wrong parameter\"}\n", body)
+
+	// correct request
+	r, err := http.Get(ts.URL + "/api/v1/qr/telegram?url=https://t.me/BotFather")
+	require.NoError(t, err)
+	bdy, err := io.ReadAll(r.Body)
+	require.NoError(t, err)
+	require.NoError(t, r.Body.Close())
+	require.NotEmpty(t, bdy)
+	assert.Equal(t, "image/png", r.Header.Get("Content-Type"))
+	assert.Equal(t, http.StatusOK, r.StatusCode)
+
+	// compare the image
+	fh, err := os.Open("testdata/qr_test.png")
+	defer func() { assert.NoError(t, fh.Close()) }()
+	assert.NoError(t, err)
+	img, err := io.ReadAll(fh)
+	assert.NoError(t, err)
+	assert.Equal(t, img, bdy)
+}
+
 func TestRest_Info(t *testing.T) {
 	ts, srv, teardown := startupT(t)
 	defer teardown()
@@ -637,7 +1006,7 @@ func TestRest_Info(t *testing.T) {
 	require.NoError(t, err)
 
 	body, code := get(t, ts.URL+"/api/v1/info?site=remark42&url=https://radio-t.com/blah1")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 
 	info := store.PostInfo{}
 	err = json.Unmarshal([]byte(body), &info)
@@ -647,9 +1016,9 @@ func TestRest_Info(t *testing.T) {
 	assert.Equal(t, exp, info)
 
 	_, code = get(t, ts.URL+"/api/v1/info?site=remark42&url=https://radio-t.com/blah-no")
-	assert.Equal(t, 400, code)
+	assert.Equal(t, http.StatusBadRequest, code)
 	_, code = get(t, ts.URL+"/api/v1/info?site=remark42-no&url=https://radio-t.com/blah-no")
-	assert.Equal(t, 400, code)
+	assert.Equal(t, http.StatusBadRequest, code)
 }
 
 func TestRest_Robots(t *testing.T) {
@@ -657,9 +1026,195 @@ func TestRest_Robots(t *testing.T) {
 	defer teardown()
 
 	body, code := get(t, ts.URL+"/robots.txt")
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	assert.Equal(t, "User-agent: *\nDisallow: /auth/\nDisallow: /api/\nAllow: /api/v1/find\n"+
 		"Allow: /api/v1/last\nAllow: /api/v1/id\nAllow: /api/v1/count\nAllow: /api/v1/counts\n"+
 		"Allow: /api/v1/list\nAllow: /api/v1/config\nAllow: /api/v1/user\nAllow: /api/v1/img\n"+
 		"Allow: /api/v1/avatar\nAllow: /api/v1/picture\n", body)
 }
+
+// TestRest_LoadPictureRejectsPathTraversal reproduces the unauthenticated path-traversal
+// vulnerability in GET /api/v1/picture/{user}/{id}. Before the fix, the handler concatenated
+// the URL params verbatim into a filesystem path via path.Join, so a request like
+// `/api/v1/picture/../remark.db` would resolve to `<base>/../remark.db`, escaping the image
+// directory. Even when the file did not exist (default Partitions=100 mitigates direct hits),
+// the FS error message leaked the constructed internal path back to the unauthenticated caller.
+func TestRest_LoadPictureRejectsPathTraversal(t *testing.T) {
+	ts, _, teardown := startupT(t)
+	defer teardown()
+
+	cases := []struct {
+		name string
+		path string
+	}{
+		{name: "dotdot in user segment", path: "/api/v1/picture/../remark.db"},
+		{name: "dotdot in id segment", path: "/api/v1/picture/dev_user/..%2Fremark.db"},
+		{name: "encoded dotdot in user segment", path: "/api/v1/picture/%2E%2E/remark.db"},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			req, err := http.NewRequest(http.MethodGet, ts.URL+c.path, http.NoBody)
+			require.NoError(t, err)
+			resp, err := http.DefaultClient.Do(req)
+			require.NoError(t, err)
+			defer func() { _ = resp.Body.Close() }()
+
+			assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
+			body, err := io.ReadAll(resp.Body)
+			require.NoError(t, err)
+			s := string(body)
+			assert.NotContains(t, s, "..", "error body must not echo traversal marker")
+			assert.NotContains(t, s, "remark.db", "error body must not echo attacker-supplied filename")
+			assert.NotContains(t, s, "no such file", "error body must not leak filesystem state")
+			assert.NotContains(t, s, "/var/", "error body must not leak internal filesystem path")
+		})
+	}
+}
+
+// TestRest_LoadPictureRejectsControlCharsInSegment makes sure a CRLF / tab / NUL
+// in the URL segment is rejected by safePictureSegment. Without the rejection
+// the [WARN] log line constructed from %q-formatted segments would still be
+// safe (Go's %q escapes control chars), but a future log change to %s would
+// turn this into log forgery — and no legitimate picture id ever needs control
+// characters, so the right place to slam the door is in the validator.
+func TestRest_LoadPictureRejectsControlCharsInSegment(t *testing.T) {
+	ts, _, teardown := startupT(t)
+	defer teardown()
+
+	cases := []struct {
+		name string
+		path string
+	}{
+		{name: "lf in user segment", path: "/api/v1/picture/dev%0Auser/abc.png"},
+		{name: "cr in user segment", path: "/api/v1/picture/dev%0Duser/abc.png"},
+		{name: "tab in user segment", path: "/api/v1/picture/dev%09user/abc.png"},
+		{name: "lf in id segment", path: "/api/v1/picture/dev_user/abc%0A.png"},
+		{name: "nul in id segment", path: "/api/v1/picture/dev_user/abc%00.png"},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			req, err := http.NewRequest(http.MethodGet, ts.URL+c.path, http.NoBody)
+			require.NoError(t, err)
+			resp, err := http.DefaultClient.Do(req)
+			require.NoError(t, err)
+			defer func() { _ = resp.Body.Close() }()
+
+			assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
+			body, err := io.ReadAll(resp.Body)
+			require.NoError(t, err)
+			s := string(body)
+			assert.Contains(t, s, "invalid picture id", "must reject as invalid input, not fall through to storage")
+			assert.NotContains(t, s, "no such file", "must not reach the filesystem")
+		})
+	}
+}
+
+// TestRest_LoadPictureDefenseHeaders saves a real PNG via the standard upload handler
+// and asserts that GET /api/v1/picture/{user}/{id} carries the layered defense headers
+// (strict CSP, nosniff, Content-Disposition with filename) and that the strict ETag
+// matcher does not 304 on a substring-of-the-real-etag (the pre-fix matcher would).
+func TestRest_LoadPictureDefenseHeaders(t *testing.T) {
+	ts, _, teardown := startupT(t)
+	defer teardown()
+
+	// upload a real PNG via /api/v1/picture
+	bodyBuf := &bytes.Buffer{}
+	bodyWriter := multipart.NewWriter(bodyBuf)
+	fileWriter, err := bodyWriter.CreateFormFile("file", "picture.png")
+	require.NoError(t, err)
+	_, err = io.Copy(fileWriter, gopherPNG())
+	require.NoError(t, err)
+	contentType := bodyWriter.FormDataContentType()
+	require.NoError(t, bodyWriter.Close())
+
+	client := http.Client{}
+	defer client.CloseIdleConnections()
+	req, err := http.NewRequest(http.MethodPost, fmt.Sprintf("%s/api/v1/picture?site=remark42", ts.URL), bodyBuf)
+	require.NoError(t, err)
+	req.Header.Add("Content-Type", contentType)
+	req.Header.Add("X-JWT", devToken)
+	resp, err := client.Do(req)
+	require.NoError(t, err)
+	require.Equal(t, http.StatusOK, resp.StatusCode)
+	body, err := io.ReadAll(resp.Body)
+	require.NoError(t, err)
+	require.NoError(t, resp.Body.Close())
+	m := map[string]string{}
+	require.NoError(t, json.Unmarshal(body, &m))
+	require.NotEmpty(t, m["id"])
+
+	// fetch the picture and assert defense headers
+	resp, err = http.Get(fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, m["id"]))
+	require.NoError(t, err)
+	defer resp.Body.Close()
+	require.Equal(t, http.StatusOK, resp.StatusCode)
+	assert.Equal(t, "default-src 'none'; sandbox; frame-ancestors 'none'",
+		resp.Header.Get("Content-Security-Policy"))
+	assert.Equal(t, "nosniff", resp.Header.Get("X-Content-Type-Options"))
+	assert.Equal(t, `inline; filename="image"`, resp.Header.Get("Content-Disposition"))
+	assert.Equal(t, "image/png", resp.Header.Get("Content-Type"))
+	realEtag := resp.Header.Get("Etag")
+	require.NotEmpty(t, realEtag)
+
+	// strict matcher: an If-None-Match value that CONTAINS the real etag as a substring
+	// but is not equal to it must NOT trigger 304. The pre-fix matcher used
+	// strings.Contains(header, etag) and would have returned true here.
+	require.True(t, len(realEtag) > 4)
+	substringMatch := "prefix-" + realEtag + "-suffix"
+	req2, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, m["id"]), http.NoBody)
+	require.NoError(t, err)
+	req2.Header.Set("If-None-Match", substringMatch)
+	resp2, err := client.Do(req2)
+	require.NoError(t, err)
+	defer resp2.Body.Close()
+	assert.Equal(t, http.StatusOK, resp2.StatusCode,
+		"strict etag matcher must NOT 304 when real etag appears only as a substring of If-None-Match; got %q vs real %q", substringMatch, realEtag)
+
+	// sanity: the exact real etag DOES validate
+	req3, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/api/v1/picture/%s", ts.URL, m["id"]), http.NoBody)
+	require.NoError(t, err)
+	req3.Header.Set("If-None-Match", realEtag)
+	resp3, err := client.Do(req3)
+	require.NoError(t, err)
+	defer resp3.Body.Close()
+	assert.Equal(t, http.StatusNotModified, resp3.StatusCode, "exact etag must round-trip as 304")
+}
+
+// TestRest_LoadPictureRejectsNonImage proves the /picture/ handler rejects bytes that
+// don't sniff as a real image — even when retrieved successfully from the image store.
+// Uses a StoreMock so we can return arbitrary attacker bytes for a valid-looking id.
+func TestRest_LoadPictureRejectsNonImage(t *testing.T) {
+	htmlBody := []byte("<html><body><script>alert(document.domain)</script></body></html>")
+
+	imageStore := image.StoreMock{LoadFunc: func(string) ([]byte, error) {
+		return htmlBody, nil
+	}}
+	// minimal public struct on purpose: the reject path only exercises imageService.Load
+	// (other fields like dataService, cache, commentFormatter are not touched here).
+	p := &public{imageService: image.NewService(&imageStore, image.ServiceParams{})}
+
+	router := chi.NewRouter()
+	router.Get("/api/v1/picture/{user}/{id}", p.loadPictureCtrl)
+	ts := httptest.NewServer(router)
+	defer ts.Close()
+
+	resp, err := http.Get(ts.URL + "/api/v1/picture/dev_user/abc.png")
+	require.NoError(t, err)
+	body, err := io.ReadAll(resp.Body)
+	require.NoError(t, err)
+	require.NoError(t, resp.Body.Close())
+
+	assert.Equal(t, http.StatusUnsupportedMediaType, resp.StatusCode,
+		"non-image bytes must be rejected as 415")
+	assert.False(t, strings.HasPrefix(resp.Header.Get("Content-Type"), "text/html"),
+		"reject response must not be text/html; got %q", resp.Header.Get("Content-Type"))
+	assert.NotContains(t, string(body), "<script>",
+		"attacker payload must not be echoed back")
+	assert.Equal(t, "no-store", resp.Header.Get("Cache-Control"),
+		"rejection path must not be cacheable")
+	// defense headers still present on the reject path
+	assert.Equal(t, "default-src 'none'; sandbox; frame-ancestors 'none'",
+		resp.Header.Get("Content-Security-Policy"))
+	assert.Equal(t, "nosniff", resp.Header.Get("X-Content-Type-Options"))
+	assert.Equal(t, `inline; filename="image"`, resp.Header.Get("Content-Disposition"))
+}

backend/app/rest/api/rest_test.go

@@ -4,7 +4,6 @@ import (
 	"bytes"
 	"crypto/tls"
 	"encoding/json"
-	"errors"
 	"fmt"
 	"io"
 	"math/rand"
@@ -17,10 +16,11 @@ import (
 	"testing"
 	"time"
 
-	"github.com/go-pkgz/auth"
+	"github.com/go-pkgz/auth/v2"
-	"github.com/go-pkgz/auth/avatar"
+	"github.com/go-pkgz/auth/v2/avatar"
-	"github.com/go-pkgz/auth/token"
+	"github.com/go-pkgz/auth/v2/provider"
-	cache "github.com/go-pkgz/lcw"
+	"github.com/go-pkgz/auth/v2/token"
+	cache "github.com/go-pkgz/lcw/v2"
 	R "github.com/go-pkgz/rest"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
@@ -38,11 +38,18 @@ import (
 	"github.com/umputun/remark42/backend/app/store/service"
 )
 
-var devToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6ImRldmVsb3BlciBvbmUiLCJpZCI6ImRldiIsInBpY3R1cmUiOiJodHRwOi8vZXhhbXBsZS5jb20vcGljLnBuZyIsImlwIjoiMTI3LjAuMC4xIiwiZW1haWwiOiJtZUBleGFtcGxlLmNvbSJ9fQ.aKUAXiZxXypgV7m1wEOgUcyPOvUDXHDi3A06YWKbcLg`
+// To generate a token, enter one of the tokens here into https://jwt.io, change the secret to one you're using in your test
+// ("secret" in case of startupT), and alter the fields you want to be changed.
+
+var devToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6ImRldmVsb3BlciBvbmUiLCJpZCI6InByb3ZpZGVyMV9kZXYiLCJwaWN0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL3BpYy5wbmciLCJpcCI6IjEyNy4wLjAuMSIsImVtYWlsIjoibWVAZXhhbXBsZS5jb20ifX0.dirTS_ahSF6375sdO2iodm2K2UmRTzQNQMFiHuTQCVs`
+
+var dev2Token = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6ImRldmVsb3BlciBvbmUiLCJpZCI6InByb3ZpZGVyMV9kZXYyIiwicGljdHVyZSI6Imh0dHA6Ly9leGFtcGxlLmNvbS9waWMucG5nIiwiaXAiOiIxMjcuMC4wLjEiLCJlbWFpbCI6Im1lQGV4YW1wbGUuY29tIn19.qsR_PupfjIq7uw0eAuyGV8nsUoMx9v541c9olnRInRQ`
 
 var anonToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6ImFub255bW91cyB0ZXN0IHVzZXIiLCJpZCI6ImFub255bW91c190ZXN0X3VzZXIiLCJwaWN0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL3BpYy5wbmciLCJpcCI6IjEyNy4wLjAuMSIsImVtYWlsIjoiYW5vbkBleGFtcGxlLmNvbSJ9fQ.gAae2WMxZNZE5ebVboptPEyQ7Nk6EQxciNnGJ_mPOuU`
 
-var devTokenBadAud = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0Ml9iYWQiLCJleHAiOjM3ODkxOTE4MjIsImp0aSI6InJhbmRvbSBpZCIsImlzcyI6InJlbWFyazQyIiwibmJmIjoxNTIxODg0MjIyLCJ1c2VyIjp7Im5hbWUiOiJkZXZlbG9wZXIgb25lIiwiaWQiOiJkZXYiLCJwaWN0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL3BpYy5wbmciLCJpcCI6IjEyNy4wLjAuMSIsImVtYWlsIjoibWVAZXhhbXBsZS5jb20ifX0.FuTTocVtcxr4VjpfIICvU2yOb3su28VkDzj94H9Q3xY`
+var emailUserToken = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6Mzc4OTE5MTgyMiwianRpIjoicmFuZG9tIGlkIiwiaXNzIjoicmVtYXJrNDIiLCJuYmYiOjE1MjE4ODQyMjIsInVzZXIiOnsibmFtZSI6Imdvb2RAZXhhbXBsZS5jb20gdGVzdCB1c2VyIiwiaWQiOiJlbWFpbF9mNWRmZTlkMmU2YmQ3NWZjNzRlYTVmYWJmMjczYjQ1YjViYWViMTk1IiwicGljdHVyZSI6Imh0dHA6Ly9leGFtcGxlLmNvbS9waWMucG5nIiwiaXAiOiIxMjcuMC4wLjEiLCJlbWFpbCI6Imdvb2RAZXhhbXBsZS5jb20ifX0.vH2HN1JpuXL8okTJq1A-zGHQ-l2ILcwxvDDEmu2zwks`
+
+var devTokenBadAud = `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZW1hcms0Ml9iYWQiLCJleHAiOjM3ODkxOTE4MjIsImp0aSI6InJhbmRvbSBpZCIsImlzcyI6InJlbWFyazQyIiwibmJmIjoxNTIxODg0MjIyLCJ1c2VyIjp7Im5hbWUiOiJkZXZlbG9wZXIgb25lIiwiaWQiOiJwcm92aWRlcjFfZGV2IiwicGljdHVyZSI6Imh0dHA6Ly9leGFtcGxlLmNvbS9waWMucG5nIiwiaXAiOiIxMjcuMC4wLjEiLCJlbWFpbCI6Im1lQGV4YW1wbGUuY29tIn19.X-lvnHvBz6VfEbVV4f-bjcZuLY5pYtvEansk_TQMrX8`
 
 var adminUmputunToken = `eyJhbGciOiJIUzI1NiJ9.eyJhdWQiOiJyZW1hcms0MiIsImV4cCI6MTk1NDU5Nzk4MCwianRpIjoiOTdhMmUwYWM0ZGM3ZDVmNjkyNmQ1ZTg2MjBhY2VmOWE0MGMwIiwiaWF0IjoxNDU0NTk3NjgwLCJpc3MiOiJyZW1hcms0MiIsInVzZXIiOnsibmFtZSI6IlVtcHV0dW4iLCJpZCI6ImdpdGh1Yl9lZjBmNzA2YTciLCJwaWN0dXJlIjoiaHR0cHM6Ly9yZW1hcms0Mi5yYWRpby10LmNvbS9hcGkvdjEvYXZhdGFyL2NiNDJmZjQ5M2FkZTY5NmQ4OGEzYTU5MGYxMzZhZTllMzRkZTdjMWIuaW1hZ2UiLCJhdHRycyI6eyJhZG1pbiI6dHJ1ZSwiYmxvY2tlZCI6ZmFsc2V9fX0.dZiOjWHguo9f42XCMooMcv4EmYFzifl_-LEvPZHCtks`
 
@@ -56,29 +63,11 @@ func TestRest_FileServer(t *testing.T) {
 	assert.NoError(t, err)
 
 	body, code := get(t, ts.URL+"/web/"+testHTMLName)
-	assert.Equal(t, 200, code)
+	assert.Equal(t, http.StatusOK, code)
 	assert.Equal(t, "some html", body)
 	_ = os.Remove(testHTMLFile)
 }
 
-func TestRest_GetStarted(t *testing.T) {
-	ts, _, teardown := startupT(t)
-	defer teardown()
-
-	getStartedHTML := os.TempDir() + "/getstarted.html"
-	err := os.WriteFile(getStartedHTML, []byte("some html blah"), 0o700)
-	assert.NoError(t, err)
-
-	body, code := get(t, ts.URL+"/index.html")
-	assert.Equal(t, 200, code)
-	assert.Equal(t, "some html blah", body)
-
-	_ = os.Remove(getStartedHTML)
-	_, code = get(t, ts.URL+"/index.html")
-	assert.Equal(t, 404, code)
-
-}
-
 func TestRest_Shutdown(t *testing.T) {
 	srv := Rest{Authenticator: &auth.Service{}, ImageProxy: &proxy.Image{}}
 	done := make(chan bool)
@@ -139,7 +128,7 @@ func TestRest_RunStaticSSLMode(t *testing.T) {
 
 	client := http.Client{
 		// prevent http redirect
-		CheckRedirect: func(req *http.Request, via []*http.Request) error {
+		CheckRedirect: func(*http.Request, []*http.Request) error {
 			return http.ErrUseLastResponse
 		},
 
@@ -148,17 +137,18 @@ func TestRest_RunStaticSSLMode(t *testing.T) {
 			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
 		},
 	}
+	defer client.CloseIdleConnections()
 
 	resp, err := client.Get(fmt.Sprintf("http://localhost:%d/blah?param=1", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 307, resp.StatusCode)
+	assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
 	assert.Equal(t, fmt.Sprintf("https://localhost:%d/blah?param=1", sslPort), resp.Header.Get("Location"))
 
 	resp, err = client.Get(fmt.Sprintf("https://localhost:%d/ping", sslPort))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 200, resp.StatusCode)
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
 	body, err := io.ReadAll(resp.Body)
 	assert.NoError(t, err)
 	assert.Equal(t, "pong", string(body))
@@ -188,23 +178,23 @@ func TestRest_RunAutocertModeHTTPOnly(t *testing.T) {
 
 	client := http.Client{
 		// prevent http redirect
-		CheckRedirect: func(req *http.Request, via []*http.Request) error {
+		CheckRedirect: func(*http.Request, []*http.Request) error {
 			return http.ErrUseLastResponse
 		},
 	}
+	defer client.CloseIdleConnections()
 
 	resp, err := client.Get(fmt.Sprintf("http://localhost:%d/blah?param=1", port))
 	require.NoError(t, err)
 	defer resp.Body.Close()
-	assert.Equal(t, 307, resp.StatusCode)
+	assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)
 	assert.Equal(t, fmt.Sprintf("https://localhost:%d/blah?param=1", sslPort), resp.Header.Get("Location"))
 
 	srv.Shutdown()
 }
 
 func TestRest_rejectAnonUser(t *testing.T) {
-
+	ts := httptest.NewServer(fakeAuth(rejectAnonUser(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
-	ts := httptest.NewServer(fakeAuth(rejectAnonUser(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		fmt.Fprintln(w, "Hello")
 	}))))
 	defer ts.Close()
@@ -237,7 +227,6 @@ func Test_URLKey(t *testing.T) {
 	}
 
 	for i, tt := range tbl {
-		tt := tt
 		t.Run(strconv.Itoa(i), func(t *testing.T) {
 			r, err := http.NewRequest("GET", tt.url, http.NoBody)
 			require.NoError(t, err)
@@ -247,7 +236,6 @@ func Test_URLKey(t *testing.T) {
 			assert.Equal(t, tt.key, URLKey(r))
 		})
 	}
-
 }
 
 func Test_URLKeyWithUser(t *testing.T) {
@@ -263,7 +251,6 @@ func Test_URLKeyWithUser(t *testing.T) {
 	}
 
 	for i, tt := range tbl {
-		tt := tt
 		t.Run(strconv.Itoa(i), func(t *testing.T) {
 			r, err := http.NewRequest("GET", tt.url, http.NoBody)
 			require.NoError(t, err)
@@ -273,7 +260,6 @@ func Test_URLKeyWithUser(t *testing.T) {
 			assert.Equal(t, tt.key, URLKeyWithUser(r))
 		})
 	}
-
 }
 
 func TestRest_parseError(t *testing.T) {
@@ -281,17 +267,16 @@ func TestRest_parseError(t *testing.T) {
 		err error
 		res int
 	}{
-		{errors.New("can not vote for his own comment"), rest.ErrVoteSelf},
+		{fmt.Errorf("can not vote for his own comment"), rest.ErrVoteSelf},
-		{errors.New("already voted for"), rest.ErrVoteDbl},
+		{fmt.Errorf("already voted for"), rest.ErrVoteDbl},
-		{errors.New("maximum number of votes exceeded for comment"), rest.ErrVoteMax},
+		{fmt.Errorf("maximum number of votes exceeded for comment"), rest.ErrVoteMax},
-		{errors.New("minimal score reached for comment"), rest.ErrVoteMinScore},
+		{fmt.Errorf("minimal score reached for comment"), rest.ErrVoteMinScore},
-		{errors.New("too late to edit"), rest.ErrCommentEditExpired},
+		{fmt.Errorf("too late to edit"), rest.ErrCommentEditExpired},
-		{errors.New("parent comment with reply can't be edited"), rest.ErrCommentEditChanged},
+		{fmt.Errorf("parent comment with reply can't be edited"), rest.ErrCommentEditChanged},
-		{errors.New("blah blah"), rest.ErrInternal},
+		{fmt.Errorf("blah blah"), rest.ErrInternal},
 	}
 
 	for n, tt := range tbl {
-		tt := tt
 		t.Run(strconv.Itoa(n), func(t *testing.T) {
 			res := parseError(tt.err, rest.ErrInternal)
 			assert.Equal(t, tt.res, res)
@@ -300,7 +285,6 @@ func TestRest_parseError(t *testing.T) {
 }
 
 func TestRest_cacheControl(t *testing.T) {
-
 	tbl := []struct {
 		url     string
 		version string
@@ -315,12 +299,11 @@ func TestRest_cacheControl(t *testing.T) {
 	}
 
 	for i, tt := range tbl {
-		tt := tt
 		t.Run(strconv.Itoa(i), func(t *testing.T) {
-			req := httptest.NewRequest("GET", tt.url, nil)
+			req := httptest.NewRequest("GET", tt.url, http.NoBody)
 			w := httptest.NewRecorder()
 
-			h := cacheControl(tt.exp, tt.version)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
+			h := cacheControl(tt.exp, tt.version)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
 			h.ServeHTTP(w, req)
 			resp := w.Result()
 			assert.Equal(t, http.StatusOK, resp.StatusCode)
@@ -328,44 +311,104 @@ func TestRest_cacheControl(t *testing.T) {
 			t.Logf("%+v", resp.Header)
 			assert.Equal(t, `"`+tt.etag+`"`, resp.Header.Get("Etag"))
 			assert.Equal(t, `max-age=`+strconv.Itoa(int(tt.exp.Seconds()))+", no-cache", resp.Header.Get("Cache-Control"))
-
 		})
 	}
-
 }
 
 func TestRest_frameAncestors(t *testing.T) {
+	ts, _, teardown := startupT(t, func(o *Rest) {
+		o.AllowedAncestors = []string{"'self'", "https://example.com"}
+	})
 
-	tbl := []struct {
+	// test case with frame-ancestors
-		hosts  []string
+	client := http.Client{}
-		header string
+	resp, err := client.Get(ts.URL + "/web/index.html")
-	}{
+	require.NoError(t, err)
-		{[]string{"http://example.com"}, "frame-ancestors http://example.com;"},
+	defer resp.Body.Close()
-		{[]string{}, ""},
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
-		{[]string{"http://example.com", "http://example2.com"}, "frame-ancestors http://example.com http://example2.com;"},
+	assert.Contains(t, resp.Header.Get("Content-Security-Policy"), "frame-ancestors 'self' https://example.com;")
-	}
+	teardown()
 
-	for i, tt := range tbl {
+	// test case without frame-ancestors
-		tt := tt
+	ts, _, teardown = startupT(t, func(srv *Rest) {
-		t.Run(strconv.Itoa(i), func(t *testing.T) {
+		srv.AllowedAncestors = []string{}
-			req := httptest.NewRequest("GET", "http://example.com", nil)
+	})
-			w := httptest.NewRecorder()
+	defer teardown()
+	resp, err = client.Get(ts.URL + "/web/index.html")
+	require.NoError(t, err)
+	defer resp.Body.Close()
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	assert.Contains(t, resp.Header.Get("Content-Security-Policy"), "frame-ancestors *;")
+}
 
-			h := frameAncestors(tt.hosts)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
+// TestRest_apiCSP locks in that /api/v1/* responses get a strict default-src 'none'
-			h.ServeHTTP(w, req)
+// override regardless of what the global CSP allows. The widget HTML pages
-			resp := w.Result()
+// (/web/*.html) still get the global CSP (with 'unsafe-inline' for bootstrap),
-			assert.Equal(t, http.StatusOK, resp.StatusCode)
+// so the test asserts the two policies diverge across origins.
-			assert.NoError(t, resp.Body.Close())
+func TestRest_apiCSP(t *testing.T) {
-			t.Logf("%+v", resp.Header)
+	ts, _, teardown := startupT(t)
-			assert.Equal(t, tt.header, resp.Header.Get("Content-Security-Policy"))
+	defer teardown()
+	client := http.Client{}
 
-		})
+	// JSON API endpoint — must carry the strict policy
-	}
+	resp, err := client.Get(ts.URL + "/api/v1/config")
+	require.NoError(t, err)
+	defer resp.Body.Close()
+	csp := resp.Header.Get("Content-Security-Policy")
+	assert.Contains(t, csp, "default-src 'none'",
+		"API responses must override the global CSP with default-src 'none'; got %q", csp)
+	assert.Contains(t, csp, "sandbox", "API CSP must include sandbox; got %q", csp)
+	assert.NotContains(t, csp, "'unsafe-inline'",
+		"API CSP must not allow inline scripts/styles; got %q", csp)
 
+	// RSS/XML endpoint — same strict policy, and the XML response itself must still be served
+	respRSS, err := client.Get(ts.URL + "/api/v1/rss/site?site=remark42")
+	require.NoError(t, err)
+	defer respRSS.Body.Close()
+	assert.Equal(t, http.StatusOK, respRSS.StatusCode, "RSS must still respond OK under strict CSP")
+	cspRSS := respRSS.Header.Get("Content-Security-Policy")
+	assert.Contains(t, cspRSS, "default-src 'none'", "RSS responses must carry the strict API CSP")
+	assert.Contains(t, cspRSS, "sandbox", "RSS CSP must include sandbox")
+
+	// widget HTML — must keep the global CSP (unchanged, lax to support inline bootstrap)
+	resp2, err := client.Get(ts.URL + "/web/index.html")
+	require.NoError(t, err)
+	defer resp2.Body.Close()
+	csp2 := resp2.Header.Get("Content-Security-Policy")
+	assert.Contains(t, csp2, "'unsafe-inline'",
+		"widget HTML CSP must keep unsafe-inline for bootstrap; got %q", csp2)
+}
+
+// check CSP, img-src should be 'self' with proxy enabled and * without it
+func TestRest_securityHeaders(t *testing.T) {
+	ts, _, teardown := startupT(t)
+
+	// with proxy disabled
+	client := http.Client{}
+	resp, err := client.Get(ts.URL + "/web/index.html")
+	require.NoError(t, err)
+	defer resp.Body.Close()
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	assert.Contains(t, resp.Header.Get("Content-Security-Policy"), "img-src *;")
+	assert.Equal(t, "nosniff", resp.Header.Get("X-Content-Type-Options"))
+	assert.Equal(t, "strict-origin-when-cross-origin", resp.Header.Get("Referrer-Policy"))
+	teardown()
+
+	// check CSP with proxy enabled
+	ts, _, teardown = startupT(t, func(srv *Rest) {
+		srv.ExternalImageProxy = true
+	})
+	defer teardown()
+	resp, err = client.Get(ts.URL + "/web/index.html")
+	require.NoError(t, err)
+	defer resp.Body.Close()
+	assert.Equal(t, http.StatusOK, resp.StatusCode)
+	assert.Contains(t, resp.Header.Get("Content-Security-Policy"), "img-src 'self';")
+	assert.Equal(t, "nosniff", resp.Header.Get("X-Content-Type-Options"))
+	assert.Equal(t, "strict-origin-when-cross-origin", resp.Header.Get("Referrer-Policy"))
 }
 
 func TestRest_subscribersOnly(t *testing.T) {
-
 	paidSubUser := &token.User{}
 	paidSubUser.SetPaidSub(true)
 
@@ -383,14 +426,45 @@ func TestRest_subscribersOnly(t *testing.T) {
 	}
 
 	for i, tt := range tbl {
-		tt := tt
 		t.Run(strconv.Itoa(i), func(t *testing.T) {
-			req := httptest.NewRequest("GET", "http://example.com", nil)
+			req := httptest.NewRequest("GET", "http://example.com", http.NoBody)
 			if tt.setUser {
 				req = token.SetUserInfo(req, tt.user)
 			}
 			w := httptest.NewRecorder()
-			h := subscribersOnly(tt.subsOnly)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
+			h := subscribersOnly(tt.subsOnly)(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
+			h.ServeHTTP(w, req)
+			resp := w.Result()
+			assert.Equal(t, tt.status, resp.StatusCode)
+			assert.NoError(t, resp.Body.Close())
+		})
+	}
+}
+
+func Test_validEmailAuth(t *testing.T) {
+	tbl := []struct {
+		req    string
+		status int
+	}{
+		{"/auth/email/login?site=remark42&address=umputun%example.com&user=someone", http.StatusOK},
+		{"/auth/email/login?site=site-with-dash_and_underscore-and.dot&address=umputun%example.com&user=someone", http.StatusOK},
+		{"/auth/email/login?site=remark42&address=umputun%example.com&user=someone+blah", http.StatusOK},
+		{"/auth/email/login?site=remark42&address=umputun%example.com&user=Евгений+Умпутун", http.StatusOK},
+		{"/auth/email/login?site=remark42&address=umputun%example.com&user=12", http.StatusForbidden},
+		{"/auth/email/login?site=remark42&address=umputun%example.com&user=..blah+blah", http.StatusForbidden},
+		{"/auth/email/login?site=remark42&address=umputun%example.com&user=someonelooong+loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong", http.StatusForbidden},
+		{"/auth/twitter/login?site=remark42&address=umputun%example.com&user=..blah+blah", http.StatusOK},
+		{"/auth/email/login?site=remark42&address=umputun%example.com", http.StatusOK},
+		{"/auth/email/login?site=remark42&address=umputun+example.com&user=someone", http.StatusForbidden},
+		{"/auth/email/login?site=bad!site&address=umputun%example.com&user=someone", http.StatusForbidden},
+		{"/auth/email/login?site=loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongsite&address=umputun%example.com&user=someone", http.StatusForbidden},
+	}
+
+	for i, tt := range tbl {
+		t.Run(strconv.Itoa(i), func(t *testing.T) {
+			req := httptest.NewRequest("GET", "http://example.com"+tt.req, http.NoBody)
+			w := httptest.NewRecorder()
+			h := validEmailAuth()(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
 			h.ServeHTTP(w, req)
 			resp := w.Result()
 			assert.Equal(t, tt.status, resp.StatusCode)
@@ -401,7 +475,7 @@ func TestRest_subscribersOnly(t *testing.T) {
 
 // randomPath pick a file or folder name which is not in use for sure
 func randomPath(tempDir, basename, suffix string) (string, error) {
-	for i := 0; i < 10; i++ {
+	for range 10 {
 		fname := fmt.Sprintf("/%s/%s-%d%s", tempDir, basename, rand.Int31(), suffix)
 		fmt.Printf("fname %q", fname)
 		_, err := os.Stat(fname)
@@ -409,10 +483,12 @@ func randomPath(tempDir, basename, suffix string) (string, error) {
 			return fname, nil
 		}
 	}
-	return "", errors.New("cannot create temp file")
+	return "", fmt.Errorf("cannot create temp file in %s", tempDir)
 }
 
-func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
+// startupT runs fully configured testing server
+// srvHook is an optional func to set some Rest param after the creation but prior to Run
+func startupT(t *testing.T, srvHook ...func(srv *Rest)) (ts *httptest.Server, srv *Rest, teardown func()) {
 	tmp := os.TempDir()
 	testDB, err := randomPath(tmp, "test-remark", ".db")
 	require.NoError(t, err)
@@ -423,7 +499,7 @@ func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
 	b, err := engine.NewBoltDB(bolt.Options{}, engine.BoltSite{FileName: testDB, SiteID: "remark42"})
 	require.NoError(t, err)
 
-	memCache := cache.NewScache(cache.NewNopCache())
+	memCache := cache.NewScache[[]byte](cache.NewNopCache[[]byte]())
 
 	astore := adminstore.NewStaticStore("123456", []string{"remark42"}, []string{"a1", "a2"}, "admin@remark-42.com")
 	restrictedWordsMatcher := service.NewRestrictedWordsMatcher(service.StaticRestrictedWordsLister{Words: []string{"duck"}})
@@ -443,7 +519,7 @@ func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
 		DataService: dataStore,
 		Authenticator: auth.NewService(auth.Opts{
 			AdminPasswd:  "password",
-			SecretReader: token.SecretFunc(func(aud string) (string, error) { return "secret", nil }),
+			SecretReader: token.SecretFunc(func(string) (string, error) { return "secret", nil }),
 			AvatarStore:  avatar.NewLocalFS(tmp + "/ava-remark42"),
 		}),
 		Cache:     memCache,
@@ -472,12 +548,26 @@ func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
 			Cache:             memCache,
 			KeyStore:          astore,
 		},
-		NotifyService: notify.NopService,
+		NotifyService:    notify.NopService,
-		EmojiEnabled:  true,
+		EmojiEnabled:     true,
+		openRouteLimiter: 100,
 	}
 	srv.ScoreThresholds.Low, srv.ScoreThresholds.Critical = -5, -10
 
-	ts = httptest.NewServer(srv.routes())
+	// add some providers. Needed because we don't allow users with unlisted providers to authenticate
+	providers := []string{"provider1", "anonymous", "github", "email"}
+	for _, p := range providers {
+		srv.Authenticator.AddDirectProvider(p, provider.CredCheckerFunc(func(_, _ string) (ok bool, err error) {
+			return true, nil
+		}))
+	}
+
+	for _, h := range srvHook {
+		h(srv)
+	}
+
+	routes := srv.routes()
+	ts = httptest.NewServer(routes)
 
 	teardown = func() {
 		ts.Close()
@@ -515,6 +605,7 @@ func get(t *testing.T, url string) (response string, statusCode int) {
 
 func sendReq(_ *testing.T, r *http.Request, tkn string) (*http.Response, error) {
 	client := http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	if tkn != "" {
 		r.Header.Set("X-JWT", tkn)
 	}
@@ -523,6 +614,7 @@ func sendReq(_ *testing.T, r *http.Request, tkn string) (*http.Response, error)
 
 func getWithDevAuth(t *testing.T, url string) (body string, code int) {
 	client := &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("GET", url, http.NoBody)
 	require.NoError(t, err)
 	req.Header.Add("X-JWT", devToken)
@@ -534,8 +626,23 @@ func getWithDevAuth(t *testing.T, url string) (body string, code int) {
 	return string(b), r.StatusCode
 }
 
+func getWithDev2Auth(t *testing.T, url string) (body string, code int) {
+	client := &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
+	req, err := http.NewRequest("GET", url, http.NoBody)
+	require.NoError(t, err)
+	req.Header.Add("X-JWT", dev2Token)
+	r, err := client.Do(req)
+	require.NoError(t, err)
+	b, err := io.ReadAll(r.Body)
+	assert.NoError(t, err)
+	require.NoError(t, r.Body.Close())
+	return string(b), r.StatusCode
+}
+
 func getWithAdminAuth(t *testing.T, url string) (response string, statusCode int) {
 	client := &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("GET", url, http.NoBody)
 	require.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
@@ -548,18 +655,24 @@ func getWithAdminAuth(t *testing.T, url string) (response string, statusCode int
 }
 func post(t *testing.T, url, body string) (*http.Response, error) {
 	client := &http.Client{Timeout: 5 * time.Second}
+	defer client.CloseIdleConnections()
 	req, err := http.NewRequest("POST", url, strings.NewReader(body))
 	assert.NoError(t, err)
 	req.SetBasicAuth("admin", "password")
 	return client.Do(req)
 }
 
-func addComment(t *testing.T, c store.Comment, ts *httptest.Server) string {
+func addCommentGetCreatedTime(t *testing.T, c store.Comment, ts *httptest.Server) (id string, created time.Time) {
 	b, err := json.Marshal(c)
 	require.NoError(t, err, "can't marshal comment %+v", c)
 
 	client := &http.Client{Timeout: 5 * time.Second}
-	req, err := http.NewRequest("POST", ts.URL+"/api/v1/comment", bytes.NewBuffer(b))
+	defer client.CloseIdleConnections()
+	postURL := ts.URL + "/api/v1/comment"
+	if c.Locator.SiteID != "" {
+		postURL += "?site=" + c.Locator.SiteID
+	}
+	req, err := http.NewRequest("POST", postURL, bytes.NewBuffer(b))
 	require.NoError(t, err)
 	req.Header.Add("X-JWT", devToken)
 	resp, err := client.Do(req)
@@ -573,23 +686,30 @@ func addComment(t *testing.T, c store.Comment, ts *httptest.Server) string {
 	err = json.Unmarshal(b, &crResp)
 	require.NoError(t, err)
 	time.Sleep(time.Nanosecond * 10)
-	return crResp["id"].(string)
+	created, err = time.Parse(time.RFC3339, crResp["time"].(string))
+	require.NoError(t, err)
+	return crResp["id"].(string), created
+}
+
+func addComment(t *testing.T, c store.Comment, ts *httptest.Server) string {
+	id, _ := addCommentGetCreatedTime(t, c, ts)
+	return id
 }
 
 func requireAdminOnly(t *testing.T, req *http.Request) {
 	resp, err := sendReq(t, req, "") // no-auth user
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 401, resp.StatusCode)
+	assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
 
 	resp, err = sendReq(t, req, devToken) // non-admin user
 	require.NoError(t, err)
 	require.NoError(t, resp.Body.Close())
-	assert.Equal(t, 403, resp.StatusCode)
+	assert.Equal(t, http.StatusForbidden, resp.StatusCode)
 }
 
 func chooseRandomUnusedPort() (port int) {
-	for i := 0; i < 10; i++ {
+	for range 10 {
 		port = 40000 + int(rand.Int31n(10000))
 		if ln, err := net.Listen("tcp", fmt.Sprintf(":%d", port)); err == nil {
 			_ = ln.Close()
@@ -601,7 +721,7 @@ func chooseRandomUnusedPort() (port int) {
 
 func waitForHTTPSServerStart(port int) {
 	// wait for up to 3 seconds for HTTPS server to start
-	for i := 0; i < 300; i++ {
+	for range 300 {
 		time.Sleep(time.Millisecond * 10)
 		conn, _ := net.DialTimeout("tcp", fmt.Sprintf("localhost:%d", port), time.Millisecond*10)
 		if conn != nil {
@@ -612,5 +732,49 @@ func waitForHTTPSServerStart(port int) {
 }
 
 func TestMain(m *testing.M) {
-	goleak.VerifyTestMain(m)
+	goleak.VerifyTestMain(
+		m,
+		// this will be fixed in https://github.com/hashicorp/golang-lru/issues/159
+		goleak.IgnoreTopFunction("github.com/hashicorp/golang-lru/v2/expirable.NewLRU[...].func1"),
+	)
+}
+
+// TestRest_matchSiteID reproduces the multi-tenant isolation gap in the matchSiteID
+// middleware. Before the fix, the check `if siteID != "" && user.SiteID != siteID`
+// silently allowed any authenticated request that omitted the ?site= query param.
+// On admin and user-mutation routes this meant the cross-site check was bypassable
+// just by dropping the parameter. The fix requires ?site= to be present and to match
+// the user's bound site.
+func TestRest_matchSiteID(t *testing.T) {
+	wrapped := matchSiteID(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+		w.WriteHeader(http.StatusOK)
+		_, _ = w.Write([]byte("ok"))
+	}))
+
+	cases := []struct {
+		name     string
+		userSite string
+		query    string
+		want     int
+	}{
+		{name: "matching site allowed", userSite: "site-a", query: "?site=site-a", want: http.StatusOK},
+		{name: "mismatched site forbidden", userSite: "site-a", query: "?site=site-b", want: http.StatusForbidden},
+		{name: "missing site param rejected", userSite: "site-a", query: "", want: http.StatusForbidden},
+		{name: "empty site param rejected", userSite: "site-a", query: "?site=", want: http.StatusForbidden},
+	}
+
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+				r = rest.SetUserInfo(r, store.User{ID: "u", Name: "u", SiteID: c.userSite})
+				wrapped.ServeHTTP(w, r)
+			})
+			ts := httptest.NewServer(h)
+			defer ts.Close()
+			resp, err := http.Get(ts.URL + c.query)
+			require.NoError(t, err)
+			require.NoError(t, resp.Body.Close())
+			assert.Equal(t, c.want, resp.StatusCode)
+		})
+	}
 }