Dmitry VerkhoturovandGitHub
fb7b6c2cdd
* Serve the build-independent web assets from the backend `privacy.html`, `markdown-help.html` and the `400x400.jpeg` it embeds carry no template variable, link no script or stylesheet, and are imported by nothing in the widget. They now live in `backend/app/webassets/assets`, embedded there, and are served under `/web` alongside the frontend build. `/web` reads the frontend build first and falls back to them, which is what lets an operator replace one by dropping a file into `--web-root`. That is what `privacy.html` needs: it describes remark42.com, while the authorization guide tells operators to hand its URL to Google and Facebook as their own application's privacy policy. Only a missing file falls through. An unreadable file in the web root keeps reporting as unreadable rather than being silently replaced by the embedded copy, and a name the filesystem rejects reports as missing rather than as a server error, both matching what `http.Dir` did. The dev server serves the same directory, so the Markdown help link in the comment form resolves on the dev port as well as in production. The two pages are served as they are written. `markdown-help.html` was minified before, and its formatted inline stylesheet is most of its 8.5 kB; that is 2.4 kB more over the wire, behind the hour-long cache header the file server already sets. Drops `copy-webpack-plugin`, which had no other pattern, and the stylelint entries that only ever matched these files. * Make pnpm dev:app start again The dev server has been failing to start on two counts, so the flow the contributing guide documents does not run at all. `webpack-cli` 4 drives `webpack-dev-server` 5 through the argument order of an older major, handing it the compiler where it expects the options object. It rejects that against its schema and exits, complaining about an unknown `_assetEmittingPreviousFiles` property, which is a field of the compiler. `webpack-cli` 7 is the release that declares `webpack-dev-server` 5 as a peer. Past that, `http-proxy-middleware` resolves to 4.1.1, which no longer accepts the two-argument call `webpack-dev-server` makes, so the `/api` and `/auth` proxies throw on startup. It is pulled in by the security override for CVE-2025-32996, the only override in the file with no upper bound: `>=2.0.10` matches every later major. Bounding it to the 2.x line keeps the fix and the API `webpack-dev-server` calls. With both in place `pnpm dev:app` serves the widget and the pages under `/web` on port 9000.
92 lines
5.6 KiB
Markdown
92 lines
5.6 KiB
Markdown
---
|
|
title: Backend Development Guidelines
|
|
aliases:
|
|
- /docs/contributing/
|
|
---
|
|
|
|
You can use a fully functional local version to develop and test both frontend and backend. It requires at least 2GB RAM or swap enabled.
|
|
|
|
To bring it up, run:
|
|
|
|
```shell
|
|
cp compose-dev-backend.yml compose-private.yml
|
|
# now, edit / debug `compose-private.yml` to your heart's content
|
|
|
|
# build and run
|
|
docker compose -f compose-private.yml up --build
|
|
```
|
|
|
|
It starts Remark42 on `127.0.0.1:8080` and adds local OAuth2 provider "Dev". To access the UI demo page go to <http://127.0.0.1:8080/web/>. By default, you would be logged in as `dev_user`, defined as admin. You can tweak any of the [supported parameters](https://remark42.com/docs/configuration/parameters/) in corresponded yml file.
|
|
|
|
{{< note "🚨" >}}
|
|
Please use `127.0.0.1` and not `localhost` to access the server; otherwise, CORS will prevent your browser from authentication to work correctly. You could alter the address for dev auth with the `REMARK_URL` environment variable.
|
|
{{< /note >}}
|
|
|
|
Backend Docker Compose config (`compose-dev-backend.yml`) by default skips running frontend related tests. Frontend Docker Compose config (`compose-dev-frontend.yml`) by default skips running backend related tests and sets `NODE_ENV=development` for frontend build.
|
|
|
|
### Backend development
|
|
|
|
#### With Docker
|
|
|
|
Run tests in your IDE, and re-run `make rundev` each time you want to see how your code changes behave to test them at <http://127.0.0.1:8080/web/>.
|
|
|
|
#### Without Docker
|
|
|
|
You have to [install](https://golang.org/doc/install) the latest stable `go` toolchain to run the backend locally.
|
|
|
|
In order to have working Remark42 installation you need once to copy frontend static files to `./backend/app/cmd/web` directory from `master` docker image, as it is expected to be where application compiles:
|
|
|
|
```shell
|
|
# frontend files
|
|
docker pull ghcr.io/umputun/remark42:master
|
|
docker create -ti --name remark42files ghcr.io/umputun/remark42:master sh
|
|
docker cp remark42files:/srv/web/ ./backend/app/cmd/
|
|
docker rm -f remark42files
|
|
# fix frontend files to point to the right URL
|
|
## Mac version
|
|
find -E ./backend/app/cmd/web -regex '.*\.(html|js|mjs)$' -print -exec sed -i '' "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \;
|
|
## Linux version
|
|
find ./backend/app/cmd/web -regex '.*\.\(html\|js\|mjs\)$' -print -exec sed -i "s|{% REMARK_URL %}|http://127.0.0.1:8080|g" {} \;
|
|
```
|
|
|
|
The assets under `/web` the frontend does not build (`privacy.html`, `markdown-help.html`,
|
|
`400x400.jpeg`) come from `backend/app/webassets/assets` and are embedded in the binary, so the copy
|
|
above neither covers them nor needs to. At runtime a file of the same name under `web-root` /
|
|
`REMARK_WEB_ROOT` is served in preference to the embedded one.
|
|
|
|
To run backend - `cd backend; go run app/main.go server --dbg --secret=12345 --url=http://127.0.0.1:8080 --admin-passwd=password --site=remark`. It stars backend service with embedded bolt store on port `8080` with basic auth, allowing to authenticate and run requests directly, like this:
|
|
|
|
`HTTP http://admin:password@127.0.0.1:8080/api/v1/find?site=remark&sort=-active&format=tree&url=http://127.0.0.1:8080`
|
|
|
|
## Technical Details
|
|
|
|
Data stored in [boltdb](https://github.com/etcd-io/bbolt) (embedded key/value database) files under `STORE_BOLT_PATH`. Each site is stored in a separate boltdb file.
|
|
|
|
To migrate/move Remark42 to another host, these boltdb files must be transferred. Additionally, the avatars directory `AVATAR_FS_PATH` (default `./var/avatars`) and images directory `IMAGE_FS_PATH` (default `./var/pictures`) must be transferred. As an alternative to storing avatars and images in the file system, boltdb can be used by setting `AVATAR_TYPE` or `IMAGE_TYPE` to `bolt`. Files in the `IMAGE_FS_STAGING` directory can be safely ignored because they are moved to the image store between app restarts.
|
|
|
|
The automatic backup process runs every 24h and exports all content in JSON-like format to `backup-remark-YYYYMMDD.gz`.
|
|
|
|
Authentication implemented with [go-pkgz/auth](https://github.com/go-pkgz/auth) stored in a cookie. It uses HttpOnly, secure cookies.
|
|
|
|
All heavy REST calls cached internally in LRU cache limited by `CACHE_MAX_ITEMS` and `CACHE_MAX_SIZE` with [go-pkgz/rest](https://github.com/go-pkgz/rest).
|
|
|
|
User's activity throttled globally (up to 1000 simultaneous requests) and limited locally (per user, usually up to 10 req/sec).
|
|
|
|
Request timeout set to 60sec.
|
|
|
|
Admin authentication (`--admin-password` set) allows to hit Remark42 API without social login and admin privileges. Adds basic-auth for username: `admin`, password: `${ADMIN_PASSWD}`. Enable it only for the initial comment import or for manual backups. Do not leave the server running with admin password set if you don't have an intention to keep creating backups manually!
|
|
|
|
User can vote for the comment multiple times but only to change the vote. Double voting is not allowed.
|
|
|
|
User can edit comments in 5 mins (configurable) window after creation.
|
|
|
|
User ID hashed and prefixed by OAuth provider name to avoid collisions and potential abuse.
|
|
|
|
All avatars resized and cached locally to prevent rate limiters from OAuth providers, part of [go-pkgz/auth](https://github.com/go-pkgz/auth) functionality.
|
|
|
|
Images served over HTTP can be proxied to HTTPS (`IMAGE_PROXY_HTTP2HTTPS=true`) to prevent mixed HTTP/HTTPS.
|
|
|
|
All images can be proxied and saved locally (`IMAGE_PROXY_CACHE_EXTERNAL=true`) instead of serving from the original location. Beware, images that are posted with this parameter enabled will be served from proxy even after it is disabled.
|
|
|
|
Docker build uses [publicly available](https://github.com/umputun/baseimage) base images.
|