# remark42 [](https://github.com/umputun/remark/actions) [](https://goreportcard.com/report/github.com/umputun/remark) [](https://coveralls.io/github/umputun/remark?branch=master)
Remark42 is a self-hosted, lightweight, and simple (yet functional) comment engine, which doesn't spy on users. It can be embedded into blogs, articles or any other place where readers add comments.
* Social login via Google, Twitter, Facebook, GitHub and Yandex
* Login via email
* Optional anonymous access
* Multi-level nested comments with both tree and plain presentations
* Import from Disqus and WordPress
* Markdown support with friendly formatter toolbar
* Moderator can remove comments and block users
* Voting, pinning and verification system
* Sortable comments
* Images upload with drag-and-drop
* Extractor for recent comments, cross-post
* RSS for all comments and each post
* Telegram and email notifications
* Export data to json with automatic backups
* No external databases, everything embedded in a single data file
* Fully dockerized and can be deployed in a single command
* Self-contained executable can be deployed directly to Linux, Windows and MacOS
* Clean, lightweight and customizable UI with white and dark themes
* Multi-site mode from a single instance
* Integration with automatic ssl (direct and via [nginx-le](https://github.com/umputun/nginx-le))
* [Privacy focused](#privacy)
[Demo site](https://remark42.com/demo/) available with all authentication methods, including email auth and anonymous access.
Screenshots
Comments example:
For admin screenshots see [Admin UI wiki](https://github.com/umputun/remark/wiki/Admin-UI)
deprecated options
| Command line | Replacement | Environment | Replacement | Default | Description | Deprecation version |
| ------------------ | ------------- | ------------------ | ------------- | ------- | -------------- | ------------------- |
| auth.email.host | smtp.host | AUTH_EMAIL_HOST | SMTP_HOST | | smtp host | 1.5.0 |
| auth.email.port | smtp.port | AUTH_EMAIL_PORT | SMTP_PORT | | smtp port | 1.5.0 |
| auth.email.user | smtp.username | AUTH_EMAIL_USER | SMTP_USERNAME | | smtp user name | 1.5.0 |
| auth.email.passwd | smtp.password | AUTH_EMAIL_PASSWD | SMTP_PASSWORD | | smtp password | 1.5.0 |
| auth.email.tls | smtp.tls | AUTH_EMAIL_TLS | SMTP_TLS | `false` | enable TLS | 1.5.0 |
| auth.email.timeout | smtp.timeout | AUTH_EMAIL_TIMEOUT | SMTP_TIMEOUT | `10s` | smtp timeout | 1.5.0 |
| img-proxy | image-proxy.http2https | IMG_PROXY | IMAGE_PROXY_HTTP2HTTPS | `false` | enable http->https proxy for images | 1.5.0 |
##### Required parameters
Most of the parameters have sane defaults and don't require customization. There are only a few parameters user has to define:
1. `SECRET` - secret key, can be any long and hard-to-guess string.
2. `REMARK_URL` - url pointing to your remark42 server, i.e. `https://demo.remark42.com`
3. At least one pair of `AUTH__CID` and `AUTH__CSEC` defining oauth2 provider(s)
The minimal `docker-compose.yml` has to include all required parameters:
```yaml
version: '2'
services:
remark42:
image: umputun/remark42:latest
restart: always
container_name: "remark42"
environment:
- REMARK_URL=https://demo.remark42.com # url pointing to your remark42 server
- SITE=YOUR_SITE_ID # site ID, same as used for `site_id`, see "Setup on your website"
- SECRET=abcd-123456-xyz-$%^& # secret key
- AUTH_GITHUB_CID=12345667890 # oauth2 client ID
- AUTH_GITHUB_CSEC=abcdefg12345678 # oauth2 client secret
volumes:
- ./var:/srv/var # persistent volume to store all remark42 data
```
#### Quick installation test
To verify if remark has been properly installed, check a demo page at `${REMARK_URL}/web` URL. Make sure to include `remark` site id to `${SITE}` list.
#### Register oauth2 providers
Authentication handled by external providers. You should setup oauth2 for all (or some) of them to allow users to make comments. It is not mandatory to have all of them, but at least one should be correctly configured.
##### Google Auth Provider
1. Create a new project: https://console.developers.google.com/project
1. Choose the new project from the top right project dropdown (only if another project is selected)
1. In the project Dashboard center pane, choose **"API Manager"**
1. In the left Nav pane, choose **"Credentials"**
1. In the center pane, choose **"OAuth consent screen"** tab. Fill in **"Product name shown to users"** and hit save.
1. In the center pane, choose **"Credentials"** tab.
* Open the **"New credentials"** drop down
* Choose **"OAuth client ID"**
* Choose **"Web application"**
* Application name is freeform, choose something appropriate
* Authorized origins is your domain ex: `https://remark42.mysite.com`
* Authorized redirect URIs is the location of oauth2/callback constructed as domain + `/auth/google/callback`, ex: `https://remark42.mysite.com/auth/google/callback`
* Choose **"Create"**
1. Take note of the **Client ID** and **Client Secret**
_instructions for google oauth2 setup borrowed from [oauth2_proxy](https://github.com/bitly/oauth2_proxy)_
##### GitHub Auth Provider
1. Create a new **"OAuth App"**: https://github.com/settings/developers
1. Fill **"Application Name"** and **"Homepage URL"** for your site
1. Under **"Authorization callback URL"** enter the correct url constructed as domain + `/auth/github/callback`. ie `https://remark42.mysite.com/auth/github/callback`
1. Take note of the **Client ID** and **Client Secret**
##### Facebook Auth Provider
1. From https://developers.facebook.com select **"My Apps"** / **"Add a new App"**
1. Set **"Display Name"** and **"Contact email"**
1. Choose **"Facebook Login"** and then **"Web"**
1. Set "Site URL" to your domain, ex: `https://remark42.mysite.com`
1. Under **"Facebook login"** / **"Settings"** fill "Valid OAuth redirect URIs" with your callback url constructed as domain + `/auth/facebook/callback`
1. Select **"App Review"** and turn public flag on. This step may ask you to provide a link to your privacy policy.
##### Twitter Auth Provider
1. Create a new twitter application https://developer.twitter.com/en/apps
1. Fill **App name**, **Description** and **URL** of your site
1. In the field **Callback URLs** enter the correct url of your callback handler e.g. domain + `/auth/twitter/callback`
1. Under **Key and tokens** take note of the **Consumer API Key** and **Consumer API Secret key**. Those will be used as `AUTH_TWITTER_CID` and
`AUTH_TWITTER_CSEC`
##### Yandex Auth Provider
1. Create a new **"OAuth App"**: https://oauth.yandex.com/client/new
1. Fill **"App name"** for your site
1. Under **Platforms** select **"Web services"** and enter **"Callback URI #1"** constructed as domain + `/auth/yandex/callback`. ie `https://remark42.mysite.com/auth/yandex/callback`
1. Select **Permissions**. You need following permissions only from the **"Yandex.Passport API"** section:
* Access to user avatar
* Access to username, first name and surname, gender
1. Fill out the rest of fields if needed
1. Take note of the **ID** and **Password**
For more details refer to [Yandex OAuth](https://tech.yandex.com/oauth/doc/dg/concepts/about-docpage/) and [Yandex.Passport](https://tech.yandex.com/passport/doc/dg/index-docpage/) API documentation.
##### Anonymous Auth Provider
Optionally, anonymous access can be turned on. In this case an extra `anonymous` provider will allow logins without any social login with any name satisfying 2 conditions:
- name should be at least 3 characters long
- name has to start from the letter and contains letters, numbers, underscores and spaces only.
#### Initial import from Disqus
1. Disqus provides an export of all comments on your site in a g-zipped file. This is found in your Moderation panel at Disqus Admin > Setup > Export. The export will be sent into a queue and then emailed to the address associated with your account once it's ready. Direct link to export will be something like `https://.disqus.com/admin/discussions/export/`. See [importing-exporting](https://help.disqus.com/customer/portal/articles/1104797-importing-exporting) for more details.
2. Move this file to your remark42 host within `./var` and unzip, i.e. `gunzip .xml.gz`.
3. Run import command - `docker exec -it remark42 import -p disqus -f {disqus-export-name}.xml -s {your site id}`
#### Initial import from WordPress
1. Install WordPress [plugin](https://wordpress.org/plugins/wp-exporter/) to export comments and follow it instructions. The plugin should produce a xml-based file with site content including comments.
2. Move this file to your remark42 host within `./var`
3. Run import command - `docker exec -it remark42 import -p wordpress -f {wordpress-export-name}.xml -s {your site id}`
#### Backup and restore
##### Automatic backups
Remark42 by default makes daily backup files under `${BACKUP_PATH}` (default `./var/backup`). Backups kept up to `${MAX_BACKUP_FILES}` (default 10). Each backup file contains exported and gzipped content, i.e., all comments. At any point, the user can restore such backup and revert all comments to the desirable state. Note: restore procedure cleans the current data store and replaces all comments with comments from the backup file.
For safety and security reasons restore functionality not exposed outside of your server by default. The recommended way to restore from the backup is to use provided `scripts/restore-backup.sh`. It can run inside the container:
`docker exec -it remark42 restore -f {backup-filename.gz} -s {your site id}`
##### Manual backup
In addition to automatic backups user can make a backup manually. This command makes `userbackup-{site id}-{timestamp}.gz` by default.
`docker exec -it remark42 backup -s {your site id}`
##### Restore from backup
Restore will clean all comments first and then will processed with complete import from a given file.
`docker exec -it remark42 restore -f {backup file name} -s {your site id}`
##### Backup format
Backup file is a text file with all exported comments separated by EOL. Each backup record is a valid json with all key/value
unmarshaled from `Comment` struct (see below).
#### Admin users
Admins/moderators should be defined in `docker-compose.yml` as a list of user IDs or passed in the command line.
```
environment:
- ADMIN_SHARED_ID=github_ef0f706a79cc24b17bbbb374cd234a691a034128,github_dae9983158e9e5e127ef2b87a411ef13c891e9e5
```
To get user id just login and click on your username or any other user you want to promote to admins.
It will expand login info and show full user ID.
#### Docker parameters
Two parameters allow to customize docker container on the system level:
- `APP_UID` - sets UID to run remark42 application in container (default=1001)
- `TIME_ZONE` - sets time zone of remark42 container (default=America/Chicago)
_see [umputun/baseimage](https://github.com/umputun/baseimage) for more details_
example of compose:
```yaml
version: '2'
services:
remark42:
image: umputun/remark42:latest
restart: always
container_name: "remark42"
environment:
- APP_UID=2000 # runs remark42 app with non-default UID
- TIME_ZONE=GTC # sets container time to UTC
- REMARK_URL=https://demo.remark42.com # url pointing to your remark42 server
- SITE=YOUR_SITE_ID # site ID, same as used for `site_id`, see "Setup on your website"
- SECRET=abcd-123456-xyz-$%^& # secret key
- AUTH_GITHUB_CID=12345667890 # oauth2 client ID
- AUTH_GITHUB_CSEC=abcdefg12345678 # oauth2 client secret
volumes:
- ./var:/srv/var # persistent volume to store all remark42 data
```
### Setup on your website
#### Comments
It's a main widget which renders list of comments.
Add this snippet to the bottom of web page:
```html
```
And then add this node in the place where you want to see Remark42 widget:
```html
```
After that widget will be rendered inside this node.
##### Themes
Right now Remark has two themes: light and dark.
You can pick one using configuration object,
but there is also a possibility to switch between themes in runtime.
For this purpose Remark adds to `window` object named `REMARK42`,
which contains function `changeTheme`.
Just call this function and pass a name of the theme that you want to turn on:
```js
window.REMARK42.changeTheme('light');
```
#### Last comments
It's a widget which renders list of last comments from your site.
Add this snippet to the bottom of web page, or adjust already present `remark_config` to have `last-comments` in `components` list:
```html
```
And then add this node in the place where you want to see last comments widget:
```html
```
`data-max` sets the max amount of comments (default: `15`).
#### Counter
It's a widget which renders a number of comments for the specified page.
Add this snippet to the bottom of web page, or adjust already present `remark_config` to have `counter` in `components` list:
```html
```
And then add a node like this in the place where you want to see a number of comments:
```html
```
You can use as many nodes like this as you need to.
The script will found all them by the class `remark__counter`,
and it will use `data-url` attribute to define the page with comments.
Also script can use `url` property from `remark_config` object, or `window.location.origin + window.location.pathname` if nothing else is defined.
## Build from the source
- to build docker container - `make docker`. This command will produce container `umputun/remark42`.
- to build a single binary for direct execution - `make OS= ARCH=`. This step will produce executable
`remark42` file with everything embedded.
## Development
You can use fully functional local version to develop and test both frontend & backend.
To bring it up run:
```bash
# if you mainly work on backend
docker-compose -f compose-dev-backend.yml build
docker-compose -f compose-dev-backend.yml up
# if you mainly work on frontend
docker-compose -f compose-dev-frontend.yml build
docker-compose -f compose-dev-frontend.yml up
```
It starts Remark42 on `127.0.0.1:8080` and adds local OAuth2 provider “Dev”.
To access UI demo page go to `127.0.0.1:8080/web`.
By default, you would be logged in as `dev_user` which defined as admin.
You can tweak any of [supported parameters](#Parameters) in corresponded yml file.
Backend docker compose config by default skips running frontend related tests.
Frontend docker compose config by default skips running backend related tests and sets `NODE_ENV=development` for frontend build.
### Backend development
In order to run backend locally (development mode, without docker) you have to have latest stable `go` toolchain [installed](https://golang.org/doc/install).
To run backend - `go run backend/app/main.go server --dbg --secret=12345 --url=http://127.0.0.1:8080 --admin-passwd=password --site=remark`
It stars backend service with embedded bolt store on port `8080` with basic auth, allowing to authenticate and run requests directly, like this:
`HTTP http://admin:password@127.0.0.1:8080/api/v1/find?site=remark&sort=-active&format=tree&url=http://127.0.0.1:8080`
### Frontend development
#### Build
* install [Node.js 8](https://nodejs.org/en/) or higher;
* install [NPM 6.1.0](https://www.npmjs.com/package/npm);
* run `npm install` inside `./frontend`;
* run `npm run build` there;
* result files will be saved in `./frontend/public`.
**Note** Running `npm install` will set up precommit hooks into your git repository.
It used to reformat your frontend code using `prettier` and lint with `eslint` before every commit.
#### Devserver
For local development mode with Hot Reloading use `npm start` instead of `npm run build`.
In this case `webpack` will serve files using `webpack-dev-server` on `localhost:9000`.
By visiting `127.0.0.1:9000/web` you will get a page with main comments widget
communicating with demo server backend running on `https://demo.remark42.com`.
But you will not be able to login with any oauth providers due to security reasons.
You can attach to locally running backend by providing `REMARK_URL` environment variable.
```sh
npx cross-env REMARK_URL=http://127.0.0.1:8080 npm start
```
Developer build running by `webpack-dev-server` supports devtools for [React](https://github.com/facebook/react-devtools) and
[Redux](https://github.com/zalmoxisus/redux-devtools-extension).
#### Frontend guide
Frontend guide can be found here: [./frontend/Readme.md](./frontend/Readme.md)
## API
### Authorization
* `GET /auth/{provider}/login?from=http://url&site=site_id&session=1` - perform "social" login with one of supported providers and redirect to `url`. Presence of `session` (any non-zero value) change the default cookie expiration and makes them session-only.
* `GET /auth/logout` - logout
```go
type User struct {
Name string `json:"name"`
ID string `json:"id"`
Picture string `json:"picture"`
Admin bool `json:"admin"`
Blocked bool `json:"block"`
Verified bool `json:"verified"`
}
```
_currently supported providers are `google`, `facebook`, `github` and `yandex`_
### Commenting
* `POST /api/v1/comment` - add a comment. _auth required_
```go
type Comment struct {
ID string `json:"id"` // comment ID, read only
ParentID string `json:"pid"` // parent ID
Text string `json:"text"` // comment text, after md processing
Orig string `json:"orig"` // original comment text
User User `json:"user"` // user info, read only
Locator Locator `json:"locator"` // post locator
Score int `json:"score"` // comment score, read only
Vote int `json:"vote"` // vote for the current user, -1/1/0.
Controversy float64 `json:"controversy,omitempty"` // comment controversy, read only
Timestamp time.Time `json:"time"` // time stamp, read only
Edit *Edit `json:"edit,omitempty" bson:"edit,omitempty"` // pointer to have empty default in json response
Pin bool `json:"pin"` // pinned status, read only
Delete bool `json:"delete"` // delete status, read only
PostTitle string `json:"title"` // post title
}
type Locator struct {
SiteID string `json:"site"` // site id
URL string `json:"url"` // post url
}
type Edit struct {
Timestamp time.Time `json:"time" bson:"time"`
Summary string `json:"summary"`
}
```
* `POST /api/v1/preview` - preview comment in html. Body is `Comment` to render
* `GET /api/v1/find?site=site-id&url=post-url&sort=fld&format=tree|plain` - find all comments for given post
This is the primary call used by UI to show comments for given post. It can return comments in two formats - `plain` and `tree`.
In plain format result will be sorted list of `Comment`. In tree format this is going to be tree-like object with this structure:
```go
type Tree struct {
Nodes []Node `json:"comments"`
Info store.PostInfo `json:"info,omitempty"`
}
type Node struct {
Comment store.Comment `json:"comment"`
Replies []Node `json:"replies,omitempty"`
}
```
Sort can be `time`, `active` or `score`. Supported sort order with prefix -/+, i.e. `-time`. For `tree` mode sort will be applied to top-level comments only and all replies always sorted by time.
* `PUT /api/v1/comment/{id}?site=site-id&url=post-url` - edit comment, allowed once in `EDIT_TIME` minutes since creation. Body is `EditRequest` json
```go
type EditRequest struct {
Text string `json:"text"` // updated text
Summary string `json:"summary"` // optional, summary of the edit
Delete bool `json:"delete"` // delete flag
}{}
```
* `GET /api/v1/last/{max}?site=site-id&since=ts-msec` - get up to `{max}` last comments, `since` (epoch time, milliseconds) is optional
* `GET /api/v1/id/{id}?site=site-id` - get comment by `comment id`
* `GET /api/v1/comments?site=site-id&user=id&limit=N` - get comment by `user id`, returns `response` object
```go
type response struct {
Comments []store.Comment `json:"comments"`
Count int `json:"count"`
}{}
```
* `GET /api/v1/count?site=site-id&url=post-url` - get comment's count for `{url}`
* `POST /api/v1/count?site=siteID` - get number of comments for posts from post body (list of post IDs)
* `GET /api/v1/list?site=site-id&limit=5&skip=2` - list commented posts, returns array or `PostInfo`, limit=0 will return all posts
```go
type PostInfo struct {
URL string `json:"url"`
Count int `json:"count"`
ReadOnly bool `json:"read_only,omitempty"`
FirstTS time.Time `json:"first_time,omitempty"`
LastTS time.Time `json:"last_time,omitempty"`
}
```
* `GET /api/v1/user` - get user info, _auth required_
* `PUT /api/v1/vote/{id}?site=site-id&url=post-url&vote=1` - vote for comment. `vote`=1 will increase score, -1 decrease. _auth required_
* `GET /api/v1/userdata?site=site-id` - export all user data to gz stream _auth required_
* `POST /api/v1/deleteme?site=site-id` - request deletion of user data. _auth required_
* `GET /api/v1/config?site=site-id` - returns configuration (parameters) for given site
```go
type Config struct {
Version string `json:"version"`
EditDuration int `json:"edit_duration"`
MaxCommentSize int `json:"max_comment_size"`
Admins []string `json:"admins"`
AdminEmail string `json:"admin_email"`
Auth []string `json:"auth_providers"`
LowScore int `json:"low_score"`
CriticalScore int `json:"critical_score"`
PositiveScore bool `json:"positive_score"`
ReadOnlyAge int `json:"readonly_age"`
MaxImageSize int `json:"max_image_size"`
EmojiEnabled bool `json:"emoji_enabled"`
}
```
* `GET /api/v1/info?site=site-idd&url=post-url` - returns `PostInfo` for site and url
### Streaming API
Streaming API provide server-sent events for post updates as well as site update
* `GET /api/v1/stream/info?site=site-idd&url=post-url&since=unix_ts_msec` - returns stream (`event: info`) with `PostInfo` records for the site and url. `since` is optional
* `GET /api/v1/stream/last?site=site-id&since=unix_ts_msec` - returns updates stream (`event: last`) with comments for the site, `since` is optional
response example
```
data: {"url":"https://radio-t.com/blah1","count":2,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.142872-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":3,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.157709-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":4,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.172991-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":5,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.188429-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":6,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.204742-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":7,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.220692-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":8,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.23817-05:00"}
event: info
data: {"url":"https://radio-t.com/blah1","count":9,"first_time":"2019-06-18T12:53:48.125686-05:00","last_time":"2019-06-18T12:53:48.254669-05:00"}
```