Compare commits

...
45 Commits
Author SHA1 Message Date
Umputun 9375f250d8 switch restricted words logging to log 2019-01-21 17:03:22 -06:00
Umputun 84488182a2 avloid tokenization if lister failed or an empty list of restricted words #237 2019-01-21 16:52:49 -06:00
Aleksandr MelnikovandUmputun ab6c9a515c Support restricted words (#257)
* prepared interfaces and naive implementation for restricted words

* fixed whitespace tokenizer last token handling

* removed unnecessary check in whitespace tokenizer last token handling

* removed unnecessary interfaces, added matcher to service

* now using more explicit naming

* added config parameter/env variable to configure restricted words
provided simple matcher and lister implementation into service

* fixed linter warnings

* fixed linter warnings (part 2)

* added functional tests for rest api
moved check for restricted words into validation stage

* fixed missing error check in rest private test

* fixed failed test that expected validation in specific order

* fixed linter warning

* removed unnecessary 'else'

* removed redundant check from 'DataStore.EditComment'

* replaced exact matching for restricted words with wildcard matching

* fixed race condition when creating wildcardTrie
removed unnecessary wildcard masks
refactored tests in a table format

* improved documentation for restricted words param

* removed wildcardTrie caching as a premature optimization
removed unnecessary validation method ValidateCommentText
moved restricted words validation into service methods that create and update comment

* fixed error name due to linter warning
2019-01-21 16:48:35 -06:00
Umputun 2c988aa226 fix tg send test 2019-01-21 13:34:42 -06:00
Umputun 1a78c6eec6 add @ prefix for non-integer tg channelID #255 2019-01-21 13:28:03 -06:00
UmputunandGitHub dbc664e102 Merge pull request #258 from VaultVulp/feature/rework-telegram-integration
Remove prefixing of a `@` character to the channel name
2019-01-21 13:15:09 -06:00
Pavel Alimpiev a9909a91f4 Remove prefixing of a @ character to the channel name 2019-01-18 12:50:13 +03:00
Umputun 62b54eb009 resort imports 2019-01-17 18:50:02 -06:00
Umputun 7fa26343bd fix a comment 2019-01-17 15:07:47 -06:00
Umputun 97d1fa3942 add update-limit to readme options list 2019-01-17 14:57:33 -06:00
Umputun 2a3051c620 allow separate limiter for updates and posts, agressive default limit 2019-01-17 14:56:11 -06:00
Umputun 953ff03e2f missed in last commit 2019-01-17 13:18:49 -06:00
Umputun f7403f08b9 inject refresh cache to auth, revendor go-pkgz/auth to 0.4 2019-01-17 13:17:16 -06:00
UmputunandGitHub 6aae306be6 Merge pull request #250 from Reeywhaar/#248-frontend
Disable comment editing if it has reply
2019-01-17 11:27:37 -06:00
UmputunandGitHub f7851bc5f0 Merge pull request #254 from Reeywhaar/252-fix
Fix wrong label on voting buttons
2019-01-17 11:27:19 -06:00
UmputunandGitHub 225cd04980 Merge pull request #251 from Arelav/master
Prioritize disabled arrows style over general
2019-01-17 11:15:48 -06:00
Umputun 168088445f add caching for replied ids 2019-01-17 01:20:19 -06:00
Umputun ff46cbd173 require authed user to perfrom logout 2019-01-16 23:28:49 -06:00
Vyrtsev Mikhail ddd2c25498 exclude pin/unpin from last comments comment controls 2019-01-17 04:24:17 +03:00
Vyrtsev Mikhail 00bf148630 restore comments controls in last comments 2019-01-17 04:14:06 +03:00
Vyrtsev Mikhail e42110a3ba fix wrong label on voting buttons 2019-01-17 04:07:13 +03:00
Valery KharshatsandGitHub 5a034f4b52 Prioritize disabled arrows style over general
It disables hover color change for not logged in users.
2019-01-16 16:21:05 -05:00
Vyrtsev Mikhail 86fde18b7d hide edit input if original text and edited are same 2019-01-17 00:06:04 +03:00
Vyrtsev Mikhail 7ba4518587 disable comment editing if it has reply 2019-01-16 23:39:23 +03:00
Umputun 79a8bfe453 don't exclude reply to myself from HasReply detection https://github.com/umputun/remark/issues/248#issuecomment-454882841 2019-01-16 12:20:05 -06:00
Umputun dff266ec01 add detection of reply and disable edit for such comments #248 2019-01-15 17:05:30 -06:00
Umputun 367c3d0cc2 blocked users returns 401 2019-01-15 13:17:59 -06:00
Umputun 78debf05d9 auth 0.3.3 prevnets expired headers to pe passed in 2019-01-15 13:09:56 -06:00
Umputun e9585dc568 pass default logger to rest 2019-01-14 16:14:02 -06:00
Umputun 177b71655d update deps (lgr and mongo) 2019-01-14 00:06:32 -06:00
Umputun 8c8f13fd73 adopt for rest errors and revendor go-pkgz 2019-01-13 23:27:15 -06:00
Umputun 492f92e660 minor fixes for #247 2019-01-13 15:02:44 -06:00
UmputunandGitHub 9767bd42d2 Merge pull request #247 from saromanov/telegram-increase-coverage
Telegram increase coverage
2019-01-13 14:55:17 -06:00
Umputun cfe8391da5 revendor with lgr 0.2 2019-01-11 22:51:43 -06:00
Sergey c08a31809c moving back assert.equal 2019-01-12 01:22:08 +05:00
Sergey dd9b7d46ac telegram: increase coverage 2019-01-12 01:15:50 +05:00
Umputun 8b0926a338 fix #246 2019-01-11 14:06:31 -06:00
Misha VyrtsevandIgor Adamenko 57ffa5d491 Show post title in user comments sidebar (#244)
* show post title in user comments sidebar #241

* trim title with ellipsis

* show post title in user comments sidebar #241: post-review

* adjust to make user comments post title the same as last comments widget style

* remove user comments post title margin

* revert to f35ecc75b0
2019-01-09 16:20:55 +02:00
Umputun e5c8ac18de bad import 2019-01-08 12:44:07 -06:00
Umputun 835e82a441 add guid to rss with comment id 2019-01-08 12:41:10 -06:00
Umputun 3c54c159d2 Merge branch 'master' of github.com:umputun/remark 2019-01-08 11:41:26 -06:00
UmputunandGitHub 02d94ead1d Merge pull request #245 from umputun/lgr
Lgr
2019-01-08 04:01:33 -06:00
Umputun 61cf54d231 add comment id to rss 2019-01-08 03:59:15 -06:00
Umputun 781d90e98b switch to lgr 2019-01-07 22:32:50 -06:00
Umputun e850dc1d3e add post title to comments api docs 2019-01-06 16:39:11 -06:00
105 changed files with 1378 additions and 995 deletions
+5
View File
@@ -1,6 +1,11 @@
FROM node:10.11-alpine as build-frontend-deps
ARG CI
ARG DRONE
ARG DRONE_TAG
ARG DRONE_COMMIT
ARG DRONE_BRANCH
ENV SKIP_FRONTEND_TEST=true
RUN apk add --no-cache --update git
+3
View File
@@ -130,9 +130,11 @@ _this is the recommended way to run remark42_
| max-votes | MAX_VOTES | `-1` | votes limit per comment, `-1` - unlimited |
| low-score | LOW_SCORE | `-5` | low score threshold |
| critical-score | CRITICAL_SCORE | `-10` | critical score threshold |
| restricted-words | RESTRICTED_WORDS | | words banned in comments (can use `*`), _multi_ |
| edit-time | EDIT_TIME | `5m` | edit window |
| read-age | READONLY_AGE | | read-only age of comments, days |
| img-proxy | IMG_PROXY | `false` | enable http->https proxy for images |
| update-limit | UPDATE_LIMIT | `0.5` | updates/sec limit |
| admin-passwd | ADMIN_PASSWD | | password for `admin` basic auth |
| dbg | DEBUG | `false` | debug mode |
@@ -483,6 +485,7 @@ type Comment struct {
Timestamp time.Time `json:"time"` // time stamp, read only
Pin bool `json:"pin"` // pinned status, read only
Delete bool `json:"delete"` // delete status, read only
PostTitle string `json:"title"` // post title
}
type Locator struct {
+19 -19
View File
@@ -112,7 +112,7 @@
version = "v1.0.0"
[[projects]]
digest = "1:5371050ba40cd7482fa2e9f0ff17fc18e2c37a6353ac012eaaacc6250eee4748"
digest = "1:d31c72099b9e195785933d11441bc92cce9b2c421f8fca712a7eaf36a8e55a47"
name = "github.com/go-pkgz/auth"
packages = [
".",
@@ -123,16 +123,24 @@
"token",
]
pruneopts = "UT"
revision = "6f889bf1c6eb61c926dbb759897e6dc577b10655"
version = "v0.3.0"
revision = "92256b685196d26a346dfbedea12898fe589b0eb"
version = "v0.4.0"
[[projects]]
digest = "1:1212e114344a5cdcc834ea69e19d456eef230f9784659080fee67e02ba2cb574"
digest = "1:1933dabfb0e07548ed9684fdef857cc6f2413bbbb74ba838a7c1f77407b8fd90"
name = "github.com/go-pkgz/lgr"
packages = ["."]
pruneopts = "UT"
revision = "8f56fccc1a15dfe9c7f72e049fc87cc378205b97"
version = "v0.2.2"
[[projects]]
digest = "1:c509e3f646c48148f2239fa66d5cd62785dfd057b5d2b46972e45b60049c19ed"
name = "github.com/go-pkgz/mongo"
packages = ["."]
pruneopts = "UT"
revision = "9a09a574c336c6ae2338a65bbebed2baab2a713c"
version = "v1.0.0"
revision = "ee1e9a167094f8cdbc27407907aa694fff350ec3"
version = "v1.1.0"
[[projects]]
digest = "1:0f24c72d7e9bcb682b907be0461ac552973cd4b3f1b60b04b725f6d74a3e59e7"
@@ -146,7 +154,7 @@
version = "v1.0.0"
[[projects]]
digest = "1:e133aa7be09588b02198e4ddb98df5033b0319b56533881d0163ee51b903305b"
digest = "1:9aba5c95373481f118e57e9740d9e82f86802d86849809ab89324bcb4f236451"
name = "github.com/go-pkgz/rest"
packages = [
".",
@@ -154,8 +162,8 @@
"logger",
]
pruneopts = "UT"
revision = "553c0e1b55b215f8f55da4682ac57aff9aec8b6d"
version = "v1.1.5"
revision = "e7d08d0194d613b8854de2e487bf7732500fa153"
version = "v1.2.0"
[[projects]]
digest = "1:ffc060c551980d37ee9e428ef528ee2813137249ccebb0bfc412ef83071cac91"
@@ -208,14 +216,6 @@
pruneopts = "UT"
revision = "0fb14efe8c47ae851c0034ed7a448854d3d34cf3"
[[projects]]
branch = "master"
digest = "1:ce3f7860fd68bd2dd4c3735e2aed8c9de7c7d05bd6ad7d97a6bedcf4fe7b84fb"
name = "github.com/hashicorp/logutils"
packages = ["."]
pruneopts = "UT"
revision = "0dc08b1671f34c4250ce212759ebd880f743d883"
[[projects]]
branch = "master"
digest = "1:3217800110ab50cd0e0784307be46b5344c0c103dbd15a16d0994ae4abdc96ab"
@@ -403,9 +403,9 @@
"github.com/go-chi/render",
"github.com/go-pkgz/auth",
"github.com/go-pkgz/auth/avatar",
"github.com/go-pkgz/auth/logger",
"github.com/go-pkgz/auth/provider",
"github.com/go-pkgz/auth/token",
"github.com/go-pkgz/lgr",
"github.com/go-pkgz/mongo",
"github.com/go-pkgz/repeater",
"github.com/go-pkgz/rest",
@@ -414,7 +414,6 @@
"github.com/google/uuid",
"github.com/gorilla/feeds",
"github.com/hashicorp/go-multierror",
"github.com/hashicorp/logutils",
"github.com/jessevdk/go-flags",
"github.com/microcosm-cc/bluemonday",
"github.com/patrickmn/go-cache",
@@ -423,6 +422,7 @@
"github.com/stretchr/testify/assert",
"github.com/stretchr/testify/require",
"golang.org/x/crypto/acme/autocert",
"golang.org/x/net/html",
"gopkg.in/russross/blackfriday.v2",
]
solver-name = "gps-cdcl"
-4
View File
@@ -12,10 +12,6 @@ required = ["github.com/patrickmn/go-cache"]
name = "github.com/google/uuid"
version = "0.2.0"
[[constraint]]
branch = "master"
name = "github.com/hashicorp/logutils"
[[constraint]]
branch = "master"
name = "github.com/microcosm-cc/bluemonday"
+1 -1
View File
@@ -1,11 +1,11 @@
package cmd
import (
"log"
"path"
"time"
bolt "github.com/coreos/bbolt"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/go-pkgz/auth/avatar"
+1 -1
View File
@@ -4,11 +4,11 @@ import (
"context"
"fmt"
"io"
"log"
"net/http"
"os"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
+2 -1
View File
@@ -3,13 +3,14 @@ package cmd
import (
"encoding/json"
"fmt"
"log"
"math"
"net/http"
"strings"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
)
+1 -1
View File
@@ -5,7 +5,6 @@ package cmd
import (
"bytes"
"io/ioutil"
"log"
"net/http"
"os"
"path/filepath"
@@ -13,6 +12,7 @@ import (
"text/template"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
+1 -1
View File
@@ -6,12 +6,12 @@ import (
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"strings"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
+1 -1
View File
@@ -3,13 +3,13 @@ package cmd
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
log "github.com/go-pkgz/lgr"
flags "github.com/jessevdk/go-flags"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
+2 -1
View File
@@ -1,8 +1,9 @@
package cmd
import (
"log"
"time"
log "github.com/go-pkgz/lgr"
)
// RestoreCommand set of flags and command for restore from backup
+48 -26
View File
@@ -3,7 +3,6 @@ package cmd
import (
"context"
"fmt"
"log"
"net/http"
"net/url"
"os"
@@ -13,12 +12,13 @@ import (
"syscall"
"time"
bolt "github.com/coreos/bbolt"
"github.com/coreos/bbolt"
log "github.com/go-pkgz/lgr"
auth_cache "github.com/patrickmn/go-cache"
"github.com/pkg/errors"
"github.com/go-pkgz/auth"
"github.com/go-pkgz/auth/avatar"
"github.com/go-pkgz/auth/logger"
"github.com/go-pkgz/auth/provider"
"github.com/go-pkgz/auth/token"
"github.com/go-pkgz/mongo"
@@ -44,19 +44,21 @@ type ServerCommand struct {
Notify NotifyGroup `group:"notify" namespace:"notify" env-namespace:"NOTIFY"`
SSL SSLGroup `group:"ssl" namespace:"ssl" env-namespace:"SSL"`
Sites []string `long:"site" env:"SITE" default:"remark" description:"site names" env-delim:","`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" default:"" description:"admin basic auth password"`
BackupLocation string `long:"backup" env:"BACKUP_PATH" default:"./var/backup" description:"backups location"`
MaxBackupFiles int `long:"max-back" env:"MAX_BACKUP_FILES" default:"10" description:"max backups to keep"`
ImageProxy bool `long:"img-proxy" env:"IMG_PROXY" description:"enable image proxy"`
MaxCommentSize int `long:"max-comment" env:"MAX_COMMENT_SIZE" default:"2048" description:"max comment size"`
MaxVotes int `long:"max-votes" env:"MAX_VOTES" default:"-1" description:"maximum number of votes per comment"`
LowScore int `long:"low-score" env:"LOW_SCORE" default:"-5" description:"low score threshold"`
CriticalScore int `long:"critical-score" env:"CRITICAL_SCORE" default:"-10" description:"critical score threshold"`
ReadOnlyAge int `long:"read-age" env:"READONLY_AGE" default:"0" description:"read-only age of comments, days"`
EditDuration time.Duration `long:"edit-time" env:"EDIT_TIME" default:"5m" description:"edit window"`
Port int `long:"port" env:"REMARK_PORT" default:"8080" description:"port"`
WebRoot string `long:"web-root" env:"REMARK_WEB_ROOT" default:"./web" description:"web root directory"`
Sites []string `long:"site" env:"SITE" default:"remark" description:"site names" env-delim:","`
AdminPasswd string `long:"admin-passwd" env:"ADMIN_PASSWD" default:"" description:"admin basic auth password"`
BackupLocation string `long:"backup" env:"BACKUP_PATH" default:"./var/backup" description:"backups location"`
MaxBackupFiles int `long:"max-back" env:"MAX_BACKUP_FILES" default:"10" description:"max backups to keep"`
ImageProxy bool `long:"img-proxy" env:"IMG_PROXY" description:"enable image proxy"`
MaxCommentSize int `long:"max-comment" env:"MAX_COMMENT_SIZE" default:"2048" description:"max comment size"`
MaxVotes int `long:"max-votes" env:"MAX_VOTES" default:"-1" description:"maximum number of votes per comment"`
LowScore int `long:"low-score" env:"LOW_SCORE" default:"-5" description:"low score threshold"`
CriticalScore int `long:"critical-score" env:"CRITICAL_SCORE" default:"-10" description:"critical score threshold"`
ReadOnlyAge int `long:"read-age" env:"READONLY_AGE" default:"0" description:"read-only age of comments, days"`
EditDuration time.Duration `long:"edit-time" env:"EDIT_TIME" default:"5m" description:"edit window"`
Port int `long:"port" env:"REMARK_PORT" default:"8080" description:"port"`
WebRoot string `long:"web-root" env:"REMARK_WEB_ROOT" default:"./web" description:"web root directory"`
UpdateLimit float64 `long:"update-limit" env:"UPDATE_LIMIT" default:"0.5" description:"updates/sec limit"`
RestrictedWords []string `long:"restricted-words" env:"RESTRICTED_WORDS" default:"" description:"words prohibited to use in comments" env-delim:","`
Auth struct {
TTL struct {
@@ -170,16 +172,16 @@ func (s *ServerCommand) Execute(args []string) error {
stop := make(chan os.Signal, 1)
signal.Notify(stop, os.Interrupt, syscall.SIGTERM)
<-stop
log.Print("[WARN] interrupt signal")
log.Printf("[WARN] interrupt signal")
cancel()
}()
app, err := s.newServerApp()
if err != nil {
log.Fatalf("[ERROR] failed to setup application, %+v", err)
log.Printf("[PANIC] failed to setup application, %+v", err)
}
if err = app.run(ctx); err != nil {
log.Printf("[WARN] remark terminated with error %+v", err)
log.Printf("[ERROR] remark terminated with error %+v", err)
return err
}
log.Printf("[INFO] remark terminated")
@@ -210,12 +212,13 @@ func (s *ServerCommand) newServerApp() (*serverApp, error) {
}
dataService := &service.DataStore{
Interface: storeEngine,
EditDuration: s.EditDuration,
AdminStore: adminStore,
MaxCommentSize: s.MaxCommentSize,
MaxVotes: s.MaxVotes,
TitleExtractor: service.NewTitleExtractor(http.Client{Timeout: time.Second * 5}),
Interface: storeEngine,
EditDuration: s.EditDuration,
AdminStore: adminStore,
MaxCommentSize: s.MaxCommentSize,
MaxVotes: s.MaxVotes,
TitleExtractor: service.NewTitleExtractor(http.Client{Timeout: time.Second * 5}),
RestrictedWordsMatcher: service.NewRestrictedWordsMatcher(service.StaticRestrictedWordsLister{Words: s.RestrictedWords}),
}
loadingCache, err := s.makeCache()
@@ -268,6 +271,7 @@ func (s *ServerCommand) newServerApp() (*serverApp, error) {
Cache: loadingCache,
NotifyService: notifyService,
SSLConfig: sslConfig,
UpdateLimiter: s.UpdateLimit,
}
srv.ScoreThresholds.Low, srv.ScoreThresholds.Critical = s.LowScore, s.CriticalScore
@@ -550,8 +554,26 @@ func (s *ServerCommand) makeAuthenticator(ds *service.DataStore, avas avatar.Sto
AvatarStore: avas,
AvatarResizeLimit: s.Avatar.RszLmt,
AvatarRoutePath: "/api/v1/avatar",
Logger: logger.Std,
Logger: log.Default(),
RefreshCache: newAuthRefreshCache(),
})
s.addAuthProviders(authenticator)
return authenticator
}
// authRefreshCache used by authenticator to minimize repeatable token refreshes
type authRefreshCache struct {
*auth_cache.Cache
}
func newAuthRefreshCache() *authRefreshCache {
return &authRefreshCache{Cache: auth_cache.New(5*time.Minute, 10*time.Minute)}
}
func (c *authRefreshCache) Get(key interface{}) (interface{}, bool) {
return c.Cache.Get(key.(string))
}
func (c *authRefreshCache) Set(key, value interface{}) {
c.Cache.Set(key.(string), value, auth_cache.DefaultExpiration)
}
+2 -2
View File
@@ -5,7 +5,6 @@ import (
"crypto/tls"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"strings"
@@ -16,6 +15,7 @@ import (
jwt "github.com/dgrijalva/jwt-go"
"github.com/globalsign/mgo"
"github.com/go-pkgz/auth/token"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/mongo"
flags "github.com/jessevdk/go-flags"
@@ -377,7 +377,7 @@ func TestServerAuthHooks(t *testing.T) {
resp, err = client.Do(req)
require.Nil(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode, "blocked user can't post")
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode, "blocked user can't post")
app.Wait()
}
+4 -13
View File
@@ -2,13 +2,12 @@ package main
import (
"fmt"
"log"
"os"
"os/signal"
"runtime"
"syscall"
"github.com/hashicorp/logutils"
log "github.com/go-pkgz/lgr"
flags "github.com/jessevdk/go-flags"
"github.com/umputun/remark/backend/app/cmd"
@@ -62,19 +61,11 @@ func main() {
}
func setupLog(dbg bool) {
filter := &logutils.LevelFilter{
Levels: []logutils.LogLevel{"DEBUG", "INFO", "WARN", "ERROR"},
MinLevel: logutils.LogLevel("INFO"),
Writer: os.Stdout,
}
log.SetFlags(log.Ldate | log.Ltime)
if dbg {
log.SetFlags(log.Ldate | log.Ltime | log.Lmicroseconds | log.Lshortfile)
filter.MinLevel = logutils.LogLevel("DEBUG")
log.Setup(log.Debug, log.CallerFile, log.Msec, log.LevelBraces)
return
}
log.SetOutput(filter)
log.Setup(log.Msec, log.LevelBraces)
}
// getDump reads runtime stack and returns as a string
+3 -2
View File
@@ -2,7 +2,6 @@ package main
import (
"io/ioutil"
"log"
"net/http"
"os"
"strings"
@@ -11,6 +10,8 @@ import (
"testing"
"time"
log "github.com/go-pkgz/lgr"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -54,5 +55,5 @@ func TestGetDump(t *testing.T) {
assert.True(t, strings.Contains(dump, "goroutine"))
assert.True(t, strings.Contains(dump, "[running]"))
assert.True(t, strings.Contains(dump, "backend/app/main.go"))
log.Print("\n dump:" + dump)
log.Printf("\n dump: %s", dump)
}
+1 -1
View File
@@ -5,12 +5,12 @@ import (
"context"
"fmt"
"io/ioutil"
"log"
"os"
"sort"
"strings"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
)
+1 -1
View File
@@ -3,10 +3,10 @@ package migrator
import (
"encoding/xml"
"io"
"log"
"strings"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
+2 -2
View File
@@ -7,13 +7,13 @@ import (
"time"
bolt "github.com/coreos/bbolt"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/umputun/remark/backend/app/store"
"github.com/umputun/remark/backend/app/store/admin"
"github.com/umputun/remark/backend/app/store/engine"
"github.com/umputun/remark/backend/app/store/service"
"github.com/stretchr/testify/assert"
)
func TestDisqus_Import(t *testing.T) {
+1 -1
View File
@@ -5,9 +5,9 @@ package migrator
import (
"io"
"log"
"os"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
+1 -1
View File
@@ -4,8 +4,8 @@ import (
"bytes"
"encoding/json"
"io"
"log"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
+1 -1
View File
@@ -4,13 +4,13 @@ import (
"bytes"
"encoding/json"
"fmt"
"log"
"os"
"strings"
"testing"
"time"
bolt "github.com/coreos/bbolt"
log "github.com/go-pkgz/lgr"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
+1 -1
View File
@@ -4,9 +4,9 @@ import (
"encoding/xml"
"html"
"io"
"log"
"time"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
+2 -1
View File
@@ -4,10 +4,11 @@ package notify
import (
"context"
"fmt"
"log"
"sync"
"sync/atomic"
log "github.com/go-pkgz/lgr"
"github.com/umputun/remark/backend/app/store"
)
+2 -1
View File
@@ -4,14 +4,15 @@ import (
"context"
"errors"
"fmt"
"log"
"math/rand"
"sync"
"sync/atomic"
"testing"
"time"
log "github.com/go-pkgz/lgr"
"github.com/stretchr/testify/assert"
"github.com/umputun/remark/backend/app/store"
)
+17 -14
View File
@@ -6,38 +6,41 @@ import (
"encoding/json"
"fmt"
"html"
"log"
"net/http"
"strings"
"strconv"
"time"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/repeater"
"github.com/pkg/errors"
)
// Telegram implements notify.Destination for telegram
type Telegram struct {
channelName string
token string
apiPrefix string
timeout time.Duration
channelID string // unique identifier for the target chat or username of the target channel (in the format @channelusername)
token string
apiPrefix string
timeout time.Duration
}
const telegramTimeOut = 5000 * time.Millisecond
const telegramAPIPrefix = "https://api.telegram.org/bot"
// NewTelegram makes telegram bot for notifications
func NewTelegram(token string, channelName string, timeout time.Duration, api string) (*Telegram, error) {
func NewTelegram(token string, channelID string, timeout time.Duration, api string) (*Telegram, error) {
res := Telegram{channelName: channelName, token: token, apiPrefix: api, timeout: timeout}
res.channelName = strings.TrimPrefix(res.channelName, "@")
if _, err := strconv.ParseInt(channelID, 10, 64); err != nil {
channelID = "@" + channelID // if channelID not a number enforce @ prefix
}
res := Telegram{channelID: channelID, token: token, apiPrefix: api, timeout: timeout}
if res.apiPrefix == "" {
res.apiPrefix = telegramAPIPrefix
}
if res.timeout == 0 {
res.timeout = telegramTimeOut
}
log.Printf("[DEBUG] create new telegram notifier for cham %s, timeout=%s, api=%s", channelName, res.timeout, res.timeout)
log.Printf("[DEBUG] create new telegram notifier for cham %s, timeout=%s, api=%s", channelID, res.timeout, res.timeout)
err := repeater.NewDefault(5, time.Millisecond*250).Do(func() error {
client := http.Client{Timeout: telegramTimeOut}
@@ -81,7 +84,7 @@ func NewTelegram(token string, channelName string, timeout time.Duration, api st
// Send to telegram channel
func (t *Telegram) Send(ctx context.Context, req request) error {
client := http.Client{Timeout: telegramTimeOut}
log.Printf("[DEBUG] send telegram notification to %s, comment id %s", t.channelName, req.comment.ID)
log.Printf("[DEBUG] send telegram notification to %s, comment id %s", t.channelID, req.comment.ID)
from := req.comment.User.Name
if req.comment.ParentID != "" {
@@ -92,8 +95,8 @@ func (t *Telegram) Send(ctx context.Context, req request) error {
if req.comment.PostTitle != "" {
link = fmt.Sprintf("↦ [%s](%s)", req.comment.PostTitle, req.comment.Locator.URL+uiNav+req.comment.ID)
}
u := fmt.Sprintf("%s%s/sendMessage?chat_id=@%s&parse_mode=Markdown&disable_web_page_preview=true",
t.apiPrefix, t.token, t.channelName)
u := fmt.Sprintf("%s%s/sendMessage?chat_id=%s&parse_mode=Markdown&disable_web_page_preview=true",
t.apiPrefix, t.token, t.channelID)
msg := fmt.Sprintf("%s\n\n%s\n\n%s", from, req.comment.Orig, link)
msg = html.UnescapeString(msg)
@@ -138,5 +141,5 @@ func (t *Telegram) Send(ctx context.Context, req request) error {
}
func (t *Telegram) String() string {
return "telegram: " + t.channelName
return "telegram: " + t.channelID
}
+22 -1
View File
@@ -9,6 +9,7 @@ import (
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/umputun/remark/backend/app/store"
)
@@ -20,6 +21,7 @@ func TestTelegram_New(t *testing.T) {
tb, err := NewTelegram("good-token", "remark_test", 2*time.Second, ts.URL+"/")
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, "@remark_test", tb.channelID, "@ added")
st := time.Now()
_, err = NewTelegram("bad-resp", "remark_test", 2*time.Second, ts.URL+"/")
@@ -35,6 +37,17 @@ func TestTelegram_New(t *testing.T) {
_, err = NewTelegram("no-such-thing", "remark_test", 2*time.Second, "http://127.0.0.1:4321/")
assert.EqualError(t, err, "can't initialize telegram notifications: Get http://127.0.0.1:4321/no-such-thing/getMe: dial tcp 127.0.0.1:4321: connect: connection refused")
_, err = NewTelegram("good-token", "remark_test", 2*time.Second, "")
assert.Error(t, err, "empty api url not allowed")
_, err = NewTelegram("good-token", "remark_test", 0, ts.URL+"/")
assert.NoError(t, err, "0 timeout allowed as default")
tb, err = NewTelegram("good-token", "1234567890", 2*time.Second, ts.URL+"/")
assert.NoError(t, err)
assert.NotNil(t, tb)
assert.Equal(t, "1234567890", tb.channelID, "no @ prefix")
}
func TestTelegram_Send(t *testing.T) {
@@ -49,10 +62,18 @@ func TestTelegram_Send(t *testing.T) {
cp := store.Comment{Text: "some parent text"}
cp.User.Name = "to"
err = tb.Send(context.TODO(), request{comment: c, parent: cp})
assert.NoError(t, err)
c.PostTitle = "test title"
err = tb.Send(context.TODO(), request{comment: c, parent: cp})
assert.NoError(t, err)
assert.Equal(t, "telegram: remark_test", tb.String())
tb, err = NewTelegram("non-json-resp", "remark_test", 2*time.Second, ts.URL+"/")
assert.NotNil(t, err, "should failed")
err = tb.Send(context.TODO(), request{comment: c, parent: cp})
assert.Contains(t, err.Error(), "unexpected telegram status code 404", "send on broken tg")
assert.Equal(t, "telegram: @remark_test", tb.String())
}
func mockTelegramServer() *httptest.Server {
+1 -2
View File
@@ -2,15 +2,14 @@ package api
import (
"errors"
"log"
"net/http"
"path"
"time"
"github.com/go-chi/chi"
"github.com/go-chi/render"
"github.com/go-pkgz/auth"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/go-pkgz/rest/cache"
+1 -1
View File
@@ -640,7 +640,7 @@ func TestAdmin_DeleteMeRequestFailed(t *testing.T) {
req.SetBasicAuth("admin", "bad-password")
resp, err = client.Do(req)
assert.Nil(t, err)
assert.Equal(t, 401, resp.StatusCode)
assert.Equal(t, 403, resp.StatusCode)
// try bad user
badClaims := claims
+1 -2
View File
@@ -6,7 +6,6 @@ import (
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"sync"
@@ -14,7 +13,7 @@ import (
"github.com/go-chi/chi"
"github.com/go-chi/render"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/go-pkgz/rest/cache"
"github.com/pkg/errors"
+1 -2
View File
@@ -15,11 +15,10 @@ import (
"testing"
"time"
"github.com/go-pkgz/auth/token"
bolt "github.com/coreos/bbolt"
"github.com/go-chi/chi"
"github.com/go-pkgz/auth"
"github.com/go-pkgz/auth/token"
"github.com/go-pkgz/rest/cache"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
+27 -12
View File
@@ -6,7 +6,6 @@ import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"path"
"strings"
@@ -19,13 +18,13 @@ import (
"github.com/go-chi/chi/middleware"
"github.com/go-chi/cors"
"github.com/go-chi/render"
"github.com/pkg/errors"
"github.com/rakyll/statik/fs"
"github.com/go-pkgz/auth"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/go-pkgz/rest/cache"
"github.com/go-pkgz/rest/logger"
"github.com/pkg/errors"
"github.com/rakyll/statik/fs"
"github.com/umputun/remark/backend/app/notify"
"github.com/umputun/remark/backend/app/rest"
@@ -54,6 +53,7 @@ type Rest struct {
Low int
Critical int
}
UpdateLimiter float64
SSLConfig SSLConfig
httpsServer *http.Server
@@ -156,7 +156,7 @@ func (s *Rest) makeHTTPServer(port int, router http.Handler) *http.Server {
func (s *Rest) routes() chi.Router {
router := chi.NewRouter()
router.Use(middleware.RealIP, R.Recoverer)
router.Use(middleware.RealIP, R.Recoverer(log.Default()))
router.Use(middleware.Throttle(1000), middleware.Timeout(60*time.Second))
router.Use(R.AppInfo("remark42", "umputun", s.Version), R.Ping)
@@ -183,7 +183,7 @@ func (s *Rest) routes() chi.Router {
authHandler, avatarHandler := s.Authenticator.Handlers()
router.Group(func(r chi.Router) {
l := logger.New(logger.Flags(logger.All), logger.IPfn(ipFn), logger.Prefix("[INFO]"))
l := logger.New(logger.Flags(logger.All), logger.Log(log.Default()), logger.IPfn(ipFn), logger.Prefix("[INFO]"))
r.Use(l.Handler, tollbooth_chi.LimitHandler(tollbooth.NewLimiter(5, nil)))
r.Mount("/auth", authHandler)
})
@@ -207,7 +207,8 @@ func (s *Rest) routes() chi.Router {
rapi.Group(func(ropen chi.Router) {
ropen.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
ropen.Use(authMiddleware.Trace)
ropen.Use(logger.New(logger.Flags(logger.All), logger.Prefix("[INFO]"), logger.IPfn(ipFn)).Handler)
ropen.Use(logger.New(logger.Flags(logger.All), logger.Log(log.Default()),
logger.Prefix("[INFO]"), logger.IPfn(ipFn)).Handler)
ropen.Get("/find", s.findCommentsCtrl)
ropen.Get("/id/{id}", s.commentByIDCtrl)
ropen.Get("/comments", s.findUserCommentsCtrl)
@@ -227,17 +228,31 @@ func (s *Rest) routes() chi.Router {
rapi.Group(func(rauth chi.Router) {
rauth.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(10, nil)))
rauth.Use(authMiddleware.Auth)
rauth.Use(logger.New(logger.Flags(logger.All), logger.Prefix("[INFO]"), logger.IPfn(ipFn)).Handler)
rauth.Post("/comment", s.createCommentCtrl)
rauth.Put("/comment/{id}", s.updateCommentCtrl)
rauth.Use(logger.New(logger.Flags(logger.All), logger.Log(log.Default()),
logger.Prefix("[INFO]"), logger.IPfn(ipFn)).Handler)
rauth.Get("/user", s.userInfoCtrl)
rauth.Put("/vote/{id}", s.voteCtrl)
rauth.Get("/userdata", s.userAllDataCtrl)
rauth.Post("/deleteme", s.deleteMeCtrl)
// admin routes, admin users only
rauth.Mount("/admin", s.adminService.routes(authMiddleware.AdminOnly))
})
// protected routes, throttled to 10/s by default, controlled by external UpdateLimiter param
rapi.Group(func(rauth chi.Router) {
lmt := 10.0
if s.UpdateLimiter > 0 {
lmt = s.UpdateLimiter
}
rauth.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(lmt, nil)))
rauth.Use(authMiddleware.Auth)
rauth.Use(logger.New(logger.Flags(logger.All), logger.Log(log.Default()),
logger.Prefix("[DEBUG]"), logger.IPfn(ipFn)).Handler)
rauth.Put("/comment/{id}", s.updateCommentCtrl)
rauth.Post("/comment", s.createCommentCtrl)
rauth.Put("/vote/{id}", s.voteCtrl)
rauth.Post("/deleteme", s.deleteMeCtrl)
})
})
// respond to /robots.txt with the list of allowed paths
+10 -3
View File
@@ -5,7 +5,6 @@ import (
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"strings"
"time"
@@ -13,11 +12,11 @@ import (
jwt "github.com/dgrijalva/jwt-go"
"github.com/go-chi/chi"
"github.com/go-chi/render"
multierror "github.com/hashicorp/go-multierror"
"github.com/go-pkgz/auth/token"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/go-pkgz/rest/cache"
multierror "github.com/hashicorp/go-multierror"
"github.com/umputun/remark/backend/app/rest"
"github.com/umputun/remark/backend/app/store"
@@ -58,6 +57,10 @@ func (s *Rest) createCommentCtrl(w http.ResponseWriter, r *http.Request) {
}
id, err := s.DataService.Create(comment)
if err == service.ErrRestrictedWordsFound {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "invalid comment")
return
}
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "can't save comment")
return
@@ -122,6 +125,10 @@ func (s *Rest) updateCommentCtrl(w http.ResponseWriter, r *http.Request) {
}
res, err := s.DataService.EditComment(locator, id, editReq)
if err == service.ErrRestrictedWordsFound {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "invalid comment")
return
}
if err != nil {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't update comment")
return
+43
View File
@@ -101,6 +101,24 @@ func TestRest_CreateTooBig(t *testing.T) {
assert.Equal(t, "can't bind comment", c["details"])
}
func TestRest_CreateWithRestrictedWord(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
badComment := fmt.Sprintf(`{"text": "What the duck is that?", "locator":{"url": "https://radio-t.com/blah1", "site": "radio-t"}}`)
resp, err := post(t, ts.URL+"/api/v1/comment", badComment)
assert.Nil(t, err)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
b, err := ioutil.ReadAll(resp.Body)
assert.Nil(t, err)
c := R.JSON{}
err = json.Unmarshal(b, &c)
assert.Nil(t, err)
assert.Equal(t, "comment contains restricted words", c["error"])
assert.Equal(t, "invalid comment", c["details"])
}
func TestRest_CreateRejected(t *testing.T) {
ts, _, teardown := startupT(t)
@@ -258,6 +276,31 @@ func TestRest_UpdateNotOwner(t *testing.T) {
assert.Equal(t, 400, b.StatusCode, string(body), "update is not json")
}
func TestRest_UpdateWithRestrictedWords(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
c1 := store.Comment{Text: "What the quack is that?", ParentID: "p1",
Locator: store.Locator{SiteID: "radio-t", URL: "https://radio-t.com/blah1"}}
id := addComment(t, c1, ts)
client := http.Client{}
req, err := http.NewRequest(http.MethodPut, ts.URL+"/api/v1/comment/"+id+"?site=radio-t&url=https://radio-t.com/blah1",
strings.NewReader(`{"text":"What the duck is that?", "summary":"my edit"}`))
assert.Nil(t, err)
req.Header.Add("X-JWT", devToken)
b, err := client.Do(req)
assert.Nil(t, err)
body, err := ioutil.ReadAll(b.Body)
assert.Nil(t, err)
c := R.JSON{}
err = json.Unmarshal(body, &c)
assert.Nil(t, err)
assert.Equal(t, 400, b.StatusCode, string(body))
assert.Equal(t, "comment contains restricted words", c["error"])
assert.Equal(t, "invalid comment", c["details"])
}
func TestRest_Vote(t *testing.T) {
ts, _, teardown := startupT(t)
defer teardown()
+1 -2
View File
@@ -3,14 +3,13 @@ package api
import (
"crypto/sha1"
"encoding/base64"
"log"
"net/http"
"strconv"
"strings"
"github.com/go-chi/chi"
"github.com/go-chi/render"
log "github.com/go-pkgz/lgr"
R "github.com/go-pkgz/rest"
"github.com/go-pkgz/rest/cache"
+7 -12
View File
@@ -180,13 +180,15 @@ func startupT(t *testing.T) (ts *httptest.Server, srv *Rest, teardown func()) {
require.Nil(t, err)
adminStore := adminstore.NewStaticStore("123456", []string{"a1", "a2"}, "admin@remark-42.com")
restrictedWordsMatcher := service.NewRestrictedWordsMatcher(service.StaticRestrictedWordsLister{Words: []string{"duck"}})
dataStore := &service.DataStore{
Interface: b,
EditDuration: 5 * time.Minute,
MaxCommentSize: 4000,
AdminStore: adminStore,
MaxVotes: service.UnlimitedVotes,
Interface: b,
EditDuration: 5 * time.Minute,
MaxCommentSize: 4000,
AdminStore: adminStore,
MaxVotes: service.UnlimitedVotes,
RestrictedWordsMatcher: restrictedWordsMatcher,
}
srv = &Rest{
@@ -292,10 +294,3 @@ func addComment(t *testing.T, c store.Comment, ts *httptest.Server) string {
time.Sleep(time.Nanosecond * 10)
return crResp["id"].(string)
}
func cleanup(ts *httptest.Server, srv *Rest) {
ts.Close()
srv.DataService.Close()
os.Remove(testDb)
os.Remove(testHTML)
}
+3 -3
View File
@@ -2,16 +2,15 @@ package api
import (
"fmt"
"log"
"net/http"
"time"
"github.com/go-chi/chi"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/rest/cache"
"github.com/gorilla/feeds"
"github.com/pkg/errors"
"github.com/go-pkgz/rest/cache"
"github.com/umputun/remark/backend/app/rest"
"github.com/umputun/remark/backend/app/store"
)
@@ -165,6 +164,7 @@ func (s *Rest) toRssFeed(url string, comments []store.Comment) (string, error) {
Description: c.Text,
Created: c.Timestamp,
Author: &feeds.Author{Name: c.User.Name},
Id: c.ID,
}
if c.ParentID != "" {
// add indication to parent comment
+76 -38
View File
@@ -8,20 +8,26 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/umputun/remark/backend/app/store"
)
func TestServer_RssPost(t *testing.T) {
ts, _, teardown := startupT(t)
ts, rst, teardown := startupT(t)
defer teardown()
waitOnSecChange()
c1 := store.Comment{
ID: "1234567890",
Text: "test 123",
Locator: store.Locator{URL: "https://radio-t.com/blah1", SiteID: "radio-t"},
User: store.User{ID: "u1", Name: "developer one"},
}
id1 := addComment(t, c1, ts)
id1, err := rst.DataService.Create(c1)
require.NoError(t, err)
assert.Equal(t, "1234567890", id1)
pubDate := time.Now().Format(time.RFC1123Z)
res, code := get(t, ts.URL+"/api/v1/rss/post?site=radio-t&url=https://radio-t.com/blah1")
@@ -36,13 +42,14 @@ func TestServer_RssPost(t *testing.T) {
<pubDate>%s</pubDate>
<item>
<title>developer one</title>
<link>https://radio-t.com/blah1#remark42__comment-%s</link>
<description>&lt;p&gt;test 123&lt;/p&gt;&#xA;</description>
<link>https://radio-t.com/blah1#remark42__comment-1234567890</link>
<description>test 123</description>
<author>developer one</author>
<guid>1234567890</guid>
<pubDate>%s</pubDate>
</item>
</channel>
</rss>`, pubDate, id1, pubDate)
</rss>`, pubDate, pubDate)
expected, res = cleanRssFormatting(expected, res)
assert.Equal(t, expected, res)
@@ -52,7 +59,7 @@ func TestServer_RssPost(t *testing.T) {
}
func TestServer_RssSite(t *testing.T) {
ts, _, teardown := startupT(t)
ts, rst, teardown := startupT(t)
defer teardown()
waitOnSecChange()
@@ -60,16 +67,24 @@ func TestServer_RssSite(t *testing.T) {
pubDate := time.Now().Format(time.RFC1123Z)
c1 := store.Comment{
ID: "comment-id-1",
Text: "test 123",
Locator: store.Locator{URL: "https://radio-t.com/blah10", SiteID: "radio-t"},
User: store.User{ID: "u1", Name: "developer one"},
}
c2 := store.Comment{
ID: "comment-id-2",
Text: "xyz test",
Locator: store.Locator{URL: "https://radio-t.com/blah11", SiteID: "radio-t"},
User: store.User{ID: "u1", Name: "developer one"},
}
id1 := addComment(t, c1, ts)
id2 := addComment(t, c2, ts)
_, err := rst.DataService.Create(c1)
require.NoError(t, err)
_, err = rst.DataService.Create(c2)
require.NoError(t, err)
require.NoError(t, err)
res, code := get(t, ts.URL+"/api/v1/rss/site?site=radio-t")
assert.Equal(t, 200, code)
t.Log(res)
@@ -82,20 +97,22 @@ func TestServer_RssSite(t *testing.T) {
<pubDate>%s</pubDate>
<item>
<title>developer one</title>
<link>https://radio-t.com/blah11#remark42__comment-%s</link>
<description>&lt;p&gt;xyz test&lt;/p&gt;&#xA;</description>
<link>https://radio-t.com/blah11#remark42__comment-comment-id-2</link>
<description>xyz test</description>
<author>developer one</author>
<guid>comment-id-2</guid>
<pubDate>%s</pubDate>
</item>
<item>
<title>developer one</title>
<link>https://radio-t.com/blah10#remark42__comment-%s</link>
<description>&lt;p&gt;test 123&lt;/p&gt;&#xA;</description>
<link>https://radio-t.com/blah10#remark42__comment-comment-id-1</link>
<description>test 123</description>
<author>developer one</author>
<guid>comment-id-1</guid>
<pubDate>%s</pubDate>
</item>
</channel>
</rss>`, pubDate, id2, pubDate, id1, pubDate)
</rss>`, pubDate, pubDate, pubDate)
expected, res = cleanRssFormatting(expected, res)
assert.Equal(t, expected, res)
@@ -105,7 +122,7 @@ func TestServer_RssSite(t *testing.T) {
}
func TestServer_RssWithReply(t *testing.T) {
ts, _, teardown := startupT(t)
ts, rst, teardown := startupT(t)
defer teardown()
waitOnSecChange()
@@ -113,16 +130,23 @@ func TestServer_RssWithReply(t *testing.T) {
pubDate := time.Now().Format(time.RFC1123Z)
c1 := store.Comment{
ID: "comment-id-1",
Text: "test 123",
Locator: store.Locator{URL: "https://radio-t.com/blah10", SiteID: "radio-t"},
User: store.User{ID: "u1", Name: "developer one"},
}
c2 := store.Comment{
Text: "xyz test",
Locator: store.Locator{URL: "https://radio-t.com/blah10", SiteID: "radio-t"},
ID: "comment-id-2",
ParentID: "comment-id-1",
Text: "xyz test",
Locator: store.Locator{URL: "https://radio-t.com/blah10", SiteID: "radio-t"},
User: store.User{ID: "u1", Name: "developer one"},
}
id1 := addComment(t, c1, ts)
c2.ParentID = id1
id2 := addComment(t, c2, ts)
_, err := rst.DataService.Create(c1)
require.NoError(t, err)
_, err = rst.DataService.Create(c2)
require.NoError(t, err)
res, code := get(t, ts.URL+"/api/v1/rss/post?site=radio-t&url=https://radio-t.com/blah10")
assert.Equal(t, 200, code)
@@ -136,20 +160,22 @@ func TestServer_RssWithReply(t *testing.T) {
<pubDate>%s</pubDate>
<item>
<title>developer one &gt; developer one</title>
<link>https://radio-t.com/blah10#remark42__comment-%s</link>
<description>&lt;p&gt;xyz test&lt;/p&gt;&#xA;</description>
<link>https://radio-t.com/blah10#remark42__comment-comment-id-2</link>
<description>xyz test</description>
<author>developer one</author>
<guid>comment-id-2</guid>
<pubDate>%s</pubDate>
</item>
<item>
<title>developer one</title>
<link>https://radio-t.com/blah10#remark42__comment-%s</link>
<description>&lt;p&gt;test 123&lt;/p&gt;&#xA;</description>
<link>https://radio-t.com/blah10#remark42__comment-comment-id-1</link>
<description>test 123</description>
<author>developer one</author>
<guid>comment-id-1</guid>
<pubDate>%s</pubDate>
</item>
</channel>
</rss>`, pubDate, id2, pubDate, id1, pubDate)
</rss>`, pubDate, pubDate, pubDate)
expected, res = cleanRssFormatting(expected, res)
assert.Equal(t, expected, res)
@@ -164,39 +190,49 @@ func TestServer_RssReplies(t *testing.T) {
pubDate := time.Now().Format(time.RFC1123Z)
c1 := store.Comment{
ID: "comment-1",
Text: "c1",
Locator: store.Locator{URL: "https://radio-t.com/blah1", SiteID: "radio-t"},
User: store.User{ID: "user1", Name: "user1"},
}
id1, err := srv.DataService.Create(c1)
assert.Nil(t, err)
c2 := store.Comment{
ID: "comment-2",
Text: "reply to c1 from user2",
ParentID: id1,
ParentID: "comment-1",
Locator: store.Locator{URL: "https://radio-t.com/blah1", SiteID: "radio-t"},
User: store.User{ID: "user2", Name: "user2"},
}
id2, err := srv.DataService.Create(c2)
assert.Nil(t, err)
c3 := store.Comment{
ID: "comment-3",
Text: "reply to c1 from user3",
ParentID: id1,
ParentID: "comment-1",
Locator: store.Locator{URL: "https://radio-t.com/blah1", SiteID: "radio-t"},
User: store.User{ID: "user3", Name: "user3"},
}
id3, err := srv.DataService.Create(c3)
assert.Nil(t, err)
c4 := store.Comment{
ID: "comment-4",
Text: "reply to c2 from developer one",
ParentID: id2,
ParentID: "comment-2",
Locator: store.Locator{URL: "https://radio-t.com/blah1", SiteID: "radio-t"},
User: store.User{ID: "dev", Name: "developer one"},
}
addComment(t, c4, ts)
c5 := store.Comment{
ID: "comment-5",
Text: "developer one",
Locator: store.Locator{URL: "https://radio-t.com/blah1", SiteID: "radio-t"},
User: store.User{ID: "dev", Name: "developer one"},
}
addComment(t, c5, ts)
_, err := srv.DataService.Create(c1)
require.NoError(t, err)
_, err = srv.DataService.Create(c2)
require.NoError(t, err)
_, err = srv.DataService.Create(c3)
require.NoError(t, err)
_, err = srv.DataService.Create(c4)
require.NoError(t, err)
_, err = srv.DataService.Create(c5)
require.NoError(t, err)
// replies to c1 (user1). Must be [c3, c2]
res, code := get(t, ts.URL+"/api/v1/rss/reply?user=user1&site=radio-t")
@@ -210,20 +246,22 @@ func TestServer_RssReplies(t *testing.T) {
<pubDate>%s</pubDate>
<item>
<title>user3 &gt; user1</title>
<link>https://radio-t.com/blah1#remark42__comment-%s</link>
<link>https://radio-t.com/blah1#remark42__comment-comment-3</link>
<description>reply to c1 from user3</description>
<author>user3</author>
<guid>comment-3</guid>
<pubDate>%s</pubDate>
</item>
<item>
<title>user2 &gt; user1</title>
<link>https://radio-t.com/blah1#remark42__comment-%s</link>
<link>https://radio-t.com/blah1#remark42__comment-comment-2</link>
<description>reply to c1 from user2</description>
<author>user2</author>
<guid>comment-2</guid>
<pubDate>%s</pubDate>
</item>
</channel>
</rss>`, pubDate, id3, pubDate, id2, pubDate)
</rss>`, pubDate, pubDate, pubDate)
expected, res = cleanRssFormatting(expected, res)
assert.Equal(t, expected, res)
+3 -3
View File
@@ -2,13 +2,13 @@ package api
import (
"crypto/tls"
"log"
"net/http"
"net/url"
"time"
"github.com/go-chi/chi"
"github.com/go-chi/chi/middleware"
log "github.com/go-pkgz/lgr"
"golang.org/x/crypto/acme/autocert"
R "github.com/go-pkgz/rest"
@@ -43,7 +43,7 @@ type SSLConfig struct {
func (s *Rest) httpToHTTPSRouter() chi.Router {
log.Printf("[DEBUG] create https-to-http redirect routes")
router := chi.NewRouter()
router.Use(middleware.RealIP, R.Recoverer)
router.Use(middleware.RealIP, R.Recoverer(log.Default()))
router.Use(middleware.Throttle(1000), middleware.Timeout(60*time.Second))
router.Handle("/*", s.redirectHandler())
@@ -57,7 +57,7 @@ func (s *Rest) httpToHTTPSRouter() chi.Router {
func (s *Rest) httpChallengeRouter(m *autocert.Manager) chi.Router {
log.Printf("[DEBUG] create http-challenge routes")
router := chi.NewRouter()
router.Use(middleware.RealIP, R.Recoverer)
router.Use(middleware.RealIP, R.Recoverer(log.Default()))
router.Use(middleware.Throttle(1000), middleware.Timeout(60*time.Second))
router.Handle("/*", m.HTTPHandler(s.redirectHandler()))
+1 -1
View File
@@ -2,13 +2,13 @@ package rest
import (
"fmt"
"log"
"net/http"
"net/url"
"runtime"
"strings"
"github.com/go-chi/render"
log "github.com/go-pkgz/lgr"
)
// SendErrorJSON makes {error: blah, details: blah} json body and responds with error code
+2 -3
View File
@@ -3,16 +3,15 @@ package proxy
import (
"encoding/base64"
"io"
"log"
"net/http"
"strings"
"time"
"github.com/PuerkitoBio/goquery"
"github.com/go-chi/chi"
"github.com/pkg/errors"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/repeater"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/rest"
)
+1 -1
View File
@@ -3,10 +3,10 @@ package rest
import (
"encoding/json"
"io/ioutil"
"log"
"testing"
"time"
log "github.com/go-pkgz/lgr"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
+2 -1
View File
@@ -3,7 +3,8 @@ package admin
import (
"errors"
"log"
log "github.com/go-pkgz/lgr"
)
// Store defines interface returning admins info for given site
+1 -2
View File
@@ -1,10 +1,9 @@
package admin
import (
"log"
"github.com/globalsign/mgo"
"github.com/globalsign/mgo/bson"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/mongo"
)
+2 -2
View File
@@ -3,11 +3,11 @@ package engine
import (
"encoding/json"
"fmt"
"log"
"strings"
"time"
bolt "github.com/coreos/bbolt"
log "github.com/go-pkgz/lgr"
multierror "github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
@@ -127,7 +127,7 @@ func (b *BoltDB) Create(comment store.Comment) (commentID string, err error) {
return errors.Wrapf(e, "failed to put user comment %s for %s", comment.ID, comment.User.ID)
}
// set info with countfor post url
// set info with the count for post url
if _, e = b.setInfo(tx, comment); e != nil {
return errors.Wrapf(e, "failed to set info for %s", comment.Locator)
}
+1 -1
View File
@@ -2,10 +2,10 @@ package engine
import (
"encoding/json"
"log"
"time"
bolt "github.com/coreos/bbolt"
log "github.com/go-pkgz/lgr"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
+1 -2
View File
@@ -5,11 +5,10 @@ import (
"github.com/globalsign/mgo"
"github.com/globalsign/mgo/bson"
"github.com/go-pkgz/mongo"
multierror "github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"github.com/go-pkgz/mongo"
"github.com/umputun/remark/backend/app/store"
)
@@ -0,0 +1,176 @@
package service
import (
"strings"
"unicode"
"unicode/utf8"
log "github.com/go-pkgz/lgr"
)
// RestrictedWordsLister provides restricted words in comments per site
type RestrictedWordsLister interface {
List(siteID string) (restricted []string, err error)
}
// StaticRestrictedWordsLister provides same restricted words in comments for every site
type StaticRestrictedWordsLister struct {
Words []string
}
// List provides restricted words in comments (ignores siteID)
func (l StaticRestrictedWordsLister) List(siteID string) (restricted []string, err error) {
return l.Words, nil
}
// RestrictedWordsMatcher matches comment text against restricted words
type RestrictedWordsMatcher struct {
lister RestrictedWordsLister
}
// NewRestrictedWordsMatcher creates new RestrictedWordsMatcher using provided RestrictedWordsLister
func NewRestrictedWordsMatcher(lister RestrictedWordsLister) *RestrictedWordsMatcher {
return &RestrictedWordsMatcher{lister: lister}
}
// Match matches comment text against restricted words for specified site
func (m *RestrictedWordsMatcher) Match(siteID string, text string) bool {
restrictedWords, err := m.lister.List(siteID)
if err != nil {
log.Printf("[WARN] failed to get restricted patterns for site %s: %v", siteID, err)
return false
}
if len(restrictedWords) == 0 {
return false
}
tokens := m.tokenize(text)
trie := newWildcardTrie(restrictedWords...)
for _, token := range tokens {
if trie.check(token) {
return true
}
}
return false
}
func (m *RestrictedWordsMatcher) tokenize(text string) []string {
tokens := make([]string, 0, 10) // accumulator for tokens
word := false // flag shows if current range is word
start := 0 // beginning of the current range
for pos, r := range text {
if unicode.IsLetter(r) || unicode.IsNumber(r) {
if !word {
// everything from start to pos - 1 is not a word, so reset start and start word tracking
start = pos
word = true
}
continue
}
if word && start < pos {
// everything from start to pos - 1 is a word, so add it as a token and reset start
tokens = append(tokens, strings.ToLower(text[start:pos]))
start = pos
}
// exited the word
word = false
}
// since we append tokens when we already left the word (on next iteration),
// we need to do it manually for the last iteration
if word {
tokens = append(tokens, strings.ToLower(text[start:]))
}
return tokens
}
type wildcardTrie struct {
terminal bool
children map[rune]*wildcardTrie
}
func newWildcardTrie(patterns ...string) *wildcardTrie {
trie := &wildcardTrie{terminal: false, children: make(map[rune]*wildcardTrie)}
for _, p := range patterns {
trie.addPattern(p)
}
return trie
}
func (trie *wildcardTrie) addPattern(pattern string) {
// since pattern matching algorithm is recursive we do not allow long patterns
if utf8.RuneCountInString(pattern) < 1 || utf8.RuneCountInString(pattern) > 64 {
log.Printf("[WARN] invalid pattern length '%s': actual - %d, min allowed - 1, max allowed - 64", pattern, utf8.RuneCountInString(pattern))
return
}
node := trie
for _, r := range strings.ToLower(strings.TrimSpace(pattern)) {
if childNode, exists := node.children[r]; exists {
node = childNode
continue
}
childNode := newWildcardTrie()
node.children[r] = childNode
node = childNode
}
node.terminal = true
}
// check tests if any pattern stored in trie matches the token. Recursive. Max depth is longest pattern in trie.
func (trie *wildcardTrie) check(token string) bool {
if len(token) == 0 {
if trie.terminal {
return true
}
if childNode, exists := trie.children['*']; exists && childNode.terminal {
return true
}
return false
}
r, width := utf8.DecodeRuneInString(token)
if childNode, exists := trie.children[r]; exists {
if childNode.check(token[width:]) {
return true
}
}
if childNode, exists := trie.children['*']; exists {
if childNode.terminal {
return true
}
if childNode.checkAllSuffixes(token) {
return true
}
}
return false
}
func (trie *wildcardTrie) checkAllSuffixes(token string) bool {
suffix := token
for {
if len(suffix) == 0 {
return false
}
if trie.check(suffix) {
return true
}
_, width := utf8.DecodeRuneInString(suffix)
suffix = suffix[width:]
}
}
@@ -0,0 +1,76 @@
package service
import (
"github.com/stretchr/testify/assert"
"testing"
)
func TestMatcher_Tokenize(t *testing.T) {
matcher := NewRestrictedWordsMatcher(StaticRestrictedWordsLister{})
tbl := []struct {
input string
output []string
}{
{
" word0 word1 word2, word3,,, !word4 !word5? word6-word7 word8",
[]string{"word0", "word1", "word2", "word3", "word4", "word5", "word6", "word7", "word8"},
},
{"русский 中文 française ไทย", []string{"русский", "中文", "française", "ไทย"}},
{"word", []string{"word"}},
{"", []string{}},
{"\t\t\n\t \n\t \r\n \t ,,, !#$%^&*()", []string{}},
{"👍", []string{}},
}
for _, td := range tbl {
tokens := matcher.tokenize(td.input)
assert.Equal(t, td.output, tokens, "unexpected result for input '%v'", td.input)
}
}
func TestWildcardTrie_Check(t *testing.T) {
tbl := []struct {
input []string
match []string
nomatch []string
}{
{[]string{"abc", "abb", "aab"}, []string{"abc", "abb", "aab"}, []string{"aaa", "aaaa", "a", "ab"}},
{[]string{"abc", "*ck", "*z"}, []string{"abc", "duck", "quack", "ck", "xyz"}, []string{"quacker", "buzzer"}},
{[]string{"abc", "du*", "c*"}, []string{"abc", "duck", "dungeon", "du", "cup"}, []string{"bbc", "ddu", "scuba"}},
{[]string{"abc", "*uc*", "*x*"}, []string{"abc", "duck", "stuck", "uc", "wwxww", "xww", "wwx"}, []string{"bbc", "duke"}},
{[]string{"abc", "d*k", "st*ck"}, []string{"abc", "duck", "dk", "stck", "stuck", "stiiick"}, []string{"bbc", "adka", "st", "ck"}},
{[]string{"abc", "*a*a*"}, []string{"abc", "safari", "banana", "aa"}, []string{"bbc", "car", "a"}},
{
[]string{"ложить", "при*", "*ий", "*бег*", "про*жа", "*ไ*ย*", "*請*请*"},
[]string{"ложить", "приклад", "ихний", "прибегать", "пропажа", "ไทย", "ทไย", "ไยท", "請問请问"},
[]string{"положить", "гранпри", "бийск", "請", "ยไท"},
},
}
for _, td := range tbl {
n := newWildcardTrie(td.input...)
for _, token := range td.match {
assert.True(t, n.check(token), "should match token '%s' for restricted words '%v'", token, td.input)
}
for _, token := range td.nomatch {
assert.False(t, n.check(token), "should not match token '%s' for restricted words '%v'", token, td.input)
}
}
}
func TestMatcher_MatchIfContainsRestrictedWords(t *testing.T) {
matcher := NewRestrictedWordsMatcher(StaticRestrictedWordsLister{[]string{"duck"}})
text := "What the duck it that?"
assert.True(t, matcher.Match("fakeID", text))
}
func TestMatcher_DoNotMatchIfNoRestrictedWords(t *testing.T) {
matcher := NewRestrictedWordsMatcher(StaticRestrictedWordsLister{[]string{"quack"}})
text := "What the duck it that?"
assert.False(t, matcher.Match("fakeID", text))
}
+61 -7
View File
@@ -1,13 +1,14 @@
package service
import (
"log"
"sort"
"sync"
"time"
log "github.com/go-pkgz/lgr"
"github.com/google/uuid"
multierror "github.com/hashicorp/go-multierror"
"github.com/hashicorp/go-multierror"
"github.com/patrickmn/go-cache"
"github.com/pkg/errors"
"github.com/umputun/remark/backend/app/store"
@@ -18,11 +19,12 @@ import (
// DataStore wraps store.Interface with additional methods
type DataStore struct {
engine.Interface
EditDuration time.Duration
AdminStore admin.Store
MaxCommentSize int
MaxVotes int
TitleExtractor *TitleExtractor
EditDuration time.Duration
AdminStore admin.Store
MaxCommentSize int
MaxVotes int
TitleExtractor *TitleExtractor
RestrictedWordsMatcher *RestrictedWordsMatcher
// granular locks
scopedLocks struct {
@@ -30,6 +32,11 @@ type DataStore struct {
sync.Once
locks map[string]sync.Locker
}
repliesCache struct {
*cache.Cache
once sync.Once
}
}
// UserMetaData keeps info about user flags
@@ -49,10 +56,14 @@ type PostMetaData struct {
}
const defaultCommentMaxSize = 2000
const maxLastCommentsReply = 1000
// UnlimitedVotes doesn't restrict MaxVotes
const UnlimitedVotes = -1
// ErrRestrictedWordsFound returned in case comment text contains restricted words
var ErrRestrictedWordsFound = errors.New("comment contains restricted words")
// Create prepares comment and forward to Interface.Create
func (s *DataStore) Create(comment store.Comment) (commentID string, err error) {
@@ -60,6 +71,10 @@ func (s *DataStore) Create(comment store.Comment) (commentID string, err error)
return "", errors.Wrap(err, "failed to prepare comment")
}
if s.RestrictedWordsMatcher != nil && s.RestrictedWordsMatcher.Match(comment.Locator.SiteID, comment.Text) {
return "", ErrRestrictedWordsFound
}
// keep input title and set to extracted if missing
if s.TitleExtractor != nil && comment.PostTitle == "" {
if title, err := s.TitleExtractor.Get(comment.Locator.URL); err == nil {
@@ -179,11 +194,19 @@ func (s *DataStore) EditComment(locator store.Locator, commentID string, req Edi
return comment, errors.Errorf("too late to edit %s", commentID)
}
if s.HasReplies(comment) {
return comment, errors.Errorf("parent comment with reply can't be edited, %s", commentID)
}
if req.Delete { // delete request
comment.Deleted = true
return comment, s.Delete(locator, commentID, store.SoftDelete)
}
if s.RestrictedWordsMatcher != nil && s.RestrictedWordsMatcher.Match(comment.Locator.SiteID, req.Text) {
return comment, ErrRestrictedWordsFound
}
comment.Text = req.Text
comment.Orig = req.Orig
comment.Edit = &store.Edit{
@@ -196,6 +219,37 @@ func (s *DataStore) EditComment(locator store.Locator, commentID string, req Edi
return comment, err
}
// HasReplies checks if there is any reply to the comments
// Loads last maxLastCommentsReply comments and compare parent id to the comment's id
// Comments with replies cached for 5 minutes
func (s *DataStore) HasReplies(comment store.Comment) bool {
s.repliesCache.once.Do(func() {
// default expiration time of 5 minutes, purge every 10 minutes
s.repliesCache.Cache = cache.New(5*time.Minute, 10*time.Minute)
})
if _, found := s.repliesCache.Get(comment.ID); found {
return true
}
comments, err := s.Last(comment.Locator.SiteID, maxLastCommentsReply)
if err != nil {
log.Printf("[WARN] can't get last comments for reply check, %v", err)
return false
}
for _, c := range comments {
if c.ParentID != "" && !c.Deleted {
if c.ParentID == comment.ID {
s.repliesCache.Set(comment.ID, true, cache.DefaultExpiration)
return true
}
}
}
return false
}
// SetTitle puts title from the locator.URL page and overwrites any existing title
func (s *DataStore) SetTitle(locator store.Locator, commentID string) (comment store.Comment, err error) {
if s.TitleExtractor == nil {
+57 -1
View File
@@ -12,7 +12,7 @@ import (
"testing"
"time"
"github.com/coreos/bbolt"
bolt "github.com/coreos/bbolt"
"github.com/pkg/errors"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -404,6 +404,32 @@ func TestService_EditCommentDurationFailed(t *testing.T) {
assert.NotNil(t, err)
}
func TestService_EditCommentReplyFailed(t *testing.T) {
defer os.Remove(testDb)
b := DataStore{Interface: prepStoreEngine(t), AdminStore: admin.NewStaticKeyStore("secret 123")}
res, err := b.Last("radio-t", 0)
t.Logf("%+v", res[1])
assert.Nil(t, err)
assert.Equal(t, 2, len(res))
assert.Nil(t, res[1].Edit)
reply := store.Comment{
ID: "123456",
ParentID: "id-1",
Text: "some text",
Timestamp: time.Date(2017, 12, 20, 15, 18, 22, 0, time.Local),
Locator: store.Locator{URL: "https://radio-t.com", SiteID: "radio-t"},
User: store.User{ID: "user2", Name: "user name 2"},
}
_, err = b.Create(reply)
assert.NoError(t, err)
_, err = b.EditComment(store.Locator{URL: "https://radio-t.com", SiteID: "radio-t"}, res[1].ID,
EditRequest{Orig: "yyy", Text: "xxx", Summary: "my edit"})
assert.EqualError(t, err, "parent comment with reply can't be edited, id-1")
}
func TestService_ValidateComment(t *testing.T) {
b := DataStore{MaxCommentSize: 2000, AdminStore: admin.NewStaticKeyStore("secret 123")}
@@ -523,6 +549,36 @@ func TestService_IsAdmin(t *testing.T) {
assert.True(t, b.IsAdmin("radio-t", "user2"))
}
func TestService_HasReplies(t *testing.T) {
defer os.Remove(testDb)
// two comments for https://radio-t.com, no reply
b := DataStore{Interface: prepStoreEngine(t), EditDuration: 100 * time.Millisecond,
AdminStore: admin.NewStaticStore("secret 123", []string{"user2"}, "user@email.com")}
comment := store.Comment{
ID: "id-1",
Text: `some text, <a href="http://radio-t.com">link</a>`,
Timestamp: time.Date(2017, 12, 20, 15, 18, 22, 0, time.Local),
Locator: store.Locator{URL: "https://radio-t.com", SiteID: "radio-t"},
User: store.User{ID: "user1", Name: "user name"},
}
assert.False(t, b.HasReplies(comment))
reply := store.Comment{
ID: "123456",
ParentID: "id-1",
Text: "some text",
Timestamp: time.Date(2017, 12, 20, 15, 18, 22, 0, time.Local),
Locator: store.Locator{URL: "https://radio-t.com", SiteID: "radio-t"},
User: store.User{ID: "user2", Name: "user name 2"},
}
_, err := b.Create(reply)
assert.NoError(t, err)
assert.True(t, b.HasReplies(comment))
}
// makes new boltdb, put two records
func prepStoreEngine(t *testing.T) engine.Interface {
os.Remove(testDb)
+1 -1
View File
@@ -2,9 +2,9 @@ package service
import (
"io"
"log"
"net/http"
log "github.com/go-pkgz/lgr"
"github.com/go-pkgz/rest/cache"
"github.com/pkg/errors"
"golang.org/x/net/html"
+2 -1
View File
@@ -8,8 +8,9 @@ import (
"hash"
"hash/crc64"
"io"
"log"
"regexp"
log "github.com/go-pkgz/lgr"
)
// User holds user-related info
+2 -1
View File
@@ -203,7 +203,8 @@ _Warning: this is not the real oauth2 server but just a small fake thing for dev
In addition to the primary method (i.e. JWT cookie with XSRF header) there are two more ways to authenticate:
1. Send JWT header as `X-JWT`. This shouldn't be used for web application, however can be helpful for service-to-service authentication.
2. [Basic access authentication](https://en.wikipedia.org/wiki/Basic_access_authentication). This mode disabled by default and will be enabled if `Opts.AdminPasswd` defined. This will allow access with basic auth admin:<Opts.AdminPasswd> with user [admin](https://github.com/go-pkgz/auth/blob/master/middleware/auth.go#L24). Such method can be used for automation scripts.
2. Send JWT token as query parameter, i.e. `/something?token=<jwt>`
3. [Basic access authentication](https://en.wikipedia.org/wiki/Basic_access_authentication). This mode disabled by default and will be enabled if `Opts.AdminPasswd` defined. This will allow access with basic auth admin:<Opts.AdminPasswd> with user [admin](https://github.com/go-pkgz/auth/blob/master/middleware/auth.go#L24). Such method can be used for automation scripts.
### Logging
+8 -8
View File
@@ -53,10 +53,10 @@ type Opts struct {
AvatarResizeLimit int // resize avatar's limit in pixels
AvatarRoutePath string // avatar routing prefix, i.e. "/api/v1/avatar", default `/avatar`
AdminPasswd string // if presented, allows basic auth with user admin and given password
AudienceReader token.Audience // list of allowed aud values, default (empty) allows any
RefreshFactor int // estimated number of request client sends in parallel during token refresh.
Logger logger.L // logger interface, default is no logging at all
AdminPasswd string // if presented, allows basic auth with user admin and given password
AudienceReader token.Audience // list of allowed aud values, default (empty) allows any
Logger logger.L // logger interface, default is no logging at all
RefreshCache middleware.RefreshCache // optional cache to keep refreshed tokens
}
// NewService initializes everything
@@ -66,9 +66,9 @@ func NewService(opts Opts) (res *Service) {
opts: opts,
logger: opts.Logger,
authMiddleware: middleware.Authenticator{
Validator: opts.Validator,
AdminPasswd: opts.AdminPasswd,
RefreshFactor: opts.RefreshFactor,
Validator: opts.Validator,
AdminPasswd: opts.AdminPasswd,
RefreshCache: opts.RefreshCache,
},
issuer: opts.Issuer,
}
@@ -78,7 +78,7 @@ func NewService(opts Opts) (res *Service) {
}
if opts.Logger == nil {
res.logger = logger.Func(func(fmt string, args ...interface{}) {}) // do-nothing logger
res.logger = logger.NoOp
}
jwtService := token.NewService(token.Opts{
+1 -1
View File
@@ -92,7 +92,7 @@ func (p *Proxy) Handler(w http.ResponseWriter, r *http.Request) {
avReader, size, err := p.Store.Get(avatarID)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusBadRequest, err, "can't load avatar")
rest.SendErrorJSON(w, r, p.L, http.StatusBadRequest, err, "can't load avatar")
return
}
+10 -3
View File
@@ -2,14 +2,21 @@ module github.com/go-pkgz/auth
require (
cloud.google.com/go v0.34.0 // indirect
github.com/boltdb/bolt v1.3.1 // indirect
github.com/coreos/bbolt v1.3.0
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
github.com/go-pkgz/mongo v1.0.0
github.com/go-pkgz/rest v1.1.5
github.com/go-pkgz/rest v1.2.0
github.com/kr/pretty v0.1.0 // indirect
github.com/nullrocks/identicon v0.0.0-20180626043057-7875f45b0022
github.com/pkg/errors v0.8.0
github.com/pkg/errors v0.8.1
github.com/stretchr/testify v1.3.0
golang.org/x/image v0.0.0-20181116024801-cd38e8056d9b
golang.org/x/net v0.0.0-20181220203305-927f97764cc3 // indirect
golang.org/x/net v0.0.0-20190107210223-45ffb0cd1ba0 // indirect
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 // indirect
golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb // indirect
google.golang.org/appengine v1.4.0 // indirect
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
)
+29 -10
View File
@@ -1,7 +1,10 @@
cloud.google.com/go v0.34.0 h1:eOI3/cP2VTU6uZLDYAoic+eyzzB9YyGmJ7eIjl8rOPg=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4=
github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
github.com/coreos/bbolt v1.3.0 h1:HIgH5xUWXT914HCI671AxuTTqjj64UOFr7pHn48LUTI=
github.com/coreos/bbolt v1.3.0/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumCAMpl/TFQ4/5kLM=
@@ -10,25 +13,41 @@ github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8 h1:DujepqpGd1hyOd7a
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
github.com/go-pkgz/mongo v1.0.0 h1:9jijAK7prCRMetiyTu3c1rv/2lMypzuf2DWcVpTlwzw=
github.com/go-pkgz/mongo v1.0.0/go.mod h1:R9si/F2aJsjz4MUxhzuppIHY8yLV3YCeuCpgcI50cu4=
github.com/go-pkgz/rest v1.1.3 h1:rMf+xJn8i1Ip9OKohusZsRxwntM0BwYu8OX8BuEwN80=
github.com/go-pkgz/rest v1.1.3/go.mod h1:DIxxm3vSt6e+IY+UQUOFsfB2YaHLmGoOfPLWN5pxQSA=
github.com/go-pkgz/rest v1.1.4 h1:/Lrg9kBWBjNah7nmCDHLszRAfVVBIy5ajf0vVgpHPi0=
github.com/go-pkgz/rest v1.1.4/go.mod h1:DIxxm3vSt6e+IY+UQUOFsfB2YaHLmGoOfPLWN5pxQSA=
github.com/go-pkgz/rest v1.1.5 h1:5br4mnscfLb27yxv5hJFLBVmAt09PrmIBP+meA3CfHc=
github.com/go-pkgz/rest v1.1.5/go.mod h1:DIxxm3vSt6e+IY+UQUOFsfB2YaHLmGoOfPLWN5pxQSA=
github.com/go-pkgz/rest v1.2.0 h1:75GVv25NmkV2l4dBr/io/ZApJ6zWQu5aZ4wFJA6QQCw=
github.com/go-pkgz/rest v1.2.0/go.mod h1:COazNj35u3RXAgQNBr6neR599tYP3URiOpsu9p0rOtk=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/nullrocks/identicon v0.0.0-20180626043057-7875f45b0022 h1:Ys0rDzh8s4UMlGaDa1UTA0sfKgvF0hQZzTYX8ktjiDc=
github.com/nullrocks/identicon v0.0.0-20180626043057-7875f45b0022/go.mod h1:x4NsS+uc7ecH/Cbm9xKQ6XzmJM57rWTkjywjfB2yQ18=
github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
golang.org/x/image v0.0.0-20181116024801-cd38e8056d9b h1:VHyIDlv3XkfCa5/a81uzaoDkHH4rr81Z62g+xlnO8uM=
golang.org/x/image v0.0.0-20181116024801-cd38e8056d9b/go.mod h1:ux5Hcp/YLpHSI86hEcLt0YII63i6oz57MZXIpbrjZUs=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3 h1:eH6Eip3UpmR+yM/qI9Ijluzb1bNv/cAU/n+6l8tRSis=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190107210223-45ffb0cd1ba0 h1:1DW40AJQ7AP4nY6ORUGUdkpXyEC9W2GAXcOPaMZK0K8=
golang.org/x/net v0.0.0-20190107210223-45ffb0cd1ba0/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890 h1:uESlIz09WIHT2I+pasSXcpLYqYK8wHcdCetU3VuMBJE=
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 h1:YUO/7uOKsKeq9UokNS62b8FYywz3ker1l1vDZRCRefw=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb h1:1w588/yEchbPNpa9sEvOcMZYbWHedwJjg4VOAdDHWHk=
golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
google.golang.org/appengine v1.4.0 h1:/wp5JvzpHIxhs/dumFmF7BXTf3Z+dd4uXta4kVyO508=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -2,7 +2,6 @@ package logger
import "log"
// L defines minimal interface used to log things
type L interface {
Logf(format string, args ...interface{})
}
@@ -16,5 +15,5 @@ func (f Func) Logf(format string, args ...interface{}) { f(format, args...) }
// NoOp logger
var NoOp = Func(func(format string, args ...interface{}) {})
// Std logger
// Std logger sends to std default logger directly
var Std = Func(func(format string, args ...interface{}) { log.Printf(format, args...) })
+30 -27
View File
@@ -2,7 +2,6 @@
package middleware
import (
"math/rand"
"net/http"
"github.com/pkg/errors"
@@ -15,17 +14,22 @@ import (
// Authenticator is top level auth object providing middlewares
type Authenticator struct {
logger.L
JWTService TokenService
Providers []provider.Service
Validator token.Validator
AdminPasswd string
RefreshFactor int
JWTService TokenService
Providers []provider.Service
Validator token.Validator
AdminPasswd string
RefreshCache RefreshCache
}
type RefreshCache interface {
Get(key interface{}) (value interface{}, ok bool)
Set(key, value interface{})
}
// TokenService defines interface accessing tokens
type TokenService interface {
Parse(tokenString string) (claims token.Claims, err error)
Set(w http.ResponseWriter, claims token.Claims) error
Set(w http.ResponseWriter, claims token.Claims) (token.Claims, error)
Get(r *http.Request) (claims token.Claims, token string, err error)
IsExpired(claims token.Claims) bool
Reset(w http.ResponseWriter)
@@ -83,7 +87,7 @@ func (a *Authenticator) auth(reqAuth bool) func(http.Handler) http.Handler {
}
if claims.User == nil {
onError(h, w, r, errors.New("failed auth, no user info presented in the claim"))
onError(h, w, r, errors.New("no user info presented in the claim"))
return
}
@@ -95,13 +99,12 @@ func (a *Authenticator) auth(reqAuth bool) func(http.Handler) http.Handler {
return
}
if a.shouldRefresh(claims) {
if claims, err = a.refreshExpiredToken(w, claims); err != nil {
if a.JWTService.IsExpired(claims) {
if claims, err = a.refreshExpiredToken(w, claims, tkn); err != nil {
a.JWTService.Reset(w)
onError(h, w, r, errors.Wrap(err, "can't refresh token"))
return
}
a.Logf("[DEBUG] token refreshed for %+v", claims.User)
}
r = token.SetUserInfo(r, *claims.User) // populate user info to request context
@@ -115,27 +118,27 @@ func (a *Authenticator) auth(reqAuth bool) func(http.Handler) http.Handler {
}
// refreshExpiredToken makes a new token with passed claims
func (a *Authenticator) refreshExpiredToken(w http.ResponseWriter, claims token.Claims) (token.Claims, error) {
claims.ExpiresAt = 0 // this will cause now+duration for refreshed token
if err := a.JWTService.Set(w, claims); err != nil {
func (a *Authenticator) refreshExpiredToken(w http.ResponseWriter, claims token.Claims, tkn string) (token.Claims, error) {
if a.RefreshCache != nil {
if c, ok := a.RefreshCache.Get(tkn); ok {
// already in cache
return c.(token.Claims), nil
}
}
claims.ExpiresAt = 0 // this will cause now+duration for refreshed token
c, err := a.JWTService.Set(w, claims) // Set changes token
if err != nil {
return token.Claims{}, err
}
return claims, nil
}
// shouldRefresh checks if token expired with an optional random rejection of refresh.
// the goal is to prevent multiple refresh request executed at the same time by allowing only some of them
func (a *Authenticator) shouldRefresh(claims token.Claims) bool {
if !a.JWTService.IsExpired(claims) {
return false
if a.RefreshCache != nil {
a.RefreshCache.Set(tkn, c)
}
// disable randomizing with 0 factor
if a.RefreshFactor == 0 {
return true
}
return rand.Int31n(int32(a.RefreshFactor)) == 0 // randomize selection
a.Logf("[DEBUG] token refreshed for %+v", claims.User)
return c, nil
}
// AdminOnly middleware allows access for admins only
+1 -1
View File
@@ -10,11 +10,11 @@ import (
"text/template"
"time"
"github.com/go-pkgz/auth/logger"
"github.com/nullrocks/identicon"
"github.com/pkg/errors"
"golang.org/x/oauth2"
"github.com/go-pkgz/auth/logger"
"github.com/go-pkgz/auth/token"
)
+6 -6
View File
@@ -4,7 +4,7 @@ import (
"errors"
"net/http"
"github.com/dgrijalva/jwt-go"
jwt "github.com/dgrijalva/jwt-go"
"github.com/go-pkgz/rest"
"github.com/go-pkgz/auth/logger"
@@ -44,16 +44,16 @@ func (p DirectHandler) LoginHandler(w http.ResponseWriter, r *http.Request) {
aud := r.URL.Query().Get("aud")
sessOnly := r.URL.Query().Get("sess") == "1"
if p.CredChecker == nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, errors.New("empty credential store"), "no credential store")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, errors.New("empty credential store"), "no credential store")
return
}
ok, err := p.CredChecker.Check(user, password)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to access creds store")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to access creds store")
return
}
if !ok {
rest.SendErrorJSON(w, r, http.StatusForbidden, nil, "incorrect user or password")
rest.SendErrorJSON(w, r, p.L, http.StatusForbidden, nil, "incorrect user or password")
return
}
claims := token.Claims{
@@ -65,8 +65,8 @@ func (p DirectHandler) LoginHandler(w http.ResponseWriter, r *http.Request) {
SessionOnly: sessOnly,
}
if err = p.TokenService.Set(w, claims); err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to set token")
if _, err = p.TokenService.Set(w, claims); err != nil {
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to set token")
return
}
rest.RenderJSON(w, r, claims.User)
+20 -16
View File
@@ -54,7 +54,7 @@ func (u userData) value(key string) string {
// initOauth2Handler makes oauth2 handler for given provider
func initOauth2Handler(p Params, service Oauth2Handler) Oauth2Handler {
if p.L == nil {
p.L = logger.Func(func(fmt string, args ...interface{}) {})
p.L = logger.NoOp
}
p.Logf("[INFO] init oauth2 service %s", service.name)
service.Params = p
@@ -81,13 +81,13 @@ func (p Oauth2Handler) LoginHandler(w http.ResponseWriter, r *http.Request) {
// make state (random) and store in session
state, err := randToken()
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to make oauth2 state")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to make oauth2 state")
return
}
cid, err := randToken()
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to make claim's id")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to make claim's id")
return
}
@@ -105,8 +105,8 @@ func (p Oauth2Handler) LoginHandler(w http.ResponseWriter, r *http.Request) {
},
}
if err := p.JwtService.Set(w, claims); err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to set token")
if _, err := p.JwtService.Set(w, claims); err != nil {
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to set token")
return
}
@@ -122,32 +122,32 @@ func (p Oauth2Handler) LoginHandler(w http.ResponseWriter, r *http.Request) {
func (p Oauth2Handler) AuthHandler(w http.ResponseWriter, r *http.Request) {
oauthClaims, _, err := p.JwtService.Get(r)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to get token")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to get token")
return
}
if oauthClaims.Handshake == nil {
rest.SendErrorJSON(w, r, http.StatusForbidden, nil, "invalid handshake token")
rest.SendErrorJSON(w, r, p.L, http.StatusForbidden, nil, "invalid handshake token")
return
}
retrievedState := oauthClaims.Handshake.State
if retrievedState == "" || retrievedState != r.URL.Query().Get("state") {
rest.SendErrorJSON(w, r, http.StatusForbidden, nil, "unexpected state")
rest.SendErrorJSON(w, r, p.L, http.StatusForbidden, nil, "unexpected state")
return
}
p.Logf("[DEBUG] token with state %s", retrievedState)
tok, err := p.conf.Exchange(context.Background(), r.URL.Query().Get("code"))
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "exchange failed")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "exchange failed")
return
}
client := p.conf.Client(context.Background(), tok)
uinfo, err := client.Get(p.infoURL)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusServiceUnavailable, err, "failed to get client info")
rest.SendErrorJSON(w, r, p.L, http.StatusServiceUnavailable, err, "failed to get client info")
return
}
@@ -159,13 +159,13 @@ func (p Oauth2Handler) AuthHandler(w http.ResponseWriter, r *http.Request) {
data, err := ioutil.ReadAll(uinfo.Body)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to read user info")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to read user info")
return
}
jData := map[string]interface{}{}
if e := json.Unmarshal(data, &jData); e != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to unmarshal user info")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to unmarshal user info")
return
}
p.Logf("[DEBUG] got raw user info %+v", jData)
@@ -173,13 +173,13 @@ func (p Oauth2Handler) AuthHandler(w http.ResponseWriter, r *http.Request) {
u := p.mapUser(jData, data)
u, err = setAvatar(p.AvatarSaver, u)
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to save avatar to proxy")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to save avatar to proxy")
return
}
cid, err := randToken()
if err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to make claim's id")
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to make claim's id")
return
}
claims := token.Claims{
@@ -192,8 +192,8 @@ func (p Oauth2Handler) AuthHandler(w http.ResponseWriter, r *http.Request) {
SessionOnly: oauthClaims.SessionOnly,
}
if err = p.JwtService.Set(w, claims); err != nil {
rest.SendErrorJSON(w, r, http.StatusInternalServerError, err, "failed to set token")
if _, err = p.JwtService.Set(w, claims); err != nil {
rest.SendErrorJSON(w, r, p.L, http.StatusInternalServerError, err, "failed to set token")
return
}
@@ -209,5 +209,9 @@ func (p Oauth2Handler) AuthHandler(w http.ResponseWriter, r *http.Request) {
// LogoutHandler - GET /logout
func (p Oauth2Handler) LogoutHandler(w http.ResponseWriter, r *http.Request) {
if _, _, err := p.JwtService.Get(r); err != nil {
rest.SendErrorJSON(w, r, p.L, http.StatusForbidden, err, "logout not allowed")
return
}
p.JwtService.Reset(w)
}
+1 -1
View File
@@ -30,7 +30,7 @@ type AvatarSaver interface {
// TokenService defines interface accessing tokens
type TokenService interface {
Parse(tokenString string) (claims token.Claims, err error)
Set(w http.ResponseWriter, claims token.Claims) error
Set(w http.ResponseWriter, claims token.Claims) (token.Claims, error)
Get(r *http.Request) (claims token.Claims, token string, err error)
Reset(w http.ResponseWriter)
}
+15 -5
View File
@@ -38,6 +38,7 @@ const (
jwtHeaderKey = "X-JWT"
xsrfCookieName = "XSRF-TOKEN"
xsrfHeaderKey = "X-XSRF-TOKEN"
tokenQuery = "token"
issuer = "go-pkgz/auth"
tokenDuration = time.Minute * 15
cookieDuration = time.Hour * 24 * 31
@@ -174,7 +175,7 @@ func (j *Service) validate(claims *Claims) error {
// Set creates token cookie with xsrf cookie and put it to ResponseWriter
// accepts claims and sets expiration if none defined. permanent flag means long-living cookie,
// false makes it session only.
func (j *Service) Set(w http.ResponseWriter, claims Claims) error {
func (j *Service) Set(w http.ResponseWriter, claims Claims) (Claims, error) {
if claims.ExpiresAt == 0 {
claims.ExpiresAt = time.Now().Add(j.TokenDuration).Unix()
}
@@ -189,7 +190,7 @@ func (j *Service) Set(w http.ResponseWriter, claims Claims) error {
tokenString, err := j.Token(claims)
if err != nil {
return errors.Wrap(err, "failed to make token token")
return Claims{}, errors.Wrap(err, "failed to make token token")
}
cookieExpiration := 0 // session cookie
@@ -205,18 +206,23 @@ func (j *Service) Set(w http.ResponseWriter, claims Claims) error {
MaxAge: cookieExpiration, Secure: j.SecureCookies}
http.SetCookie(w, &xsrfCookie)
return nil
return claims, nil
}
// Get token from header or cookie
// Get token from url, header or cookie
// if cookie used, verify xsrf token to match
func (j *Service) Get(r *http.Request) (Claims, string, error) {
fromCookie := false
tokenString := ""
// try to get from "token" query param
if tkQuery := r.URL.Query().Get(tokenQuery); tkQuery != "" {
tokenString = tkQuery
}
// try to get from X-JWT header
if tokenHeader := r.Header.Get(jwtHeaderKey); tokenHeader != "" {
if tokenHeader := r.Header.Get(jwtHeaderKey); tokenHeader != "" && tokenString == "" {
tokenString = tokenHeader
}
@@ -235,6 +241,10 @@ func (j *Service) Get(r *http.Request) (Claims, string, error) {
return Claims{}, "", errors.Wrap(err, "failed to get token")
}
if !fromCookie && j.IsExpired(claims) {
return Claims{}, "", errors.New("token expired")
}
if j.DisableXSRF {
return claims, tokenString, nil
}
+13
View File
@@ -0,0 +1,13 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, build with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
vendor
+19
View File
@@ -0,0 +1,19 @@
language: go
go:
- "1.11.x"
install: true
before_install:
- export TZ=America/Chicago
- curl -L https://git.io/vp6lP | sh
- go get github.com/mattn/goveralls
- export PATH=$(pwd)/bin:$PATH
script:
- GO111MODULE=on go get ./...
- GO111MODULE=on go mod vendor
- GO111MODULE=on go test -v -mod=vendor -covermode=count -coverprofile=profile.cov ./... || travis_terminate 1;
- ./bin/gometalinter --deadline=120s --exclude=test --exclude=mock --exclude=vendor --exclude=_example --disable-all --enable=errcheck --enable=vet --enable=vetshadow --enable=megacheck --enable=ineffassign --enable=varcheck --enable=unconvert --enable=deadcode --enable=interfacer --enable=gotype ./... || travis_terminate 1;
- $GOPATH/bin/goveralls -coverprofile=profile.cov -service=travis-ci
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 Umputun
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+57
View File
@@ -0,0 +1,57 @@
# lgr - simple logger with some extras [![Build Status](https://travis-ci.org/go-pkgz/lgr.svg?branch=master)](https://travis-ci.org/go-pkgz/lgr) [![Coverage Status](https://coveralls.io/repos/github/go-pkgz/lgr/badge.svg?branch=master)](https://coveralls.io/github/go-pkgz/lgr?branch=master) [![godoc](https://godoc.org/github.com/go-pkgz/lgr?status.svg)](https://godoc.org/github.com/go-pkgz/lgr)
## install
`go get github/go-pkgz/lgr`
## usage
```go
l := lgr.New(lgr.Debug, lgr.CallerFile) // allow debug and caller file info
l.Logf("INFO some important err message, %v", err)
l.Logf("DEBUG some less important err message, %v", err)
```
output looks like this:
```
2018/01/07 13:02:34.000 INFO {svc/handler.go:101 h.MyFunc1} some important err message, can't open file`
2018/01/07 13:02:34.015 DEBUG {svc/handler.go:155 h.MyFunc2} some less important err message, file is too small`
```
_Without `lgr.CallerFile` it will drop `{caller}` part_
## details
### interfaces and default loggers
- `lgr` package provides a single interface `lgr.L` with a single method `Logf(format string, args ...interface{})`. Function wrapper `lgr.Func` allows to make `lgr.L` from a function directly.
- Default logger functionality can be used without `lgr.New`, but just `lgr.Printf`
- Two predefined loggers available: `lgr.NoOp` (do-nothing logger) and `lgr.Std` (passing directly to stdlib log)
### options
`lgr.New` call accepts functional options:
- `lgr.Debug` - turn debug mode on. This allows messages with "DEBUG" level (filtered overwise)
- `lgr.CallerFile` - adds the caller file info each message
- `lgr.CallerFunc` - adds the caller function info each message
- `lgr.LevelBraces` - wraps levels with "[" and "]"
- `lgr.Msec` - adds milliseconds to timestamp
- `lgr.Out(io.Writer)` - sets the output writer, default `os.Stdout`
- `lgr.Err(io.Writer)` - sets the error writer, default `os.Stderr`
### levels
`lgr.Logf` recognizes prefixes like "INFO" or "[INFO]" as levels. The full list of supported levels - "DEBUG", "INFO", "WARN", "ERROR", "PANIC" and "FATAL"
- `DEBUG` will be filtered unless `lgr.Debug` option defined
- `INFO` and `WARN` don't have any special behavior attached
- `ERROR` sends messages to both out and err writers
- `PANIC` and `FATAL` send messages to both out and err writers. In addition sends dump of callers and runtime info to err only, and call `os.Exit(1)`.
### global logger
Users should avoid global logger and pass the concrete logger as a dependency. However, in some cases global logger may be needed, for example migration from stdlib `log` to `lgr`. For such cases `log "github.com/go-pkgz/lgr"` can be imported instead of `log` package.
Global logger provides `lgr.Printf`, `lgr.Print` and `lgr.Fatalf` functions. User can customize the logger by calling `lgr.Setup(options ...)`. The instance of this logger can be retried with `lgr.Default()`
+3
View File
@@ -0,0 +1,3 @@
module github.com/go-pkgz/lgr
require github.com/stretchr/testify v1.3.0
+7
View File
@@ -0,0 +1,7 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+50
View File
@@ -0,0 +1,50 @@
package lgr
import (
stdlog "log"
"os"
)
var def = New() // default logger doesn't allow DEBUG and doesn't add caller info
// L defines minimal interface used to log things
type L interface {
Logf(format string, args ...interface{})
}
// Func type is an adapter to allow the use of ordinary functions as Logger.
type Func func(format string, args ...interface{})
// Logf calls f(id)
func (f Func) Logf(format string, args ...interface{}) { f(format, args...) }
// NoOp logger
var NoOp = Func(func(format string, args ...interface{}) {})
// Std logger sends to std default logger directly
var Std = Func(func(format string, args ...interface{}) { stdlog.Printf(format, args...) })
// Printf simplifies replacement of std logger
func Printf(format string, args ...interface{}) {
def.Logf(format, args...)
}
// Print simplifies replacement of std logger
func Print(line string) {
def.Logf(line)
}
// Fatalf simplifies replacement of std logger
func Fatalf(format string, args ...interface{}) {
def.Logf(format, args...)
os.Exit(1)
}
// Setup default logger with options
func Setup(opts ...Option) {
def = New(opts...)
def.skipCallers = 2
}
// Default returns pre-constructed def logger (debug off, callers disabled)
func Default() L { return def }
+194
View File
@@ -0,0 +1,194 @@
package lgr
import (
"fmt"
"io"
"os"
"runtime"
"strings"
"sync"
"time"
)
var levels = []string{"DEBUG", "INFO", "WARN", "ERROR", "PANIC", "FATAL"}
// Logger provided simple logger with basic support of levels. Thread safe
type Logger struct {
stdout, stderr io.Writer
dbg bool
lock sync.Mutex
callerFile, callerFunc bool
now nowFn
fatal panicFn
skipCallers int
levelBraces bool
msec bool
}
type nowFn func() time.Time
type panicFn func()
// New makes new leveled logger. Accepts dbg flag turing on info about the caller and allowing DEBUG messages/
// Two writers can be passed optionally - first for out and second for err
func New(options ...Option) *Logger {
res := Logger{
now: time.Now,
fatal: func() { os.Exit(1) },
stdout: os.Stdout,
stderr: os.Stderr,
skipCallers: 1,
}
for _, opt := range options {
opt(&res)
}
return &res
}
// Logf implements L interface to output with printf style.
// Each line prefixed with ts, level and optionally (dbg mode only) by caller info.
// ERROR and FATAL also send the same line to err writer.
// FATAL adds runtime stack and os.exit(1), like panic.
func (l *Logger) Logf(format string, args ...interface{}) {
// format timestamp with or without msecs
ts := func() (res string) {
if l.msec {
return l.now().Format("2006/01/02 15:04:05.000")
}
return l.now().Format("2006/01/02 15:04:05")
}
lv, msg := l.extractLevel(fmt.Sprintf(format, args...))
if lv == "DEBUG" && !l.dbg {
return
}
var bld strings.Builder
bld.WriteString(ts())
bld.WriteString(l.formatLevel(lv))
bld.WriteString(" ")
if l.dbg && (l.callerFile || l.callerFunc) {
if pc, file, line, ok := runtime.Caller(l.skipCallers); ok {
funcName := ""
if l.callerFunc {
funcNameElems := strings.Split(runtime.FuncForPC(pc).Name(), "/")
funcName = funcNameElems[len(funcNameElems)-1]
}
fileInfo := ""
if l.callerFile {
fnameElems := strings.Split(file, "/")
fileInfo = fmt.Sprintf("%s:%d", strings.Join(fnameElems[len(fnameElems)-2:], "/"), line)
if l.callerFunc {
fileInfo += " "
}
}
srcFileInfo := fmt.Sprintf("{%s%s} ", fileInfo, funcName)
bld.WriteString(srcFileInfo)
}
}
bld.WriteString(msg) //nolint
bld.WriteString("\n") //nolint
l.lock.Lock()
msgb := []byte(bld.String())
l.stdout.Write(msgb) //nolint
switch lv {
case "PANIC", "FATAL":
l.stderr.Write(msgb) //nolint
bld.WriteString("\n") //nolint
l.stderr.Write(getDump()) //nolint
l.fatal()
case "ERROR":
l.stderr.Write(msgb) //nolint
}
l.lock.Unlock()
}
func (l *Logger) formatLevel(lv string) string {
brace := func(b string) string {
if l.levelBraces {
return b
}
return ""
}
if lv == "" {
return ""
}
spaces := ""
if len(lv) == 4 {
spaces = " "
}
return " " + brace("[") + lv + brace("]") + spaces
}
func (l *Logger) extractLevel(line string) (level, msg string) {
for _, lv := range levels {
if strings.HasPrefix(line, lv) {
return lv, line[len(lv)+1:]
}
if strings.HasPrefix(line, "["+lv+"]") {
return lv, line[len(lv)+3:]
}
}
return "INFO", line
}
// getDump reads runtime stack and returns as a string
func getDump() []byte {
maxSize := 5 * 1024 * 1024
stacktrace := make([]byte, maxSize)
length := runtime.Stack(stacktrace, true)
if length > maxSize {
length = maxSize
}
return stacktrace[:length]
}
// Option func type
type Option func(l *Logger)
// Out sets out writer
func Out(w io.Writer) Option {
return func(l *Logger) {
l.stdout = w
}
}
// Err sets error writer
func Err(w io.Writer) Option {
return func(l *Logger) {
l.stderr = w
}
}
// Debug turn on dbg mode
func Debug(l *Logger) {
l.dbg = true
}
// CallerFile adds caller info with file, and line number
func CallerFile(l *Logger) {
l.callerFile = true
}
// CallerFunc adds caller info with function name
func CallerFunc(l *Logger) {
l.callerFunc = true
}
// LevelBraces adds [] to level
func LevelBraces(l *Logger) {
l.levelBraces = true
}
// Msec adds .msec to timestamp
func Msec(l *Logger) {
l.msec = true
}
+12 -10
View File
@@ -1,23 +1,25 @@
language: go
go:
- "1.10.x"
- "1.11.x"
install: true
go_import_path: github.com/go-pkgz/mongo
services: mongodb
before_install:
- cp -rf .vendor vendor
- export TZ=America/Chicago
- curl -L https://git.io/vp6lP | sh
- go get github.com/mattn/goveralls
- go get gopkg.in/alecthomas/gometalinter.v2
- $GOPATH/bin/gometalinter.v2 --install
- export MONGO_TEST=mongodb://127.0.0.1:27017
- export PATH=$(pwd)/bin:$PATH
before_script:
- sleep 3
script:
- export MONGO_TEST=mongodb://127.0.0.1:27017
- go test ./...
- $GOPATH/bin/gometalinter.v2 --exclude=test --exclude=mock --exclude=vendor ./...
- $GOPATH/bin/goveralls -service=travis-ci
- GO111MODULE=on go get ./...
- GO111MODULE=on go mod vendor
- GO111MODULE=on go test -v -mod=vendor -covermode=count -coverprofile=profile.cov ./... || travis_terminate 1;
- ./bin/gometalinter --deadline=120s --exclude=test --exclude=mock --exclude=vendor --exclude=_example --disable-all --enable=errcheck --enable=vet --enable=vetshadow --enable=megacheck --enable=ineffassign --enable=varcheck --enable=unconvert --enable=deadcode --enable=interfacer --enable=gotype ./... || travis_terminate 1;
- $GOPATH/bin/goveralls -coverprofile=profile.cov -service=travis-ci
@@ -1,15 +0,0 @@
ISC License
Copyright (c) 2012-2016 Dave Collins <dave@davec.name>
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
@@ -1,25 +0,0 @@
mgo - MongoDB driver for Go
Copyright (c) 2010-2013 - Gustavo Niemeyer <gustavo@niemeyer.net>
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -1,25 +0,0 @@
BSON library for Go
Copyright (c) 2010-2012 - Gustavo Niemeyer <gustavo@niemeyer.net>
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -1,27 +0,0 @@
Copyright (c) 2012 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -1,27 +0,0 @@
Copyright (c) 2013, Patrick Mezard
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
The names of its contributors may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -1,22 +0,0 @@
Copyright (c) 2012 - 2013 Mat Ryer and Tyler Bunnell
Please consider promoting this project if you find it useful.
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT
OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+9
View File
@@ -0,0 +1,9 @@
module github.com/go-pkgz/mongo
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/globalsign/mgo v0.0.0-20180615134936-113d3961e731
github.com/go-pkgz/lgr v0.2.2
github.com/stretchr/objx v0.1.1 // indirect
github.com/stretchr/testify v1.3.0
)
+17
View File
@@ -0,0 +1,17 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/globalsign/mgo v0.0.0-20180615134936-113d3961e731 h1:y7wyeiA6T+TT+HGC9DYypvLkUeg99N4rqHMzn2MmjYk=
github.com/globalsign/mgo v0.0.0-20180615134936-113d3961e731/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8 h1:DujepqpGd1hyOd7aW59XpK7Qymp8iy83xq74fLr21is=
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
github.com/go-pkgz/lgr v0.1.5 h1:oWj3VNlyYL2uUpdL6Gbi21BAH619Xb/8E78ozpyI2xo=
github.com/go-pkgz/lgr v0.1.5/go.mod h1:hBM1NM/SoYdlrykgdgJWGrZ/TM/XaZIjRbJfx7NkMm8=
github.com/go-pkgz/lgr v0.2.2 h1:HSOqMVoetAfvA40Gpy/X/HGyV0UUafIMPgp+SdZends=
github.com/go-pkgz/lgr v0.2.2/go.mod h1:hBM1NM/SoYdlrykgdgJWGrZ/TM/XaZIjRbJfx7NkMm8=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+15 -11
View File
@@ -6,13 +6,12 @@ import (
"crypto/tls"
"errors"
"fmt"
"log"
"net"
"os"
"strings"
"time"
"github.com/globalsign/mgo"
log "github.com/go-pkgz/lgr"
)
// Server represents mongo instance and provides session accessor
@@ -27,11 +26,11 @@ type ServerParams struct {
ConsistencyMode mgo.Mode
Delay int // initial delay to give mongo server some time to start, in case if mongo part of the same compose
Debug bool // turn on mgo debug mode
SSL bool
SSL bool // enforce SSL connection
}
// NewServerWithURL makes mongo server from url like
// mongodb://remark42:password@127.0.0.1t:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin
// mongodb://remark42:password@127.0.0.1:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin
func NewServerWithURL(url string, timeout time.Duration) (res *Server, err error) {
dial, params, err := parseURL(url, timeout)
if err != nil {
@@ -44,10 +43,8 @@ func NewServerWithURL(url string, timeout time.Duration) (res *Server, err error
func NewServer(dial mgo.DialInfo, params ServerParams) (res *Server, err error) {
result := Server{dial: dial, params: params}
if params.Debug {
mgo.SetDebug(true)
mgo.SetLogger(log.New(os.Stdout, "MGO ", log.Ldate|log.Ltime|log.Lmicroseconds))
}
mgo.SetDebug(true)
mgo.SetLogger(&mgdLogger{})
if len(dial.Addrs) == 0 {
return nil, errors.New("missing mongo address")
@@ -89,11 +86,11 @@ func NewServer(dial mgo.DialInfo, params ServerParams) (res *Server, err error)
}
// SessionCopy returns copy of main session. Client should close it
func (m Server) SessionCopy() *mgo.Session {
func (m *Server) SessionCopy() *mgo.Session {
return m.sess.Copy()
}
func (m Server) String() string {
func (m *Server) String() string {
return fmt.Sprintf("%v%s", m.dial.Addrs, m.dial.Database)
}
@@ -110,8 +107,15 @@ func parseURL(mongoURL string, connectTimeout time.Duration) (mgo.DialInfo, Serv
dial, err := mgo.ParseURL(mongoURL)
if err != nil {
return mgo.DialInfo{}, ServerParams{}, fmt.Errorf("failed to pars mongo url %s, %s", mongoURL, err)
return mgo.DialInfo{}, ServerParams{}, fmt.Errorf("failed to parse mongo url %s, %s", mongoURL, err)
}
dial.Timeout = connectTimeout
return *dial, params, nil
}
type mgdLogger struct{}
func (l *mgdLogger) Output(calldepth int, s string) error {
log.Printf("[DEBUG] MGO %s", s)
return nil
}
+1 -1
View File
@@ -2,13 +2,13 @@ package mongo
import (
"fmt"
"log"
"os"
"sync"
"testing"
"time"
"github.com/globalsign/mgo"
log "github.com/go-pkgz/lgr"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
+1 -1
View File
@@ -3,11 +3,11 @@ package mongo
import (
"context"
"fmt"
"log"
"sync"
"time"
"github.com/globalsign/mgo"
log "github.com/go-pkgz/lgr"
)
// BufferedWriter defines interface for writes and flush
+6 -5
View File
@@ -6,13 +6,14 @@ go:
install: true
before_install:
- export TZ=America/Chicago
- curl -L https://git.io/vp6lP | sh
- go get github.com/mattn/goveralls
- go get gopkg.in/alecthomas/gometalinter.v2
- $GOPATH/bin/gometalinter.v2 --install
- export PATH=$(pwd)/bin:$PATH
script:
- GO111MODULE=on go get ./...
- GO111MODULE=on go mod vendor
- GO111MODULE=on go test -v -mod=vendor ./...
- $GOPATH/bin/gometalinter.v2 --exclude=test --exclude=mock --exclude=vendor ./...
- $GOPATH/bin/goveralls -service=travis-ci
- GO111MODULE=on go test -v -mod=vendor -covermode=count -coverprofile=profile.cov ./... || travis_terminate 1
- ./bin/gometalinter --deadline=120s --exclude=test --exclude=mock --exclude=vendor --exclude=_example --disable-all --enable=errcheck --enable=vet --enable=vetshadow --enable=megacheck --enable=ineffassign --enable=varcheck --enable=unconvert --enable=deadcode --enable=interfacer --enable=gotype ./... || travis_terminate 1;
- $GOPATH/bin/goveralls -coverprofile=profile.cov -service=travis-ci
+1 -5
View File
@@ -1,10 +1,9 @@
package cache
import (
"log"
"sync/atomic"
"github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru"
"github.com/pkg/errors"
)
@@ -20,7 +19,6 @@ type memoryCache struct {
// NewMemoryCache makes memoryCache implementation
func NewMemoryCache(options ...Option) (LoadingCache, error) {
log.Print("[INFO] make memory cache")
res := memoryCache{
postFlushFn: func() {},
@@ -44,8 +42,6 @@ func NewMemoryCache(options ...Option) (LoadingCache, error) {
return nil, errors.Wrap(err, "failed to make cache")
}
log.Printf("[DEBUG] create lru cache, maxKeys=%d, maxValueSize=%d, maxCacheSize=%d",
res.maxKeys, res.maxValueSize, res.maxCacheSize)
return &res, nil
}
+1 -2
View File
@@ -4,6 +4,5 @@ require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/hashicorp/golang-lru v0.5.0
github.com/pkg/errors v0.8.0
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/testify v1.2.2
github.com/stretchr/testify v1.3.0
)
+4 -2
View File
@@ -1,3 +1,4 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo=
@@ -6,5 +7,6 @@ github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+6 -3
View File
@@ -3,16 +3,19 @@ package rest
import (
"errors"
"fmt"
"log"
"net/http"
"net/url"
"runtime"
"strings"
"github.com/go-pkgz/rest/logger"
)
// SendErrorJSON sends {error: msg} with error code and logging error and caller
func SendErrorJSON(w http.ResponseWriter, r *http.Request, code int, err error, msg string) {
log.Printf("[DEBUG] %s", errDetailsMsg(r, code, err, msg))
func SendErrorJSON(w http.ResponseWriter, r *http.Request, l logger.Backend, code int, err error, msg string) {
if l != nil {
l.Logf("%s", errDetailsMsg(r, code, err, msg))
}
w.WriteHeader(code)
RenderJSON(w, r, JSON{"error": msg})
}
+16 -3
View File
@@ -23,6 +23,7 @@ type Middleware struct {
flags []Flag
ipFn func(ip string) string
userFn func(r *http.Request) (string, error)
log Backend
}
// Flag type
@@ -36,9 +37,20 @@ const (
None
)
// Logger returns default logger middleware
// Backend is logging backend
type Backend interface {
Logf(format string, args ...interface{})
}
type stdBackend struct{}
func (s stdBackend) Logf(format string, args ...interface{}) {
log.Printf(format, args...)
}
// Logger returns default logger middleware with REST prefix
func Logger(next http.Handler) http.Handler {
l := New(Flags(All), Prefix("[REST]"))
l := New(Prefix("REST"))
return l.Handler(next)
}
@@ -49,6 +61,7 @@ func New(options ...Option) *Middleware {
prefix: "",
maxBodySize: 1024,
flags: []Flag{All},
log: stdBackend{},
}
for _, opt := range options {
opt(&res)
@@ -86,7 +99,7 @@ func (l *Middleware) Handler(next http.Handler) http.Handler {
remoteIP = l.ipFn(remoteIP)
}
log.Printf("%s %s - %s - %s - %d (%d) - %v %s %s",
l.log.Logf("%s %s - %s - %s - %d (%d) - %v %s %s",
l.prefix, r.Method, q, remoteIP, ww.status, ww.size, t2.Sub(t1), user, body)
}()
+7
View File
@@ -43,3 +43,10 @@ func UserFn(userFn func(r *http.Request) (string, error)) Option {
l.userFn = userFn
}
}
// Log functional option defines loging backend.
func Log(log Backend) Option {
return func(l *Middleware) {
l.log = log
}
}
+16 -15
View File
@@ -1,11 +1,12 @@
package rest
import (
"log"
"net/http"
"os"
"runtime/debug"
"strings"
"github.com/go-pkgz/rest/logger"
)
// AppInfo adds custom app-info to the response header
@@ -32,9 +33,7 @@ func Ping(next http.Handler) http.Handler {
if r.Method == "GET" && strings.HasSuffix(strings.ToLower(r.URL.Path), "/ping") {
w.Header().Set("Content-Type", "text/plain")
w.WriteHeader(http.StatusOK)
if _, err := w.Write([]byte("pong")); err != nil {
log.Printf("[WARN] can't send pong, %s", err)
}
w.Write([]byte("pong")) //nolint
return
}
next.ServeHTTP(w, r)
@@ -43,16 +42,18 @@ func Ping(next http.Handler) http.Handler {
}
// Recoverer is a middleware that recovers from panics, logs the panic and returns a HTTP 500 status if possible.
func Recoverer(next http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
defer func() {
if rvr := recover(); rvr != nil {
log.Printf("[WARN] request panic, %v", rvr)
log.Print(string(debug.Stack()))
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
}
}()
next.ServeHTTP(w, r)
func Recoverer(l logger.Backend) func(http.Handler) http.Handler {
return func(h http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
defer func() {
if rvr := recover(); rvr != nil {
l.Logf("request panic, %v", rvr)
l.Logf(string(debug.Stack()))
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
}
}()
h.ServeHTTP(w, r)
}
return http.HandlerFunc(fn)
}
return http.HandlerFunc(fn)
}
+1 -1
View File
@@ -48,7 +48,7 @@ func RenderJSONWithHTML(w http.ResponseWriter, r *http.Request, v interface{}) e
data, err := encodeJSONWithHTML(v)
if err != nil {
return errors.Wrap(err, "json encoding failed")
return err
}
return RenderJSONFromBytes(w, r, data)
}
-22
View File
@@ -1,22 +0,0 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
-354
View File
@@ -1,354 +0,0 @@
Mozilla Public License, version 2.0
1. Definitions
1.1. “Contributor”
means each individual or legal entity that creates, contributes to the
creation of, or owns Covered Software.
1.2. “Contributor Version”
means the combination of the Contributions of others (if any) used by a
Contributor and that particular Contributors Contribution.
1.3. “Contribution”
means Covered Software of a particular Contributor.
1.4. “Covered Software”
means Source Code Form to which the initial Contributor has attached the
notice in Exhibit A, the Executable Form of such Source Code Form, and
Modifications of such Source Code Form, in each case including portions
thereof.
1.5. “Incompatible With Secondary Licenses”
means
a. that the initial Contributor has attached the notice described in
Exhibit B to the Covered Software; or
b. that the Covered Software was made available under the terms of version
1.1 or earlier of the License, but not also under the terms of a
Secondary License.
1.6. “Executable Form”
means any form of the work other than Source Code Form.
1.7. “Larger Work”
means a work that combines Covered Software with other material, in a separate
file or files, that is not Covered Software.
1.8. “License”
means this document.
1.9. “Licensable”
means having the right to grant, to the maximum extent possible, whether at the
time of the initial grant or subsequently, any and all of the rights conveyed by
this License.
1.10. “Modifications”
means any of the following:
a. any file in Source Code Form that results from an addition to, deletion
from, or modification of the contents of Covered Software; or
b. any new file in Source Code Form that contains any Covered Software.
1.11. “Patent Claims” of a Contributor
means any patent claim(s), including without limitation, method, process,
and apparatus claims, in any patent Licensable by such Contributor that
would be infringed, but for the grant of the License, by the making,
using, selling, offering for sale, having made, import, or transfer of
either its Contributions or its Contributor Version.
1.12. “Secondary License”
means either the GNU General Public License, Version 2.0, the GNU Lesser
General Public License, Version 2.1, the GNU Affero General Public
License, Version 3.0, or any later versions of those licenses.
1.13. “Source Code Form”
means the form of the work preferred for making modifications.
1.14. “You” (or “Your”)
means an individual or a legal entity exercising rights under this
License. For legal entities, “You” includes any entity that controls, is
controlled by, or is under common control with You. For purposes of this
definition, “control” means (a) the power, direct or indirect, to cause
the direction or management of such entity, whether by contract or
otherwise, or (b) ownership of more than fifty percent (50%) of the
outstanding shares or beneficial ownership of such entity.
2. License Grants and Conditions
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
a. under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or as
part of a Larger Work; and
b. under Patent Claims of such Contributor to make, use, sell, offer for
sale, have made, import, and otherwise transfer either its Contributions
or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution become
effective for each Contribution on the date the Contributor first distributes
such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under this
License. No additional rights or licenses will be implied from the distribution
or licensing of Covered Software under this License. Notwithstanding Section
2.1(b) above, no patent license is granted by a Contributor:
a. for any code that a Contributor has removed from Covered Software; or
b. for infringements caused by: (i) Your and any other third partys
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
c. under Patent Claims infringed by Covered Software in the absence of its
Contributions.
This License does not grant any rights in the trademarks, service marks, or
logos of any Contributor (except as may be necessary to comply with the
notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this License
(see Section 10.2) or under the terms of a Secondary License (if permitted
under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its Contributions
are its original creation(s) or it has sufficient rights to grant the
rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under applicable
copyright doctrines of fair use, fair dealing, or other equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
Section 2.1.
3. Responsibilities
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under the
terms of this License. You must inform recipients that the Source Code Form
of the Covered Software is governed by the terms of this License, and how
they can obtain a copy of this License. You may not attempt to alter or
restrict the recipients rights in the Source Code Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
a. such Covered Software must also be made available in Source Code Form,
as described in Section 3.1, and You must inform recipients of the
Executable Form how they can obtain a copy of such Source Code Form by
reasonable means in a timely manner, at a charge no more than the cost
of distribution to the recipient; and
b. You may distribute such Executable Form under the terms of this License,
or sublicense it under different terms, provided that the license for
the Executable Form does not attempt to limit or alter the recipients
rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for the
Covered Software. If the Larger Work is a combination of Covered Software
with a work governed by one or more Secondary Licenses, and the Covered
Software is not Incompatible With Secondary Licenses, this License permits
You to additionally distribute such Covered Software under the terms of
such Secondary License(s), so that the recipient of the Larger Work may, at
their option, further distribute the Covered Software under the terms of
either this License or such Secondary License(s).
3.4. Notices
You may not remove or alter the substance of any license notices (including
copyright notices, patent notices, disclaimers of warranty, or limitations
of liability) contained within the Source Code Form of the Covered
Software, except that You may alter any license notices to the extent
required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on behalf
of any Contributor. You must make it absolutely clear that any such
warranty, support, indemnity, or liability obligation is offered by You
alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
If it is impossible for You to comply with any of the terms of this License
with respect to some or all of the Covered Software due to statute, judicial
order, or regulation then You must: (a) comply with the terms of this License
to the maximum extent possible; and (b) describe the limitations and the code
they affect. Such description must be placed in a text file included with all
distributions of the Covered Software under this License. Except to the
extent prohibited by statute or regulation, such description must be
sufficiently detailed for a recipient of ordinary skill to be able to
understand it.
5. Termination
5.1. The rights granted under this License will terminate automatically if You
fail to comply with any of its terms. However, if You become compliant,
then the rights granted under this License from a particular Contributor
are reinstated (a) provisionally, unless and until such Contributor
explicitly and finally terminates Your grants, and (b) on an ongoing basis,
if such Contributor fails to notify You of the non-compliance by some
reasonable means prior to 60 days after You have come back into compliance.
Moreover, Your grants from a particular Contributor are reinstated on an
ongoing basis if such Contributor notifies You of the non-compliance by
some reasonable means, this is the first time You have received notice of
non-compliance with this License from such Contributor, and You become
compliant prior to 30 days after Your receipt of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions, counter-claims,
and cross-claims) alleging that a Contributor Version directly or
indirectly infringes any patent, then the rights granted to You by any and
all Contributors for the Covered Software under Section 2.1 of this License
shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
license agreements (excluding distributors and resellers) which have been
validly granted by You or Your distributors under this License prior to
termination shall survive termination.
6. Disclaimer of Warranty
Covered Software is provided under this License on an “as is” basis, without
warranty of any kind, either expressed, implied, or statutory, including,
without limitation, warranties that the Covered Software is free of defects,
merchantable, fit for a particular purpose or non-infringing. The entire
risk as to the quality and performance of the Covered Software is with You.
Should any Covered Software prove defective in any respect, You (not any
Contributor) assume the cost of any necessary servicing, repair, or
correction. This disclaimer of warranty constitutes an essential part of this
License. No use of any Covered Software is authorized under this License
except under this disclaimer.
7. Limitation of Liability
Under no circumstances and under no legal theory, whether tort (including
negligence), contract, or otherwise, shall any Contributor, or anyone who
distributes Covered Software as permitted above, be liable to You for any
direct, indirect, special, incidental, or consequential damages of any
character including, without limitation, damages for lost profits, loss of
goodwill, work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses, even if such party shall have been
informed of the possibility of such damages. This limitation of liability
shall not apply to liability for death or personal injury resulting from such
partys negligence to the extent applicable law prohibits such limitation.
Some jurisdictions do not allow the exclusion or limitation of incidental or
consequential damages, so this exclusion and limitation may not apply to You.
8. Litigation
Any litigation relating to this License may be brought only in the courts of
a jurisdiction where the defendant maintains its principal place of business
and such litigation shall be governed by laws of that jurisdiction, without
reference to its conflict-of-law provisions. Nothing in this Section shall
prevent a partys ability to bring cross-claims or counter-claims.
9. Miscellaneous
This License represents the complete agreement concerning the subject matter
hereof. If any provision of this License is held to be unenforceable, such
provision shall be reformed only to the extent necessary to make it
enforceable. Any law or regulation which provides that the language of a
contract shall be construed against the drafter shall not be used to construe
this License against a Contributor.
10. Versions of the License
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version of
the License under which You originally received the Covered Software, or
under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a modified
version of this License if you rename the license and remove any
references to the name of the license steward (except to note that such
modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses
If You choose to distribute Source Code Form that is Incompatible With
Secondary Licenses under the terms of this version of the License, the
notice described in Exhibit B of this License must be attached.
Exhibit A - Source Code Form License Notice
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular file, then
You may include the notice in a location (such as a LICENSE file in a relevant
directory) where a recipient would be likely to look for such a notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - “Incompatible With Secondary Licenses” Notice
This Source Code Form is “Incompatible
With Secondary Licenses”, as defined by
the Mozilla Public License, v. 2.0.
-36
View File
@@ -1,36 +0,0 @@
# logutils
logutils is a Go package that augments the standard library "log" package
to make logging a bit more modern, without fragmenting the Go ecosystem
with new logging packages.
## The simplest thing that could possibly work
Presumably your application already uses the default `log` package. To switch, you'll want your code to look like the following:
```go
package main
import (
"log"
"os"
"github.com/hashicorp/logutils"
)
func main() {
filter := &logutils.LevelFilter{
Levels: []logutils.LogLevel{"DEBUG", "WARN", "ERROR"},
MinLevel: logutils.LogLevel("WARN"),
Writer: os.Stderr,
}
log.SetOutput(filter)
log.Print("[DEBUG] Debugging") // this will not print
log.Print("[WARN] Warning") // this will
log.Print("[ERROR] Erring") // and so will this
log.Print("Message I haven't updated") // and so will this
}
```
This logs to standard error exactly like go's standard logger. Any log messages you haven't converted to have a level will continue to print as before.
-81
View File
@@ -1,81 +0,0 @@
// Package logutils augments the standard log package with levels.
package logutils
import (
"bytes"
"io"
"sync"
)
type LogLevel string
// LevelFilter is an io.Writer that can be used with a logger that
// will filter out log messages that aren't at least a certain level.
//
// Once the filter is in use somewhere, it is not safe to modify
// the structure.
type LevelFilter struct {
// Levels is the list of log levels, in increasing order of
// severity. Example might be: {"DEBUG", "WARN", "ERROR"}.
Levels []LogLevel
// MinLevel is the minimum level allowed through
MinLevel LogLevel
// The underlying io.Writer where log messages that pass the filter
// will be set.
Writer io.Writer
badLevels map[LogLevel]struct{}
once sync.Once
}
// Check will check a given line if it would be included in the level
// filter.
func (f *LevelFilter) Check(line []byte) bool {
f.once.Do(f.init)
// Check for a log level
var level LogLevel
x := bytes.IndexByte(line, '[')
if x >= 0 {
y := bytes.IndexByte(line[x:], ']')
if y >= 0 {
level = LogLevel(line[x+1 : x+y])
}
}
_, ok := f.badLevels[level]
return !ok
}
func (f *LevelFilter) Write(p []byte) (n int, err error) {
// Note in general that io.Writer can receive any byte sequence
// to write, but the "log" package always guarantees that we only
// get a single line. We use that as a slight optimization within
// this method, assuming we're dealing with a single, complete line
// of log data.
if !f.Check(p) {
return len(p), nil
}
return f.Writer.Write(p)
}
// SetMinLevel is used to update the minimum log level
func (f *LevelFilter) SetMinLevel(min LogLevel) {
f.MinLevel = min
f.init()
}
func (f *LevelFilter) init() {
badLevels := make(map[LogLevel]struct{})
for _, level := range f.Levels {
if level == f.MinLevel {
break
}
badLevels[level] = struct{}{}
}
f.badLevels = badLevels
}
@@ -6,4 +6,5 @@
padding-right: 84px;
font-size: 14px;
line-height: 16px;
position: relative;
}
@@ -1,7 +1,10 @@
.comment__vote_disabled {
&,
&:hover {
background-image: url('comment__vote_disabled.svg');
cursor: not-allowed;
&.comment__vote_type_up,
&.comment__vote_type_down {
&,
&:hover {
background-image: url('comment__vote_disabled.svg');
cursor: not-allowed;
}
}
}
@@ -5,4 +5,22 @@
display: block;
}
}
.comment__title {
margin-bottom: 0.2rem;
}
.comment__title-link {
color: #0e7e9d;
font-weight: bold;
text-decoration: none;
text-overflow: ellipsis;
overflow: hidden;
white-space: nowrap;
display: block;
&:hover {
opacity: 0.75;
}
}
}

Some files were not shown because too many files have changed in this diff Show More