chore(deps): update Go modules

Bump Go dependencies in both backend/ and backend/_example/memory_store.

Notable updates:
- github.com/go-pkgz/lgr v0.12.1 -> v0.12.3
- github.com/klauspost/compress v1.18.2 -> v1.18.5
- github.com/PuerkitoBio/goquery v1.11.0 -> v1.12.0
- github.com/montanaflynn/stats v0.7.1 -> v0.9.0
- github.com/redis/go-redis/v9 v9.17.2 -> v9.18.0
- github.com/slack-go/slack v0.17.3 -> v0.21.1
- go.mongodb.org/mongo-driver v1.17.6 -> v1.17.9
- golang.org/x/crypto v0.48.0 -> v0.50.0
- golang.org/x/net v0.49.0 -> v0.53.0
- golang.org/x/image v0.36.0 -> v0.39.0
- golang.org/x/sys v0.41.0 -> v0.43.0
- golang.org/x/{oauth2,sync,text} minor bumps

Key markdown/sanitisation libs (bluemonday v1.0.27,
alecthomas/chroma/v2 v2.23.1, russross/blackfriday/v2 v2.1.0,
Depado/bfchroma/v2 v2.0.0) are already at the latest available
versions and were not bumped.

Verified the Chroma span-class allowlist regex in
backend/app/store/comment.go:128-131 is still fully in sync with
chroma/v2 types.go StandardTypes map (86 classes, byte-equal after
sorting). The inline comment references commit c263f6f which is
stale (Chroma is at v2 now), but the class list content is current.

Ran `go mod tidy` + `go mod vendor` + full race test suite on both
modules. All green. Added a reminder in CLAUDE.md that updating
backend/ Go modules also requires `go mod tidy` in
backend/_example/memory_store since the example module uses a
local replace directive and inherits indirect deps from the main
module.
This commit is contained in:
Dmitry Verkhoturov
2026-04-12 11:52:57 -05:00
committed by Umputun
parent bea67f0136
commit 80c12a3f10
218 changed files with 16297 additions and 1621 deletions
+2
View File
@@ -17,6 +17,8 @@
- **IMPORTANT**: Example lint: `cd backend/_example/memory_store && golangci-lint run --config ../../.golangci.yml`
- Frontend: `cd frontend && pnpm lint`
- **Before committing**: Always run tests and linter on both main backend AND examples
- **Dependency Updates**:
- When updating Go modules in `backend/`, also run `go mod tidy` (and `go mod vendor`) in `backend/_example/memory_store` to keep indirect deps in sync. The example module replaces `github.com/umputun/remark42/backend` with `../../` so stale indirect deps there will break the example build.
## Code Style
- **Backend**: Formatting with golangci-lint, strict error handling
+6 -6
View File
@@ -4,7 +4,7 @@ go 1.25.0
require (
github.com/go-pkgz/jrpc v0.4.0
github.com/go-pkgz/lgr v0.12.1
github.com/go-pkgz/lgr v0.12.3
github.com/jessevdk/go-flags v1.6.1
github.com/stretchr/testify v1.11.1
github.com/umputun/remark42/backend v1.1000.0
@@ -12,7 +12,7 @@ require (
require (
github.com/Depado/bfchroma/v2 v2.0.0 // indirect
github.com/PuerkitoBio/goquery v1.11.0 // indirect
github.com/PuerkitoBio/goquery v1.12.0 // indirect
github.com/alecthomas/chroma/v2 v2.23.1 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
@@ -30,10 +30,10 @@ require (
github.com/rs/xid v1.6.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
go.etcd.io/bbolt v1.4.3 // indirect
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/image v0.38.0 // indirect
golang.org/x/net v0.49.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/image v0.39.0 // indirect
golang.org/x/net v0.53.0 // indirect
golang.org/x/sys v0.43.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
+14 -14
View File
@@ -1,7 +1,7 @@
github.com/Depado/bfchroma/v2 v2.0.0 h1:IRpN9BPkNwEpR6w1ectIcNWOuhDSLx+8f1pn83fzxx8=
github.com/Depado/bfchroma/v2 v2.0.0/go.mod h1:wFwW/Pw8Tnd0irzgO9Zxtxgzp3aPS8qBWlyadxujxmw=
github.com/PuerkitoBio/goquery v1.11.0 h1:jZ7pwMQXIITcUXNH83LLk+txlaEy6NVOfTuP43xxfqw=
github.com/PuerkitoBio/goquery v1.11.0/go.mod h1:wQHgxUOU3JGuj3oD/QFfxUdlzW6xPHfqyHre6VMY4DQ=
github.com/PuerkitoBio/goquery v1.12.0 h1:pAcL4g3WRXekcB9AU/y1mbKez2dbY2AajVhtkO8RIBo=
github.com/PuerkitoBio/goquery v1.12.0/go.mod h1:802ej+gV2y7bbIhOIoPY5sT183ZW0YFofScC4q/hIpQ=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/chroma/v2 v2.23.1 h1:nv2AVZdTyClGbVQkIzlDm/rnhk1E9bU9nXwmZ/Vk/iY=
@@ -19,8 +19,8 @@ github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZ
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/go-pkgz/jrpc v0.4.0 h1:oD7xiGrzDkndkuCjeHGugQXxbggLSV7O1QmHhoc5pYY=
github.com/go-pkgz/jrpc v0.4.0/go.mod h1:JFoY3bRjRyx4M3CbEVDFQStMB1m2gmQ7OjqFK7q3kOo=
github.com/go-pkgz/lgr v0.12.1 h1:8GVfG2rSARq3Eaj5PP158rtBR2LHVGkwioIkQBGbvKg=
github.com/go-pkgz/lgr v0.12.1/go.mod h1:A4AxjOthFVFK6jRnVYMeusno5SeDAxcLVHd0kI/lN/Y=
github.com/go-pkgz/lgr v0.12.3 h1:QDug7kRkEsuQtruT9fNF5PVT2kZUqCDPc4GmsgS3fP8=
github.com/go-pkgz/lgr v0.12.3/go.mod h1:lpCDgVvCIxBHZp8+sGCj9MPctIzKZyZ3QdE19ddqd54=
github.com/go-pkgz/rest v1.21.0 h1:Y/C4d/TpclJJDxqnH1RAcS6Hmox0RIReAlkwMcUWXK4=
github.com/go-pkgz/rest v1.21.0/go.mod h1:+AHzjHazq7Z3Tk/kRWOhbbAz/YZlUV40feC1Hf4NtbE=
github.com/go-pkgz/routegroup v1.6.0 h1:44XHZgF6JIIldRlv+zjg6SygULASmjifnfIQjwCT0e4=
@@ -62,10 +62,10 @@ golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliY
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/image v0.38.0 h1:5l+q+Y9JDC7mBOMjo4/aPhMDcxEptsX+Tt3GgRQRPuE=
golang.org/x/image v0.38.0/go.mod h1:/3f6vaXC+6CEanU4KJxbcUZyEePbyKbaLoDOe4ehFYY=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/image v0.39.0 h1:skVYidAEVKgn8lZ602XO75asgXBgLj9G/FE3RbuPFww=
golang.org/x/image v0.39.0/go.mod h1:sIbmppfU+xFLPIG0FoVUTvyBMmgng1/XAMhQ2ft0hpA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
@@ -80,8 +80,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o=
golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -89,8 +89,8 @@ golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -102,8 +102,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+16 -15
View File
@@ -1,10 +1,10 @@
module github.com/umputun/remark42/backend
go 1.25
go 1.25.0
require (
github.com/Depado/bfchroma/v2 v2.0.0
github.com/PuerkitoBio/goquery v1.11.0
github.com/PuerkitoBio/goquery v1.12.0
github.com/alecthomas/chroma/v2 v2.23.1
github.com/didip/tollbooth/v8 v8.0.1
github.com/go-chi/chi/v5 v5.2.5
@@ -12,7 +12,7 @@ require (
github.com/go-pkgz/auth/v2 v2.1.2-0.20260211003156-fbba7f2baa6b
github.com/go-pkgz/jrpc v0.4.0
github.com/go-pkgz/lcw/v2 v2.0.0
github.com/go-pkgz/lgr v0.12.1
github.com/go-pkgz/lgr v0.12.3
github.com/go-pkgz/notify v1.3.0
github.com/go-pkgz/repeater/v2 v2.2.0
github.com/go-pkgz/rest v1.21.0
@@ -30,9 +30,9 @@ require (
github.com/stretchr/testify v1.11.1
go.etcd.io/bbolt v1.4.3
go.uber.org/goleak v1.3.0
golang.org/x/crypto v0.48.0
golang.org/x/image v0.36.0
golang.org/x/net v0.49.0
golang.org/x/crypto v0.50.0
golang.org/x/image v0.39.0
golang.org/x/net v0.53.0
)
require (
@@ -54,20 +54,21 @@ require (
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/montanaflynn/stats v0.7.1 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/montanaflynn/stats v0.9.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/redis/go-redis/v9 v9.17.2 // indirect
github.com/redis/go-redis/v9 v9.18.0 // indirect
github.com/rrivera/identicon v0.0.0-20240116195454-d5ba35832c0d // indirect
github.com/slack-go/slack v0.17.3 // indirect
github.com/slack-go/slack v0.21.1 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.2.0 // indirect
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
go.mongodb.org/mongo-driver v1.17.6 // indirect
golang.org/x/oauth2 v0.34.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/text v0.34.0 // indirect
go.mongodb.org/mongo-driver v1.17.9 // indirect
go.uber.org/atomic v1.11.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.43.0 // indirect
golang.org/x/text v0.36.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
+34 -28
View File
@@ -2,8 +2,8 @@ cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdB
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
github.com/Depado/bfchroma/v2 v2.0.0 h1:IRpN9BPkNwEpR6w1ectIcNWOuhDSLx+8f1pn83fzxx8=
github.com/Depado/bfchroma/v2 v2.0.0/go.mod h1:wFwW/Pw8Tnd0irzgO9Zxtxgzp3aPS8qBWlyadxujxmw=
github.com/PuerkitoBio/goquery v1.11.0 h1:jZ7pwMQXIITcUXNH83LLk+txlaEy6NVOfTuP43xxfqw=
github.com/PuerkitoBio/goquery v1.11.0/go.mod h1:wQHgxUOU3JGuj3oD/QFfxUdlzW6xPHfqyHre6VMY4DQ=
github.com/PuerkitoBio/goquery v1.12.0 h1:pAcL4g3WRXekcB9AU/y1mbKez2dbY2AajVhtkO8RIBo=
github.com/PuerkitoBio/goquery v1.12.0/go.mod h1:802ej+gV2y7bbIhOIoPY5sT183ZW0YFofScC4q/hIpQ=
github.com/ajg/form v1.5.1 h1:t9c7v8JUKu/XxOGBU0yjNpaMloxGEJhUkqFRq0ibGeU=
github.com/ajg/form v1.5.1/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
@@ -58,8 +58,8 @@ github.com/go-pkgz/jrpc v0.4.0 h1:oD7xiGrzDkndkuCjeHGugQXxbggLSV7O1QmHhoc5pYY=
github.com/go-pkgz/jrpc v0.4.0/go.mod h1:JFoY3bRjRyx4M3CbEVDFQStMB1m2gmQ7OjqFK7q3kOo=
github.com/go-pkgz/lcw/v2 v2.0.0 h1:gTwXpiJBhQeA1rXuqkRuLcV79uATFna8CckH8ZBBrH0=
github.com/go-pkgz/lcw/v2 v2.0.0/go.mod h1:yxJHOn+IbQBQHxUqkCtMrbGjIfdYcsBAZcVCBaL1Va8=
github.com/go-pkgz/lgr v0.12.1 h1:8GVfG2rSARq3Eaj5PP158rtBR2LHVGkwioIkQBGbvKg=
github.com/go-pkgz/lgr v0.12.1/go.mod h1:A4AxjOthFVFK6jRnVYMeusno5SeDAxcLVHd0kI/lN/Y=
github.com/go-pkgz/lgr v0.12.3 h1:QDug7kRkEsuQtruT9fNF5PVT2kZUqCDPc4GmsgS3fP8=
github.com/go-pkgz/lgr v0.12.3/go.mod h1:lpCDgVvCIxBHZp8+sGCj9MPctIzKZyZ3QdE19ddqd54=
github.com/go-pkgz/notify v1.3.0 h1:YxF/ThEoCetdcoghWdyeqaBpCkZ8mvyve7HXbCAOzYU=
github.com/go-pkgz/notify v1.3.0/go.mod h1:qdfi5OsViKlIFPryIOaINHTOtS9GFhOYXPqJmAMlaGU=
github.com/go-pkgz/repeater v1.2.0 h1:oJFvjyKdTDd5RCzpzxlzYIZFFj6Zfl17rE1aUfu6UjQ=
@@ -108,8 +108,10 @@ github.com/jessevdk/go-flags v1.6.1 h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bB
github.com/jessevdk/go-flags v1.6.1/go.mod h1:Mk8T1hIAWpOiJiHa9rJASDK2UGWji0EuPGBnNLMooyc=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@@ -118,14 +120,14 @@ github.com/kyokomi/emoji/v2 v2.2.13 h1:GhTfQa67venUUvmleTNFnb+bi7S3aocF7ZCXU9fSO
github.com/kyokomi/emoji/v2 v2.2.13/go.mod h1:JUcn42DTdsXJo1SWanHh4HKDEyPaR5CqkmoirZZP9qE=
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
github.com/montanaflynn/stats v0.7.1 h1:etflOAAHORrCC44V+aR6Ftzort912ZU+YLiSTuV8eaE=
github.com/montanaflynn/stats v0.7.1/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow=
github.com/montanaflynn/stats v0.9.0 h1:tsBJ0RXwph9BmAuFoCmqGv6e8xa0MENQ8m0ptKq29mQ=
github.com/montanaflynn/stats v0.9.0/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow=
github.com/moul/http2curl v1.0.0 h1:dRMWoAtb+ePxMlLkrCbAqh4TlPHXvoGUSQ323/9Zahs=
github.com/moul/http2curl v1.0.0/go.mod h1:8UbvGypXm98wA/IqH45anm5Y2Z6ep6O31QGOAZ3H0fQ=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/redis/go-redis/v9 v9.17.2 h1:P2EGsA4qVIM3Pp+aPocCJ7DguDHhqrXNhVcEp4ViluI=
github.com/redis/go-redis/v9 v9.17.2/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
github.com/redis/go-redis/v9 v9.18.0 h1:pMkxYPkEbMPwRdenAzUNyFNrDgHx9U+DrBabWNfSRQs=
github.com/redis/go-redis/v9 v9.18.0/go.mod h1:k3ufPphLU5YXwNTUcCRXGxUoF1fqxnhFQmscfkCoDA0=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rrivera/identicon v0.0.0-20240116195454-d5ba35832c0d h1:l3+2LWCbVxn5itfvXAfH9n4YL9jh8l1g5zcncbIc1cs=
@@ -138,8 +140,8 @@ github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e h1:MRM5ITcdelLK2j1vwZ3Je0FKVCfqOLp5zO6trqMLYs0=
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e/go.mod h1:XV66xRDqSt+GTGFMVlhk3ULuV0y9ZmzeVGR4mloJI3M=
github.com/slack-go/slack v0.17.3 h1:zV5qO3Q+WJAQ/XwbGfNFrRMaJ5T/naqaonyPV/1TP4g=
github.com/slack-go/slack v0.17.3/go.mod h1:X+UqOufi3LYQHDnMG1vxf0J8asC6+WllXrVrhl8/Prk=
github.com/slack-go/slack v0.21.1 h1:vBHR+IkaXbv9RLY6w/RiN82D+5/OTI06CGqrlZ3Vyas=
github.com/slack-go/slack v0.21.1/go.mod h1:K81UmCivcYd/5Jmz8vLBfuyoZ3B4rQC2GHVXHteXiAE=
github.com/smartystreets/assertions v1.1.0 h1:MkTeG1DMwsrdH7QtLXy5W+fUxWq+vmb6cLmyJ7aRtF0=
github.com/smartystreets/assertions v1.1.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo=
github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s=
@@ -189,10 +191,14 @@ github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82/go.mod h1:lgjkn3NuSvDf
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.mongodb.org/mongo-driver v1.17.6 h1:87JUG1wZfWsr6rIz3ZmpH90rL5tea7O3IHuSwHUpsss=
go.mongodb.org/mongo-driver v1.17.6/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ=
go.mongodb.org/mongo-driver v1.17.9 h1:IexDdCuuNJ3BHrELgBlyaH9p60JXAvdzWR128q+U5tU=
go.mongodb.org/mongo-driver v1.17.9/go.mod h1:LlOhpH5NUEfhxcAwG0UEkMqwYcc4JU18gtCdGudk/tQ=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
@@ -201,10 +207,10 @@ golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliY
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/image v0.36.0 h1:Iknbfm1afbgtwPTmHnS2gTM/6PPZfH+z2EFuOkSbqwc=
golang.org/x/image v0.36.0/go.mod h1:YsWD2TyyGKiIX1kZlu9QfKIsQ4nAAK9bdgdrIsE7xy4=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/image v0.39.0 h1:skVYidAEVKgn8lZ602XO75asgXBgLj9G/FE3RbuPFww=
golang.org/x/image v0.39.0/go.mod h1:sIbmppfU+xFLPIG0FoVUTvyBMmgng1/XAMhQ2ft0hpA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
@@ -219,10 +225,10 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o=
golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8=
golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw=
golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -230,8 +236,8 @@ golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -243,8 +249,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -264,8 +270,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+2
View File
@@ -24,6 +24,7 @@ Syntax-wise, it is as close as possible to jQuery, with the same function names
Required Go version:
* Starting with version `v1.12.0` of goquery, Go 1.25+ is required due to its dependencies.
* Starting with version `v1.11.0` of goquery, Go 1.24+ is required due to its dependencies.
* Starting with version `v1.10.0` of goquery, Go 1.23+ is required due to the use of function-based iterators.
* For `v1.9.0` of goquery, Go 1.18+ is required due to the use of generics.
@@ -47,6 +48,7 @@ Ongoing goquery development is tested on the latest 2 versions of Go.
**Note that goquery's API is now stable, and will not break.**
* **2026-03-15 (v1.12.0)** : Update `go.mod` dependencies, add go1.26 to the test matrix, **goquery now requires Go version 1.25+**.
* **2025-11-16 (v1.11.0)** : Update `go.mod` dependencies, add go1.25 to the test matrix, **goquery now requires Go version 1.24+**.
* **2025-04-11 (v1.10.3)** : Update `go.mod` dependencies, small optimization (thanks [@myxzlpltk](https://github.com/myxzlpltk)).
* **2025-02-13 (v1.10.2)** : Update `go.mod` dependencies, add go1.24 to the test matrix.
+28
View File
@@ -0,0 +1,28 @@
# Go-PKGZ/LGR Development Guidelines
## Build & Test Commands
- Build: `go build -race`
- Test all: `go test -timeout=60s -race -covermode=atomic -coverprofile=profile.cov`
- Test single file: `go test -run TestName`
- Benchmark: `go test -bench=. -run=Bench`
- Lint: `golangci-lint run`
## Code Style Guidelines
- Go 1.21 compatibility required
- Maximum line length: 140 characters
- No package names with underscores
- Use early returns (enforced by prealloc linter)
- Test files use testify for assertions: `require` for fatal assertions, `assert` for non-fatal ones
- Indent with tabs, not spaces
## Error Handling
- FATAL logs to stderr and calls os.Exit(1)
- ERROR logs to both stdout and stderr
- PANIC logs stack trace and runtime info to stderr
- Stack traces for ERROR level can be enabled with StackTraceOnError option
## Project Conventions
- Public API follows interface-based design (`lgr.L` interface)
- Avoid global loggers, prefer dependency injection
- Functional options pattern for logger configuration
- Secret logging sanitization with `lgr.Secret` option
+4 -3
View File
@@ -166,9 +166,10 @@ func (l *Logger) logf(format string, args ...interface{}) {
// if slog handler is set, use it
if l.slogHandler != nil {
// use NewRecord for consistency with adapter setup
// skip=0 because we don't need caller information from this context
record := slog.NewRecord(l.now(), stringToLevel(lv), msg, 0)
// get the caller's PC so slog handlers can resolve source info when AddSource is enabled
var pcs [1]uintptr
runtime.Callers(3+l.callerDepth, pcs[:]) // skip runtime.Callers, logf, Logf (+ any extra depth)
record := slog.NewRecord(l.now(), stringToLevel(lv), msg, pcs[0])
_ = l.slogHandler.Handle(context.Background(), record)
// handle FATAL and PANIC levels as they have special behavior
+8 -38
View File
@@ -23,13 +23,13 @@ func FromSlogHandler(h slog.Handler) L {
// SetupWithSlog sets up the global logger with a slog logger
func SetupWithSlog(logger *slog.Logger) {
options := []Option{SlogHandler(logger.Handler())}
// check if the slog handler is enabled for debug level
// if so, enable debug mode in lgr to prevent filtering
if logger.Handler().Enabled(context.Background(), slog.LevelDebug) {
options = append(options, Debug)
}
Setup(options...)
}
@@ -59,12 +59,6 @@ func (h *lgrSlogHandler) Handle(_ context.Context, record slog.Record) error {
// build message with attributes
msg := record.Message
// add time if record has it, otherwise current time is used by lgr
var timeStr string
if !record.Time.IsZero() {
timeStr = record.Time.Format("2006/01/02 15:04:05.000 ")
}
// format attributes as key=value pairs
var attrs strings.Builder
if len(h.attrs) > 0 || record.NumAttrs() > 0 {
@@ -82,8 +76,8 @@ func (h *lgrSlogHandler) Handle(_ context.Context, record slog.Record) error {
return true
})
// combine everything into final message
logMsg := fmt.Sprintf("%s%s %s%s", timeStr, level, msg, attrs.String())
// combine level prefix and message; lgr.Logf adds its own timestamp and level formatting
logMsg := fmt.Sprintf("%s %s%s", level, msg, attrs.String())
h.lgr.Logf(logMsg)
return nil
}
@@ -115,39 +109,15 @@ type slogLgrAdapter struct {
// Logf implements lgr.L interface
func (a *slogLgrAdapter) Logf(format string, args ...interface{}) {
// parse log level from the beginning of the message
msg := fmt.Sprintf(format, args...)
level, msg := extractLevel(msg)
// create a record with caller information
// skip level is critical:
// - 0 = this line
// - 1 = this function (Logf)
// - 2 = caller of Logf (user code)
//
// note: We use PC=0 to ensure slog.Record.PC() returns 0,
// which causes slog to skip obtaining the caller info itself
record := slog.NewRecord(time.Now(), stringToLevel(level), msg, 2)
// get the caller's PC so slog handlers can resolve source info when AddSource is enabled
var pcs [1]uintptr
runtime.Callers(2, pcs[:]) // skip runtime.Callers and Logf
record := slog.NewRecord(time.Now(), stringToLevel(level), msg, pcs[0])
// we need to manually add the source information ourselves, since
// slog.Handler might have AddSource=true but won't get the caller
// right due to how we're adapting lgr → slog
pc, file, line, ok := runtime.Caller(2) // skip to caller of Logf
if ok {
// only add source info if we can find it
funcName := runtime.FuncForPC(pc).Name()
record.AddAttrs(
slog.Group("source",
slog.String("function", funcName),
slog.String("file", file),
slog.Int("line", line),
),
)
}
// handle the record
if err := a.handler.Handle(context.Background(), record); err != nil {
// if handling fails, fallback to stderr
fmt.Fprintf(os.Stderr, "slog handler error: %v\n", err)
}
}
+10 -1
View File
@@ -31,6 +31,9 @@ builds:
- mips64le
goarm:
- 7
ignore:
- goos: windows
goarch: arm
-
id: "s2d"
binary: s2d
@@ -57,6 +60,9 @@ builds:
- mips64le
goarm:
- 7
ignore:
- goos: windows
goarch: arm
-
id: "s2sx"
binary: s2sx
@@ -84,6 +90,9 @@ builds:
- mips64le
goarm:
- 7
ignore:
- goos: windows
goarch: arm
archives:
-
@@ -91,7 +100,7 @@ archives:
name_template: "s2-{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"
format_overrides:
- goos: windows
format: zip
formats: ['zip']
files:
- unpack/*
- s2/LICENSE
+22 -8
View File
@@ -7,7 +7,7 @@ This package provides various compression algorithms.
* Optimized [deflate](https://godoc.org/github.com/klauspost/compress/flate) packages which can be used as a dropin replacement for [gzip](https://godoc.org/github.com/klauspost/compress/gzip), [zip](https://godoc.org/github.com/klauspost/compress/zip) and [zlib](https://godoc.org/github.com/klauspost/compress/zlib).
* [snappy](https://github.com/klauspost/compress/tree/master/snappy) is a drop-in replacement for `github.com/golang/snappy` offering better compression and concurrent streams.
* [huff0](https://github.com/klauspost/compress/tree/master/huff0) and [FSE](https://github.com/klauspost/compress/tree/master/fse) implementations for raw entropy encoding.
* [gzhttp](https://github.com/klauspost/compress/tree/master/gzhttp) Provides client and server wrappers for handling gzipped requests efficiently.
* [gzhttp](https://github.com/klauspost/compress/tree/master/gzhttp) Provides client and server wrappers for handling gzipped/zstd HTTP requests efficiently.
* [pgzip](https://github.com/klauspost/pgzip) is a separate package that provides a very fast parallel gzip implementation.
[![Go Reference](https://pkg.go.dev/badge/klauspost/compress.svg)](https://pkg.go.dev/github.com/klauspost/compress?tab=subdirectories)
@@ -27,7 +27,19 @@ Use the links above for more information on each.
# changelog
* Oct 20, 2025 - [1.18.1](https://github.com/klauspost/compress/releases/tag/v1.18.1)
* Feb 9th, 2026 [1.18.4](https://github.com/klauspost/compress/releases/tag/v1.18.4)
* gzhttp: Add zstandard to server handler wrapper https://github.com/klauspost/compress/pull/1121
* zstd: Add ResetWithOptions to encoder/decoder https://github.com/klauspost/compress/pull/1122
* gzhttp: preserve qvalue when extra parameters follow in Accept-Encoding by @analytically in https://github.com/klauspost/compress/pull/1116
* Jan 16th, 2026 [1.18.3](https://github.com/klauspost/compress/releases/tag/v1.18.3)
* Downstream CVE-2025-61728. See [golang/go#77102](https://github.com/golang/go/issues/77102).
* Dec 1st, 2025 - [1.18.2](https://github.com/klauspost/compress/releases/tag/v1.18.2)
* flate: Fix invalid encoding on level 9 with single value input in https://github.com/klauspost/compress/pull/1115
* flate: reduce stateless allocations by @RXamzin in https://github.com/klauspost/compress/pull/1106
* Oct 20, 2025 - [1.18.1](https://github.com/klauspost/compress/releases/tag/v1.18.1) - RETRACTED
* zstd: Add simple zstd EncodeTo/DecodeTo functions https://github.com/klauspost/compress/pull/1079
* zstd: Fix incorrect buffer size in dictionary encodes https://github.com/klauspost/compress/pull/1059
* s2: check for cap, not len of buffer in EncodeBetter/Best by @vdarulis in https://github.com/klauspost/compress/pull/1080
@@ -603,7 +615,7 @@ While the release has been extensively tested, it is recommended to testing when
# deflate usage
The packages are drop-in replacements for standard libraries. Simply replace the import path to use them:
The packages are drop-in replacements for standard library [deflate](https://godoc.org/github.com/klauspost/compress/flate), [gzip](https://godoc.org/github.com/klauspost/compress/gzip), [zip](https://godoc.org/github.com/klauspost/compress/zip), and [zlib](https://godoc.org/github.com/klauspost/compress/zlib). Simply replace the import path to use them:
Typical speed is about 2x of the standard library packages.
@@ -614,17 +626,15 @@ Typical speed is about 2x of the standard library packages.
| `archive/zip` | `github.com/klauspost/compress/zip` | [zip](https://pkg.go.dev/github.com/klauspost/compress/zip?tab=doc) |
| `compress/flate` | `github.com/klauspost/compress/flate` | [flate](https://pkg.go.dev/github.com/klauspost/compress/flate?tab=doc) |
* Optimized [deflate](https://godoc.org/github.com/klauspost/compress/flate) packages which can be used as a dropin replacement for [gzip](https://godoc.org/github.com/klauspost/compress/gzip), [zip](https://godoc.org/github.com/klauspost/compress/zip) and [zlib](https://godoc.org/github.com/klauspost/compress/zlib).
You may also be interested in [pgzip](https://github.com/klauspost/pgzip), which is a drop-in replacement for gzip, which support multithreaded compression on big files and the optimized [crc32](https://github.com/klauspost/crc32) package used by these packages.
You may also be interested in [pgzip](https://github.com/klauspost/pgzip), which is a drop in replacement for gzip, which support multithreaded compression on big files and the optimized [crc32](https://github.com/klauspost/crc32) package used by these packages.
The packages contains the same as the standard library, so you can use the godoc for that: [gzip](http://golang.org/pkg/compress/gzip/), [zip](http://golang.org/pkg/archive/zip/), [zlib](http://golang.org/pkg/compress/zlib/), [flate](http://golang.org/pkg/compress/flate/).
The packages implement the same API as the standard library, so you can use the original godoc documentation: [gzip](http://golang.org/pkg/compress/gzip/), [zip](http://golang.org/pkg/archive/zip/), [zlib](http://golang.org/pkg/compress/zlib/), [flate](http://golang.org/pkg/compress/flate/).
Currently there is only minor speedup on decompression (mostly CRC32 calculation).
Memory usage is typically 1MB for a Writer. stdlib is in the same range.
If you expect to have a lot of concurrently allocated Writers consider using
the stateless compress described below.
the stateless compression described below.
For compression performance, see: [this spreadsheet](https://docs.google.com/spreadsheets/d/1nuNE2nPfuINCZJRMt6wFWhKpToF95I47XjSsc-1rbPQ/edit?usp=sharing).
@@ -684,3 +694,7 @@ Here are other packages of good quality and pure Go (no cgo wrappers or autoconv
This code is licensed under the same conditions as the original Go code. See LICENSE file.
@@ -1,5 +1,4 @@
//go:build amd64 && !appengine && !noasm && gc
// +build amd64,!appengine,!noasm,gc
// This file contains the specialisation of Decoder.Decompress4X
// and Decoder.Decompress1X that use an asm implementation of thir main loops.
@@ -1,5 +1,4 @@
//go:build !amd64 || appengine || !gc || noasm
// +build !amd64 appengine !gc noasm
// This file contains a generic implementation of Decoder.Decompress4X.
package huff0
@@ -1,5 +1,4 @@
//go:build amd64 && !appengine && !noasm && gc
// +build amd64,!appengine,!noasm,gc
package cpuinfo
+1
View File
@@ -78,6 +78,7 @@ func (b *blockEnc) initNewEncode() {
b.recentOffsets = [3]uint32{1, 4, 8}
b.litEnc.Reuse = huff0.ReusePolicyNone
b.coders.setPrev(nil, nil, nil)
b.dictLitEnc = nil
}
// reset will reset the block for a new encode, but in the same stream,
+19 -9
View File
@@ -39,9 +39,6 @@ type Decoder struct {
frame *frameDec
// Custom dictionaries.
dicts map[uint32]*dict
// streamWg is the waitgroup for all streams
streamWg sync.WaitGroup
}
@@ -101,12 +98,10 @@ func NewReader(r io.Reader, opts ...DOption) (*Decoder, error) {
d.current.err = ErrDecoderNilInput
}
// Transfer option dicts.
d.dicts = make(map[uint32]*dict, len(d.o.dicts))
for _, dc := range d.o.dicts {
d.dicts[dc.id] = dc
// Initialize dict map if needed.
if d.o.dicts == nil {
d.o.dicts = make(map[uint32]*dict)
}
d.o.dicts = nil
// Create decoders
d.decoders = make(chan *blockDec, d.o.concurrent)
@@ -238,6 +233,21 @@ func (d *Decoder) Reset(r io.Reader) error {
return nil
}
// ResetWithOptions will reset the decoder and apply the given options
// for the next stream or DecodeAll operation.
// Options are applied on top of the existing options.
// Some options cannot be changed on reset and will return an error.
func (d *Decoder) ResetWithOptions(r io.Reader, opts ...DOption) error {
d.o.resetOpt = true
defer func() { d.o.resetOpt = false }()
for _, o := range opts {
if err := o(&d.o); err != nil {
return err
}
}
return d.Reset(r)
}
// drainOutput will drain the output until errEndOfStream is sent.
func (d *Decoder) drainOutput() {
if d.current.cancel != nil {
@@ -930,7 +940,7 @@ decodeStream:
}
func (d *Decoder) setDict(frame *frameDec) (err error) {
dict, ok := d.dicts[frame.DictionaryID]
dict, ok := d.o.dicts[frame.DictionaryID]
if ok {
if debugDecoder {
println("setting dict", frame.DictionaryID)
+52 -8
View File
@@ -20,10 +20,11 @@ type decoderOptions struct {
concurrent int
maxDecodedSize uint64
maxWindowSize uint64
dicts []*dict
dicts map[uint32]*dict
ignoreChecksum bool
limitToCap bool
decodeBufsBelow int
resetOpt bool
}
func (o *decoderOptions) setDefault() {
@@ -42,8 +43,15 @@ func (o *decoderOptions) setDefault() {
// WithDecoderLowmem will set whether to use a lower amount of memory,
// but possibly have to allocate more while running.
// Cannot be changed with ResetWithOptions.
func WithDecoderLowmem(b bool) DOption {
return func(o *decoderOptions) error { o.lowMem = b; return nil }
return func(o *decoderOptions) error {
if o.resetOpt && b != o.lowMem {
return errors.New("WithDecoderLowmem cannot be changed on Reset")
}
o.lowMem = b
return nil
}
}
// WithDecoderConcurrency sets the number of created decoders.
@@ -53,18 +61,23 @@ func WithDecoderLowmem(b bool) DOption {
// inflight blocks.
// When decoding streams and setting maximum to 1,
// no async decoding will be done.
// The value supplied must be at least 0.
// When a value of 0 is provided GOMAXPROCS will be used.
// By default this will be set to 4 or GOMAXPROCS, whatever is lower.
// Cannot be changed with ResetWithOptions.
func WithDecoderConcurrency(n int) DOption {
return func(o *decoderOptions) error {
if n < 0 {
return errors.New("concurrency must be at least 1")
return errors.New("concurrency must be at least 0")
}
newVal := n
if n == 0 {
o.concurrent = runtime.GOMAXPROCS(0)
} else {
o.concurrent = n
newVal = runtime.GOMAXPROCS(0)
}
if o.resetOpt && newVal != o.concurrent {
return errors.New("WithDecoderConcurrency cannot be changed on Reset")
}
o.concurrent = newVal
return nil
}
}
@@ -73,6 +86,7 @@ func WithDecoderConcurrency(n int) DOption {
// non-streaming operations or maximum window size for streaming operations.
// This can be used to control memory usage of potentially hostile content.
// Maximum is 1 << 63 bytes. Default is 64GiB.
// Can be changed with ResetWithOptions.
func WithDecoderMaxMemory(n uint64) DOption {
return func(o *decoderOptions) error {
if n == 0 {
@@ -92,16 +106,20 @@ func WithDecoderMaxMemory(n uint64) DOption {
// "zstd --train" from the Zstandard reference implementation.
//
// If several dictionaries with the same ID are provided, the last one will be used.
// Can be changed with ResetWithOptions.
//
// [dictionary format]: https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#dictionary-format
func WithDecoderDicts(dicts ...[]byte) DOption {
return func(o *decoderOptions) error {
if o.dicts == nil {
o.dicts = make(map[uint32]*dict)
}
for _, b := range dicts {
d, err := loadDict(b)
if err != nil {
return err
}
o.dicts = append(o.dicts, d)
o.dicts[d.id] = d
}
return nil
}
@@ -109,12 +127,16 @@ func WithDecoderDicts(dicts ...[]byte) DOption {
// WithDecoderDictRaw registers a dictionary that may be used by the decoder.
// The slice content can be arbitrary data.
// Can be changed with ResetWithOptions.
func WithDecoderDictRaw(id uint32, content []byte) DOption {
return func(o *decoderOptions) error {
if bits.UintSize > 32 && uint(len(content)) > dictMaxLength {
return fmt.Errorf("dictionary of size %d > 2GiB too large", len(content))
}
o.dicts = append(o.dicts, &dict{id: id, content: content, offsets: [3]int{1, 4, 8}})
if o.dicts == nil {
o.dicts = make(map[uint32]*dict)
}
o.dicts[id] = &dict{id: id, content: content, offsets: [3]int{1, 4, 8}}
return nil
}
}
@@ -124,6 +146,7 @@ func WithDecoderDictRaw(id uint32, content []byte) DOption {
// The Decoder will likely allocate more memory based on the WithDecoderLowmem setting.
// If WithDecoderMaxMemory is set to a lower value, that will be used.
// Default is 512MB, Maximum is ~3.75 TB as per zstandard spec.
// Can be changed with ResetWithOptions.
func WithDecoderMaxWindow(size uint64) DOption {
return func(o *decoderOptions) error {
if size < MinWindowSize {
@@ -141,6 +164,7 @@ func WithDecoderMaxWindow(size uint64) DOption {
// or any size set in WithDecoderMaxMemory.
// This can be used to limit decoding to a specific maximum output size.
// Disabled by default.
// Can be changed with ResetWithOptions.
func WithDecodeAllCapLimit(b bool) DOption {
return func(o *decoderOptions) error {
o.limitToCap = b
@@ -153,17 +177,37 @@ func WithDecodeAllCapLimit(b bool) DOption {
// This typically uses less allocations but will have the full decompressed object in memory.
// Note that DecodeAllCapLimit will disable this, as well as giving a size of 0 or less.
// Default is 128KiB.
// Cannot be changed with ResetWithOptions.
func WithDecodeBuffersBelow(size int) DOption {
return func(o *decoderOptions) error {
if o.resetOpt && size != o.decodeBufsBelow {
return errors.New("WithDecodeBuffersBelow cannot be changed on Reset")
}
o.decodeBufsBelow = size
return nil
}
}
// IgnoreChecksum allows to forcibly ignore checksum checking.
// Can be changed with ResetWithOptions.
func IgnoreChecksum(b bool) DOption {
return func(o *decoderOptions) error {
o.ignoreChecksum = b
return nil
}
}
// WithDecoderDictDelete removes dictionaries by ID.
// If no ids are passed, all dictionaries are deleted.
// Should be used with ResetWithOptions.
func WithDecoderDictDelete(ids ...uint32) DOption {
return func(o *decoderOptions) error {
if len(ids) == 0 {
clear(o.dicts)
}
for _, id := range ids {
delete(o.dicts, id)
}
return nil
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ type fastBase struct {
crc *xxhash.Digest
tmp [8]byte
blk *blockEnc
lastDictID uint32
lastDict *dict
lowMem bool
}
+9 -5
View File
@@ -479,10 +479,13 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
if d == nil {
return
}
dictChanged := d != e.lastDict
// Init or copy dict table
if len(e.dictTable) != len(e.table) || d.id != e.lastDictID {
if len(e.dictTable) != len(e.table) || dictChanged {
if len(e.dictTable) != len(e.table) {
e.dictTable = make([]prevEntry, len(e.table))
} else {
clear(e.dictTable)
}
end := int32(len(d.content)) - 8 + e.maxMatchOff
for i := e.maxMatchOff; i < end; i += 4 {
@@ -510,13 +513,14 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
offset: i + 3,
}
}
e.lastDictID = d.id
}
// Init or copy dict table
if len(e.dictLongTable) != len(e.longTable) || d.id != e.lastDictID {
// Init or copy dict long table
if len(e.dictLongTable) != len(e.longTable) || dictChanged {
if len(e.dictLongTable) != len(e.longTable) {
e.dictLongTable = make([]prevEntry, len(e.longTable))
} else {
clear(e.dictLongTable)
}
if len(d.content) >= 8 {
cv := load6432(d.content, 0)
@@ -538,8 +542,8 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
off++
}
}
e.lastDictID = d.id
}
e.lastDict = d
// Reset table to initial state
copy(e.longTable[:], e.dictLongTable)
+9 -5
View File
@@ -1102,10 +1102,13 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
if d == nil {
return
}
dictChanged := d != e.lastDict
// Init or copy dict table
if len(e.dictTable) != len(e.table) || d.id != e.lastDictID {
if len(e.dictTable) != len(e.table) || dictChanged {
if len(e.dictTable) != len(e.table) {
e.dictTable = make([]tableEntry, len(e.table))
} else {
clear(e.dictTable)
}
end := int32(len(d.content)) - 8 + e.maxMatchOff
for i := e.maxMatchOff; i < end; i += 4 {
@@ -1133,14 +1136,15 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
offset: i + 3,
}
}
e.lastDictID = d.id
e.allDirty = true
}
// Init or copy dict table
if len(e.dictLongTable) != len(e.longTable) || d.id != e.lastDictID {
// Init or copy dict long table
if len(e.dictLongTable) != len(e.longTable) || dictChanged {
if len(e.dictLongTable) != len(e.longTable) {
e.dictLongTable = make([]prevEntry, len(e.longTable))
} else {
clear(e.dictLongTable)
}
if len(d.content) >= 8 {
cv := load6432(d.content, 0)
@@ -1162,9 +1166,9 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
off++
}
}
e.lastDictID = d.id
e.allDirty = true
}
e.lastDict = d
// Reset table to initial state
{
+4 -2
View File
@@ -1040,15 +1040,18 @@ func (e *doubleFastEncoder) Reset(d *dict, singleBlock bool) {
// ResetDict will reset and set a dictionary if not nil
func (e *doubleFastEncoderDict) Reset(d *dict, singleBlock bool) {
allDirty := e.allDirty
dictChanged := d != e.lastDict
e.fastEncoderDict.Reset(d, singleBlock)
if d == nil {
return
}
// Init or copy dict table
if len(e.dictLongTable) != len(e.longTable) || d.id != e.lastDictID {
if len(e.dictLongTable) != len(e.longTable) || dictChanged {
if len(e.dictLongTable) != len(e.longTable) {
e.dictLongTable = make([]tableEntry, len(e.longTable))
} else {
clear(e.dictLongTable)
}
if len(d.content) >= 8 {
cv := load6432(d.content, 0)
@@ -1065,7 +1068,6 @@ func (e *doubleFastEncoderDict) Reset(d *dict, singleBlock bool) {
}
}
}
e.lastDictID = d.id
allDirty = true
}
// Reset table to initial state
+4 -2
View File
@@ -805,9 +805,11 @@ func (e *fastEncoderDict) Reset(d *dict, singleBlock bool) {
}
// Init or copy dict table
if len(e.dictTable) != len(e.table) || d.id != e.lastDictID {
if len(e.dictTable) != len(e.table) || d != e.lastDict {
if len(e.dictTable) != len(e.table) {
e.dictTable = make([]tableEntry, len(e.table))
} else {
clear(e.dictTable)
}
if true {
end := e.maxMatchOff + int32(len(d.content)) - 8
@@ -827,7 +829,7 @@ func (e *fastEncoderDict) Reset(d *dict, singleBlock bool) {
}
}
}
e.lastDictID = d.id
e.lastDict = d
e.allDirty = true
}
+29
View File
@@ -131,6 +131,29 @@ func (e *Encoder) Reset(w io.Writer) {
s.frameContentSize = 0
}
// ResetWithOptions will re-initialize the writer and apply the given options
// as a new, independent stream.
// Options are applied on top of the existing options.
// Some options cannot be changed on reset and will return an error.
func (e *Encoder) ResetWithOptions(w io.Writer, opts ...EOption) error {
e.o.resetOpt = true
defer func() { e.o.resetOpt = false }()
hadDict := e.o.dict != nil
for _, o := range opts {
if err := o(&e.o); err != nil {
return err
}
}
hasDict := e.o.dict != nil
if hadDict != hasDict {
// Dict presence changed — encoder type must be recreated.
e.state.encoder = nil
e.init = sync.Once{}
}
e.Reset(w)
return nil
}
// ResetContentSize will reset and set a content size for the next stream.
// If the bytes written does not match the size given an error will be returned
// when calling Close().
@@ -432,6 +455,12 @@ func (e *Encoder) Close() error {
if s.encoder == nil {
return nil
}
if s.w == nil {
if len(s.filling) == 0 && !s.headerWritten && !s.eofWritten && s.nInput == 0 {
return nil
}
return errors.New("zstd: encoder has no writer")
}
err := e.nextBlock(true)
if err != nil {
if errors.Is(s.err, ErrEncoderClosed) {
+42 -3
View File
@@ -14,6 +14,7 @@ type EOption func(*encoderOptions) error
// options retains accumulated state of multiple options.
type encoderOptions struct {
resetOpt bool
concurrent int
level EncoderLevel
single *bool
@@ -41,6 +42,7 @@ func (o *encoderOptions) setDefault() {
level: SpeedDefault,
allLitEntropy: false,
lowMem: false,
fullZero: true,
}
}
@@ -71,19 +73,28 @@ func (o encoderOptions) encoder() encoder {
// WithEncoderCRC will add CRC value to output.
// Output will be 4 bytes larger.
// Can be changed with ResetWithOptions.
func WithEncoderCRC(b bool) EOption {
return func(o *encoderOptions) error { o.crc = b; return nil }
}
// WithEncoderConcurrency will set the concurrency,
// meaning the maximum number of encoders to run concurrently.
// The value supplied must be at least 1.
// The value supplied must be at least 0.
// When a value of 0 is provided GOMAXPROCS will be used.
// For streams, setting a value of 1 will disable async compression.
// By default this will be set to GOMAXPROCS.
// Cannot be changed with ResetWithOptions.
func WithEncoderConcurrency(n int) EOption {
return func(o *encoderOptions) error {
if n <= 0 {
return fmt.Errorf("concurrency must be at least 1")
if n < 0 {
return errors.New("concurrency must at least 0")
}
if n == 0 {
n = runtime.GOMAXPROCS(0)
}
if o.resetOpt && n != o.concurrent {
return errors.New("WithEncoderConcurrency cannot be changed on Reset")
}
o.concurrent = n
return nil
@@ -95,6 +106,7 @@ func WithEncoderConcurrency(n int) EOption {
// A larger value will enable better compression but allocate more memory and,
// for above-default values, take considerably longer.
// The default value is determined by the compression level and max 8MB.
// Cannot be changed with ResetWithOptions.
func WithWindowSize(n int) EOption {
return func(o *encoderOptions) error {
switch {
@@ -105,6 +117,9 @@ func WithWindowSize(n int) EOption {
case (n & (n - 1)) != 0:
return errors.New("window size must be a power of 2")
}
if o.resetOpt && n != o.windowSize {
return errors.New("WithWindowSize cannot be changed on Reset")
}
o.windowSize = n
o.customWindow = true
@@ -122,6 +137,7 @@ func WithWindowSize(n int) EOption {
// n must be > 0 and <= 1GB, 1<<30 bytes.
// The padded area will be filled with data from crypto/rand.Reader.
// If `EncodeAll` is used with data already in the destination, the total size will be multiple of this.
// Can be changed with ResetWithOptions.
func WithEncoderPadding(n int) EOption {
return func(o *encoderOptions) error {
if n <= 0 {
@@ -215,12 +231,16 @@ func (e EncoderLevel) String() string {
}
// WithEncoderLevel specifies a predefined compression level.
// Cannot be changed with ResetWithOptions.
func WithEncoderLevel(l EncoderLevel) EOption {
return func(o *encoderOptions) error {
switch {
case l <= speedNotSet || l >= speedLast:
return fmt.Errorf("unknown encoder level")
}
if o.resetOpt && l != o.level {
return errors.New("WithEncoderLevel cannot be changed on Reset")
}
o.level = l
if !o.customWindow {
switch o.level {
@@ -248,6 +268,7 @@ func WithEncoderLevel(l EncoderLevel) EOption {
// WithZeroFrames will encode 0 length input as full frames.
// This can be needed for compatibility with zstandard usage,
// but is not needed for this package.
// Can be changed with ResetWithOptions.
func WithZeroFrames(b bool) EOption {
return func(o *encoderOptions) error {
o.fullZero = b
@@ -259,6 +280,7 @@ func WithZeroFrames(b bool) EOption {
// Disabling this will skip incompressible data faster, but in cases with no matches but
// skewed character distribution compression is lost.
// Default value depends on the compression level selected.
// Can be changed with ResetWithOptions.
func WithAllLitEntropyCompression(b bool) EOption {
return func(o *encoderOptions) error {
o.customALEntropy = true
@@ -270,6 +292,7 @@ func WithAllLitEntropyCompression(b bool) EOption {
// WithNoEntropyCompression will always skip entropy compression of literals.
// This can be useful if content has matches, but unlikely to benefit from entropy
// compression. Usually the slight speed improvement is not worth enabling this.
// Can be changed with ResetWithOptions.
func WithNoEntropyCompression(b bool) EOption {
return func(o *encoderOptions) error {
o.noEntropy = b
@@ -287,6 +310,7 @@ func WithNoEntropyCompression(b bool) EOption {
// This is only a recommendation, each decoder is free to support higher or lower limits, depending on local limitations.
// If this is not specified, block encodes will automatically choose this based on the input size and the window size.
// This setting has no effect on streamed encodes.
// Can be changed with ResetWithOptions.
func WithSingleSegment(b bool) EOption {
return func(o *encoderOptions) error {
o.single = &b
@@ -298,8 +322,12 @@ func WithSingleSegment(b bool) EOption {
// slower encoding speed.
// This will not change the window size which is the primary function for reducing
// memory usage. See WithWindowSize.
// Cannot be changed with ResetWithOptions.
func WithLowerEncoderMem(b bool) EOption {
return func(o *encoderOptions) error {
if o.resetOpt && b != o.lowMem {
return errors.New("WithLowerEncoderMem cannot be changed on Reset")
}
o.lowMem = b
return nil
}
@@ -311,6 +339,7 @@ func WithLowerEncoderMem(b bool) EOption {
// "zstd --train" from the Zstandard reference implementation.
//
// The encoder *may* choose to use no dictionary instead for certain payloads.
// Can be changed with ResetWithOptions.
//
// [dictionary format]: https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#dictionary-format
func WithEncoderDict(dict []byte) EOption {
@@ -328,6 +357,7 @@ func WithEncoderDict(dict []byte) EOption {
//
// The slice content may contain arbitrary data. It will be used as an initial
// history.
// Can be changed with ResetWithOptions.
func WithEncoderDictRaw(id uint32, content []byte) EOption {
return func(o *encoderOptions) error {
if bits.UintSize > 32 && uint(len(content)) > dictMaxLength {
@@ -337,3 +367,12 @@ func WithEncoderDictRaw(id uint32, content []byte) EOption {
return nil
}
}
// WithEncoderDictDelete clears the dictionary, so no dictionary will be used.
// Should be used with ResetWithOptions.
func WithEncoderDictDelete() EOption {
return func(o *encoderOptions) error {
o.dict = nil
return nil
}
}
@@ -1,5 +1,4 @@
//go:build amd64 && !appengine && !noasm && gc
// +build amd64,!appengine,!noasm,gc
package zstd
@@ -1,5 +1,4 @@
//go:build !amd64 || appengine || !gc || noasm
// +build !amd64 appengine !gc noasm
package zstd
@@ -1,5 +1,4 @@
//go:build (!amd64 && !arm64) || appengine || !gc || purego || noasm
// +build !amd64,!arm64 appengine !gc purego noasm
package xxhash
-1
View File
@@ -1,5 +1,4 @@
//go:build amd64 && !appengine && !noasm && gc
// +build amd64,!appengine,!noasm,gc
// Copyright 2019+ Klaus Post. All rights reserved.
// License information can be found in the LICENSE file.
@@ -1,5 +1,4 @@
//go:build !amd64 || appengine || !gc || noasm
// +build !amd64 appengine !gc noasm
// Copyright 2019+ Klaus Post. All rights reserved.
// License information can be found in the LICENSE file.
-1
View File
@@ -1,5 +1,4 @@
//go:build amd64 && !appengine && !noasm && gc
// +build amd64,!appengine,!noasm,gc
package zstd
-1
View File
@@ -1,5 +1,4 @@
//go:build !amd64 || appengine || !gc || noasm
// +build !amd64 appengine !gc noasm
package zstd
+1
View File
@@ -0,0 +1 @@
test
+14
View File
@@ -0,0 +1,14 @@
version: 2
builds:
- skip: true
changelog:
sort: asc
groups:
- title: Fix
regexp: '^fix:'
- title: Features
regexp: '^feat:'
- title: Other
order: 999
+12 -1
View File
@@ -2,6 +2,16 @@
## [Unreleased]
<a name="v0.8.0"></a>
## [v0.8.0] - 2026-03-11
### Fix
- Fix Percentile to use standard NIST linear interpolation method ([#92](https://github.com/montanaflynn/stats/issues/92))
- Fix Percentile underflow bug ([#88](https://github.com/montanaflynn/stats/issues/88))
### Update
- Update Codecov upload to v4 and pass token
<a name="v0.7.1"></a>
## [v0.7.1] - 2023-05-11
### Add
@@ -517,7 +527,8 @@
- Merge pull request [#4](https://github.com/montanaflynn/stats/issues/4) from saromanov/sample
[Unreleased]: https://github.com/montanaflynn/stats/compare/v0.7.1...HEAD
[Unreleased]: https://github.com/montanaflynn/stats/compare/v0.8.0...HEAD
[v0.8.0]: https://github.com/montanaflynn/stats/compare/v0.7.1...v0.8.0
[v0.7.1]: https://github.com/montanaflynn/stats/compare/v0.7.0...v0.7.1
[v0.7.0]: https://github.com/montanaflynn/stats/compare/v0.6.6...v0.7.0
[v0.6.6]: https://github.com/montanaflynn/stats/compare/v0.6.5...v0.6.6
+82 -6
View File
@@ -29,7 +29,7 @@ Example Usage:
roundedMedian, _ := stats.Round(median, 0)
fmt.Println(roundedMedian) // 4
MIT License Copyright (c) 2014-2020 Montana Flynn (<a href="https://montanaflynn.com">https://montanaflynn.com</a>)
MIT License Copyright (c) 2014-2026 Montana Flynn (<a href="https://montanaflynn.com">https://montanaflynn.com</a>)
@@ -104,6 +104,10 @@ MIT License Copyright (c) 2014-2020 Montana Flynn (<a href="https://montanaflynn
* [func ExpReg(s []Coordinate) (regressions []Coordinate, err error)](#ExpReg)
* [func LinReg(s []Coordinate) (regressions []Coordinate, err error)](#LinReg)
* [func LogReg(s []Coordinate) (regressions []Coordinate, err error)](#LogReg)
* [type Description](#Description)
* [func Describe(input Float64Data, allowNaN bool, percentiles *[]float64) (*Description, error)](#Describe)
* [func DescribePercentileFunc(input Float64Data, allowNaN bool, percentiles *[]float64, percentileFunc func(Float64Data, float64) (float64, error)) (*Description, error)](#DescribePercentileFunc)
* [func (d *Description) String(decimals int) string](#Description.String)
* [type Float64Data](#Float64Data)
* [func LoadRawData(raw interface{}) (f Float64Data)](#LoadRawData)
* [func (f Float64Data) AutoCorrelation(lags int) (float64, error)](#Float64Data.AutoCorrelation)
@@ -173,7 +177,7 @@ MIT License Copyright (c) 2014-2020 Montana Flynn (<a href="https://montanaflynn
* [VarGeom](#example_VarGeom)
#### <a name="pkg-files">Package files</a>
[correlation.go](/src/github.com/montanaflynn/stats/correlation.go) [cumulative_sum.go](/src/github.com/montanaflynn/stats/cumulative_sum.go) [data.go](/src/github.com/montanaflynn/stats/data.go) [deviation.go](/src/github.com/montanaflynn/stats/deviation.go) [distances.go](/src/github.com/montanaflynn/stats/distances.go) [doc.go](/src/github.com/montanaflynn/stats/doc.go) [entropy.go](/src/github.com/montanaflynn/stats/entropy.go) [errors.go](/src/github.com/montanaflynn/stats/errors.go) [geometric_distribution.go](/src/github.com/montanaflynn/stats/geometric_distribution.go) [legacy.go](/src/github.com/montanaflynn/stats/legacy.go) [load.go](/src/github.com/montanaflynn/stats/load.go) [max.go](/src/github.com/montanaflynn/stats/max.go) [mean.go](/src/github.com/montanaflynn/stats/mean.go) [median.go](/src/github.com/montanaflynn/stats/median.go) [min.go](/src/github.com/montanaflynn/stats/min.go) [mode.go](/src/github.com/montanaflynn/stats/mode.go) [norm.go](/src/github.com/montanaflynn/stats/norm.go) [outlier.go](/src/github.com/montanaflynn/stats/outlier.go) [percentile.go](/src/github.com/montanaflynn/stats/percentile.go) [quartile.go](/src/github.com/montanaflynn/stats/quartile.go) [ranksum.go](/src/github.com/montanaflynn/stats/ranksum.go) [regression.go](/src/github.com/montanaflynn/stats/regression.go) [round.go](/src/github.com/montanaflynn/stats/round.go) [sample.go](/src/github.com/montanaflynn/stats/sample.go) [sigmoid.go](/src/github.com/montanaflynn/stats/sigmoid.go) [softmax.go](/src/github.com/montanaflynn/stats/softmax.go) [sum.go](/src/github.com/montanaflynn/stats/sum.go) [util.go](/src/github.com/montanaflynn/stats/util.go) [variance.go](/src/github.com/montanaflynn/stats/variance.go)
[correlation.go](/src/github.com/montanaflynn/stats/correlation.go) [cumulative_sum.go](/src/github.com/montanaflynn/stats/cumulative_sum.go) [data.go](/src/github.com/montanaflynn/stats/data.go) [describe.go](/src/github.com/montanaflynn/stats/describe.go) [deviation.go](/src/github.com/montanaflynn/stats/deviation.go) [distances.go](/src/github.com/montanaflynn/stats/distances.go) [doc.go](/src/github.com/montanaflynn/stats/doc.go) [entropy.go](/src/github.com/montanaflynn/stats/entropy.go) [errors.go](/src/github.com/montanaflynn/stats/errors.go) [geometric_distribution.go](/src/github.com/montanaflynn/stats/geometric_distribution.go) [legacy.go](/src/github.com/montanaflynn/stats/legacy.go) [load.go](/src/github.com/montanaflynn/stats/load.go) [max.go](/src/github.com/montanaflynn/stats/max.go) [mean.go](/src/github.com/montanaflynn/stats/mean.go) [median.go](/src/github.com/montanaflynn/stats/median.go) [min.go](/src/github.com/montanaflynn/stats/min.go) [mode.go](/src/github.com/montanaflynn/stats/mode.go) [norm.go](/src/github.com/montanaflynn/stats/norm.go) [outlier.go](/src/github.com/montanaflynn/stats/outlier.go) [percentile.go](/src/github.com/montanaflynn/stats/percentile.go) [quartile.go](/src/github.com/montanaflynn/stats/quartile.go) [ranksum.go](/src/github.com/montanaflynn/stats/ranksum.go) [regression.go](/src/github.com/montanaflynn/stats/regression.go) [round.go](/src/github.com/montanaflynn/stats/round.go) [sample.go](/src/github.com/montanaflynn/stats/sample.go) [sigmoid.go](/src/github.com/montanaflynn/stats/sigmoid.go) [softmax.go](/src/github.com/montanaflynn/stats/softmax.go) [sum.go](/src/github.com/montanaflynn/stats/sum.go) [util.go](/src/github.com/montanaflynn/stats/util.go) [variance.go](/src/github.com/montanaflynn/stats/variance.go)
@@ -380,7 +384,7 @@ Min finds the lowest number in a set of data
## <a name="MinkowskiDistance">func</a> [MinkowskiDistance](/distances.go?s=2152:2256#L75)
## <a name="MinkowskiDistance">func</a> [MinkowskiDistance](/distances.go?s=2133:2237#L78)
``` go
func MinkowskiDistance(dataPointX, dataPointY Float64Data, lambda float64) (distance float64, err error)
```
@@ -593,15 +597,28 @@ Pearson calculates the Pearson product-moment correlation coefficient between tw
## <a name="Percentile">func</a> [Percentile](/percentile.go?s=98:181#L8)
## <a name="Percentile">func</a> [Percentile](/percentile.go?s=598:681#L20)
``` go
func Percentile(input Float64Data, percent float64) (percentile float64, err error)
```
Percentile finds the relative standing in a slice of floats
Percentile finds the relative standing in a slice of floats.
The function uses the Linear Interpolation Between Closest Ranks method
as recommended by NIST [1] and used by Excel (PERCENTILE), Google Sheets,
NumPy (default), and other standard tools.
Algorithm (for percent p and sorted data of length n):
1. Compute the rank: rank = (p / 100) * (n - 1)
2. Split into integer part k and fractional part f
3. Result = data[k] + f * (data[k+1] - data[k])
[1] <a href="https://www.itl.nist.gov/div898/handbook/prc/section2/prc262.htm">https://www.itl.nist.gov/div898/handbook/prc/section2/prc262.htm</a>
## <a name="PercentileNearestRank">func</a> [PercentileNearestRank](/percentile.go?s=1079:1173#L54)
## <a name="PercentileNearestRank">func</a> [PercentileNearestRank](/percentile.go?s=1382:1476#L55)
``` go
func PercentileNearestRank(input Float64Data, percent float64) (percentile float64, err error)
```
@@ -809,6 +826,65 @@ LogReg is a shortcut to LogarithmicRegression
## <a name="Description">type</a> [Description](/describe.go?s=89:349#L6)
``` go
type Description struct {
Count int
Mean float64
Std float64
Max float64
Min float64
DescriptionPercentiles []descriptionPercentile
AllowedNaN bool
}
```
Holds information about the dataset provided to Describe
### <a name="Describe">func</a> [Describe](/describe.go?s=579:672#L23)
``` go
func Describe(input Float64Data, allowNaN bool, percentiles *[]float64) (*Description, error)
```
Describe generates descriptive statistics about a provided dataset, similar to python's pandas.describe()
### <a name="DescribePercentileFunc">func</a> [DescribePercentileFunc](/describe.go?s=917:1084#L29)
``` go
func DescribePercentileFunc(input Float64Data, allowNaN bool, percentiles *[]float64, percentileFunc func(Float64Data, float64) (float64, error)) (*Description, error)
```
Describe generates descriptive statistics about a provided dataset, similar to python's pandas.describe()
Takes in a function to use for percentile calculation
### <a name="Description.String">func</a> (\*Description) [String](/describe.go?s=2078:2127#L68)
``` go
func (d *Description) String(decimals int) string
```
Represents the Description instance in a string format with specified number of decimals
count 3
mean 2.00
std 0.82
max 3.00
min 1.00
25.00% NaN
50.00% 1.50
75.00% 2.50
NaN OK true
## <a name="Float64Data">type</a> [Float64Data](/data.go?s=80:106#L4)
``` go
type Float64Data []float64
+1 -1
View File
@@ -1,6 +1,6 @@
The MIT License (MIT)
Copyright (c) 2014-2023 Montana Flynn (https://montanaflynn.com)
Copyright (c) 2014-2026 Montana Flynn (https://montanaflynn.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+7 -23
View File
@@ -115,9 +115,12 @@ func NormVar(loc float64, scale float64) float64 {}
func Pearson(data1, data2 Float64Data) (float64, error) {}
func Percentile(input Float64Data, percent float64) (percentile float64, err error) {}
func PercentileNearestRank(input Float64Data, percent float64) (percentile float64, err error) {}
func PopulationSkewness(input Float64Data) (float64, error) {}
func PopulationVariance(input Float64Data) (pvar float64, err error) {}
func Sample(input Float64Data, takenum int, replacement bool) ([]float64, error) {}
func SampleSkewness(input Float64Data) (float64, error) {}
func SampleVariance(input Float64Data) (svar float64, err error) {}
func Skewness(input Float64Data) (float64, error) {}
func Sigmoid(input Float64Data) ([]float64, error) {}
func SoftMax(input Float64Data) ([]float64, error) {}
func StableSample(input Float64Data, takenum int) ([]float64, error) {}
@@ -182,35 +185,16 @@ To make things as seamless as possible please also consider the following steps:
## Releasing
This is not required by contributors and mostly here as a reminder to myself as the maintainer of this repo. To release a new version we should update the [CHANGELOG.md](/CHANGELOG.md) and [DOCUMENTATION.md](/DOCUMENTATION.md).
First install the tools used to generate the markdown files and release:
Releases are automated with [GoReleaser](https://goreleaser.com/) via GitHub Actions. To create a new release, push a version tag:
```
go install github.com/davecheney/godoc2md@latest
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
brew tap git-chglog/git-chglog
brew install gnu-sed hub git-chglog
git tag v0.x.x
git push origin v0.x.x
```
Then you can run these `make` directives:
```
# Generate DOCUMENTATION.md
make docs
```
Then we can create a [CHANGELOG.md](/CHANGELOG.md) a new git tag and a github release:
```
make release TAG=v0.x.x
```
To authenticate `hub` for the release you will need to create a personal access token and use it as the password when it's requested.
## MIT License
Copyright (c) 2014-2023 Montana Flynn (https://montanaflynn.com)
Copyright (c) 2014-2026 Montana Flynn (https://montanaflynn.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+1 -1
View File
@@ -18,6 +18,6 @@ Example Usage:
roundedMedian, _ := stats.Round(median, 0)
fmt.Println(roundedMedian) // 4
MIT License Copyright (c) 2014-2020 Montana Flynn (https://montanaflynn.com)
MIT License Copyright (c) 2014-2026 Montana Flynn (https://montanaflynn.com)
*/
package stats
+22 -21
View File
@@ -4,7 +4,19 @@ import (
"math"
)
// Percentile finds the relative standing in a slice of floats
// Percentile finds the relative standing in a slice of floats.
//
// The function uses the Linear Interpolation Between Closest Ranks method
// as recommended by NIST [1] and used by Excel (PERCENTILE), Google Sheets,
// NumPy (default), and other standard tools.
//
// Algorithm (for percent p and sorted data of length n):
//
// 1. Compute the rank: rank = (p / 100) * (n - 1)
// 2. Split into integer part k and fractional part f
// 3. Result = data[k] + f * (data[k+1] - data[k])
//
// [1] https://www.itl.nist.gov/div898/handbook/prc/section2/prc262.htm
func Percentile(input Float64Data, percent float64) (percentile float64, err error) {
length := input.Len()
if length == 0 {
@@ -22,28 +34,17 @@ func Percentile(input Float64Data, percent float64) (percentile float64, err err
// Start by sorting a copy of the slice
c := sortedCopy(input)
// Multiply percent by length of input
index := (percent / 100) * float64(len(c))
// Check if the index is a whole number
if index == float64(int64(index)) {
// Convert float to int
i := int(index)
// Find the value at the index
percentile = c[i-1]
} else if index > 1 {
// Convert float to int via truncation
i := int(index)
// Find the average of the index and following values
percentile, _ = Mean(Float64Data{c[i-1], c[i]})
// Use the standard linear interpolation method:
// rank = (percent / 100) * (n - 1)
// result = c[k] + f * (c[k+1] - c[k])
rank := (percent / 100) * float64(length-1)
k := int(rank)
f := rank - float64(k)
if k+1 < length {
percentile = c[k] + f*(c[k+1]-c[k])
} else {
return math.NaN(), BoundsErr
percentile = c[k]
}
return percentile, nil
+1 -2
View File
@@ -18,7 +18,7 @@ func LinearRegression(s Series) (regressions Series, err error) {
}
// Placeholder for the math to be done
var sum [5]float64
var sum [4]float64
// Loop over data keeping index in place
i := 0
@@ -27,7 +27,6 @@ func LinearRegression(s Series) (regressions Series, err error) {
sum[1] += s[i].Y
sum[2] += s[i].X * s[i].X
sum[3] += s[i].X * s[i].Y
sum[4] += s[i].Y * s[i].Y
}
// Find gradient and intercept
+62
View File
@@ -0,0 +1,62 @@
package stats
import "math"
// Skewness computes the population skewness of the dataset
func Skewness(input Float64Data) (float64, error) {
return PopulationSkewness(input)
}
// PopulationSkewness computes the population skewness using the third
// central moment normalized by the cube of the standard deviation.
func PopulationSkewness(input Float64Data) (float64, error) {
if input.Len() < 2 {
return math.NaN(), ErrEmptyInput
}
mean, _ := Mean(input)
// Compute sum of squared and cubed differences from the mean
var sumOfSquares, sumOfCubes float64
for _, v := range input {
d := v - mean
sumOfSquares += d * d
sumOfCubes += d * d * d
}
if sumOfSquares == 0 {
return math.NaN(), ErrEmptyInput
}
if sumOfCubes == 0 {
return 0.0, nil
}
n := float64(input.Len())
variance := sumOfSquares / n
stdDevCubed := math.Pow(variance, 3.0/2.0)
return (sumOfCubes / n) / stdDevCubed, nil
}
// SampleSkewness computes the adjusted Fisher-Pearson standardized moment
// coefficient, correcting for bias in small samples.
func SampleSkewness(input Float64Data) (float64, error) {
n := input.Len()
if n < 3 {
return math.NaN(), ErrEmptyInput
}
g1, err := PopulationSkewness(input)
if err != nil {
return math.NaN(), err
}
if g1 == 0 {
return 0.0, nil
}
// Adjusted Fisher-Pearson: G1 = g1 * sqrt(n*(n-1)) / (n-2)
nf := float64(n)
return g1 * math.Sqrt(nf*(nf-1)) / (nf - 2), nil
}
+5 -1
View File
@@ -10,6 +10,10 @@ coverage.txt
.vscode
tmp/*
*.test
extra/redisotel-native/metrics-collector-app/
# maintenanceNotifications upgrade documentation (temporary)
maintenanceNotifications/docs/
# Docker-generated files (TLS certificates, cluster data, etc.)
dockers/*/tls/
dockers/osscluster-tls/
+2
View File
@@ -26,6 +26,8 @@ linters:
- builtin$
- examples$
formatters:
enable:
- gofmt
exclusions:
generated: lax
paths:
+38 -3
View File
@@ -1,8 +1,8 @@
GO_MOD_DIRS := $(shell find . -type f -name 'go.mod' -exec dirname {} \; | sort)
REDIS_VERSION ?= 8.4
REDIS_VERSION ?= 8.6
RE_CLUSTER ?= false
RCE_DOCKER ?= true
CLIENT_LIBS_TEST_IMAGE ?= redislabs/client-libs-test:8.4.0
CLIENT_LIBS_TEST_IMAGE ?= redislabs/client-libs-test:custom-21860421418-debian-amd64
docker.start:
export RE_CLUSTER=$(RE_CLUSTER) && \
@@ -14,6 +14,17 @@ docker.start:
docker.stop:
docker compose --profile all down
docker.e2e.start:
@echo "Starting Redis and cae-resp-proxy for E2E tests..."
docker compose --profile e2e up -d --quiet-pull
@echo "Waiting for services to be ready..."
@sleep 3
@echo "Services ready!"
docker.e2e.stop:
@echo "Stopping E2E services..."
docker compose --profile e2e down
test:
$(MAKE) docker.start
@if [ -z "$(REDIS_VERSION)" ]; then \
@@ -66,7 +77,31 @@ bench:
export REDIS_VERSION=$(REDIS_VERSION) && \
go test ./... -test.run=NONE -test.bench=. -test.benchmem -skip Example
.PHONY: all test test.ci test.ci.skip-vectorsets bench fmt
test.e2e:
@echo "Running E2E tests with auto-start proxy..."
$(MAKE) docker.e2e.start
@echo "Running tests..."
@E2E_SCENARIO_TESTS=true go test -v ./maintnotifications/e2e/ -timeout 30m || ($(MAKE) docker.e2e.stop && exit 1)
$(MAKE) docker.e2e.stop
@echo "E2E tests completed!"
test.e2e.docker:
@echo "Running Docker-compatible E2E tests..."
$(MAKE) docker.e2e.start
@echo "Running unified injector tests..."
@E2E_SCENARIO_TESTS=true go test -v -run "TestUnifiedInjector|TestCreateTestFaultInjectorLogic|TestFaultInjectorClientCreation" ./maintnotifications/e2e/ -timeout 10m || ($(MAKE) docker.e2e.stop && exit 1)
$(MAKE) docker.e2e.stop
@echo "Docker E2E tests completed!"
test.e2e.logic:
@echo "Running E2E logic tests (no proxy required)..."
@E2E_SCENARIO_TESTS=true \
REDIS_ENDPOINTS_CONFIG_PATH=/tmp/test_endpoints_verify.json \
FAULT_INJECTION_API_URL=http://localhost:8080 \
go test -v -run "TestCreateTestFaultInjectorLogic|TestFaultInjectorClientCreation" ./maintnotifications/e2e/
@echo "Logic tests completed!"
.PHONY: all test test.ci test.ci.skip-vectorsets bench fmt test.e2e test.e2e.logic docker.e2e.start docker.e2e.stop
build:
export RE_CLUSTER=$(RE_CLUSTER) && \
+5 -5
View File
@@ -21,13 +21,12 @@ In `go-redis` we are aiming to support the last three releases of Redis. Current
- [Redis 8.2](https://raw.githubusercontent.com/redis/redis/8.2/00-RELEASENOTES) - using Redis CE 8.2
- [Redis 8.4](https://raw.githubusercontent.com/redis/redis/8.4/00-RELEASENOTES) - using Redis CE 8.4
Although the `go.mod` states it requires at minimum `go 1.18`, our CI is configured to run the tests against all three
versions of Redis and latest two versions of Go ([1.23](https://go.dev/doc/devel/release#go1.23.0),
[1.24](https://go.dev/doc/devel/release#go1.24.0)). We observe that some modules related test may not pass with
Although the `go.mod` states it requires at minimum `go 1.21`, our CI is configured to run the tests against all three
versions of Redis and multiple versions of Go ([1.21](https://go.dev/doc/devel/release#go1.21.0),
[1.23](https://go.dev/doc/devel/release#go1.23.0), oldstable, and stable). We observe that some modules related test may not pass with
Redis Stack 7.2 and some commands are changed with Redis CE 8.0.
Although it is not officially supported, `go-redis/v9` should be able to work with any Redis 7.0+.
Please do refer to the documentation and the tests if you experience any issues. We do plan to update the go version
in the `go.mod` to `go 1.24` in one of the next releases.
Please do refer to the documentation and the tests if you experience any issues.
## How do I Redis?
@@ -111,6 +110,7 @@ func ExampleClient() {
Password: "", // no password set
DB: 0, // use default DB
})
defer rdb.Close()
err := rdb.Set(ctx, "key", "value", 0).Err()
if err != nil {
+160 -6
View File
@@ -1,24 +1,178 @@
# Release Notes
# 9.17.2 (2025-12-01)
# 9.18.0 (2026-02-16)
## 🚀 Highlights
### Redis 8.6 Support
Added support for Redis 8.6, including new commands and features for streams idempotent production and HOTKEYS.
### Smart Client Handoff (Maintenance Notifications) for Cluster
This release introduces comprehensive support for Redis Cluster maintenance notifications via SMIGRATING/SMIGRATED push notifications. The client now automatically handles slot migrations by:
- **Relaxing timeouts during migration** (SMIGRATING) to prevent false failures
- **Triggering lazy cluster state reloads** upon completion (SMIGRATED)
- Enabling seamless operations during Redis Enterprise maintenance windows
([#3643](https://github.com/redis/go-redis/pull/3643)) by [@ndyakov](https://github.com/ndyakov)
### OpenTelemetry Native Metrics Support
Added comprehensive OpenTelemetry metrics support following the [OpenTelemetry Database Client Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/database/database-metrics/). The implementation uses a Bridge Pattern to keep the core library dependency-free while providing optional metrics instrumentation through the new `extra/redisotel-native` package.
**Metric groups include:**
- Command metrics: Operation duration with retry tracking
- Connection basic: Connection count and creation time
- Resiliency: Errors, handoffs, timeout relaxation
- Connection advanced: Wait time and use time
- Pubsub metrics: Published and received messages
- Stream metrics: Processing duration and maintenance notifications
([#3637](https://github.com/redis/go-redis/pull/3637)) by [@ofekshenawa](https://github.com/ofekshenawa)
## ✨ New Features
- **HOTKEYS Commands**: Added support for Redis HOTKEYS feature for identifying hot keys based on CPU consumption and network utilization ([#3695](https://github.com/redis/go-redis/pull/3695)) by [@ofekshenawa](https://github.com/ofekshenawa)
- **Streams Idempotent Production**: Added support for Redis 8.6+ Streams Idempotent Production with `ProducerID`, `IdempotentID`, `IdempotentAuto` in `XAddArgs` and new `XCFGSET` command ([#3693](https://github.com/redis/go-redis/pull/3693)) by [@ofekshenawa](https://github.com/ofekshenawa)
- **NaN Values for TimeSeries**: Added support for NaN (Not a Number) values in Redis time series commands ([#3687](https://github.com/redis/go-redis/pull/3687)) by [@ofekshenawa](https://github.com/ofekshenawa)
- **DialerRetries Options**: Added `DialerRetries` and `DialerRetryTimeout` to `ClusterOptions`, `RingOptions`, and `FailoverOptions` ([#3686](https://github.com/redis/go-redis/pull/3686)) by [@naveenchander30](https://github.com/naveenchander30)
- **ConnMaxLifetimeJitter**: Added jitter configuration to distribute connection expiration times and prevent thundering herd ([#3666](https://github.com/redis/go-redis/pull/3666)) by [@cyningsun](https://github.com/cyningsun)
- **Digest Helper Functions**: Added `DigestString` and `DigestBytes` helper functions for client-side xxh3 hashing compatible with Redis DIGEST command ([#3679](https://github.com/redis/go-redis/pull/3679)) by [@ofekshenawa](https://github.com/ofekshenawa)
- **SMIGRATED New Format**: Updated SMIGRATED parser to support new format and remember original host:port ([#3697](https://github.com/redis/go-redis/pull/3697)) by [@ndyakov](https://github.com/ndyakov)
- **Cluster State Reload Interval**: Added cluster state reload interval option for maintenance notifications ([#3663](https://github.com/redis/go-redis/pull/3663)) by [@ndyakov](https://github.com/ndyakov)
## 🐛 Bug Fixes
- **Connection Pool**: Fixed critical race condition in turn management that could cause connection leaks when dial goroutines complete after request timeout ([#3626](https://github.com/redis/go-redis/pull/3626)) by [@cyningsun](https://github.com/cyningsun)
- **Context Timeout**: Improved context timeout calculation to use minimum of remaining time and DialTimeout, preventing goroutines from waiting longer than necessary ([#3626](https://github.com/redis/go-redis/pull/3626)) by [@cyningsun](https://github.com/cyningsun)
- **PubSub nil pointer dereference**: Fixed nil pointer dereference in PubSub after `WithTimeout()` - `pubSubPool` is now properly cloned ([#3710](https://github.com/redis/go-redis/pull/3710)) by [@Copilot](https://github.com/apps/copilot-swe-agent)
- **MaintNotificationsConfig nil check**: Guard against nil `MaintNotificationsConfig` in `initConn` ([#3707](https://github.com/redis/go-redis/pull/3707)) by [@veeceey](https://github.com/veeceey)
- **wantConnQueue zombie elements**: Fixed zombie `wantConn` elements accumulation in `wantConnQueue` ([#3680](https://github.com/redis/go-redis/pull/3680)) by [@cyningsun](https://github.com/cyningsun)
- **XADD/XTRIM approx flag**: Fixed XADD and XTRIM to use `=` when approx is false ([#3684](https://github.com/redis/go-redis/pull/3684)) by [@ndyakov](https://github.com/ndyakov)
- **Sentinel timeout retry**: When connection to a sentinel times out, attempt to connect to other sentinels ([#3654](https://github.com/redis/go-redis/pull/3654)) by [@cxljs](https://github.com/cxljs)
## ⚡ Performance
- **Fuzz test optimization**: Eliminated repeated string conversions, used functional approach for cleaner operation selection ([#3692](https://github.com/redis/go-redis/pull/3692)) by [@feiguoL](https://github.com/feiguoL)
- **Pre-allocate capacity**: Pre-allocate slice capacity to prevent multiple capacity expansions ([#3689](https://github.com/redis/go-redis/pull/3689)) by [@feelshu](https://github.com/feelshu)
## 🧪 Testing
- **Comprehensive TLS tests**: Added comprehensive TLS tests and example for standalone, cluster, and certificate authentication ([#3681](https://github.com/redis/go-redis/pull/3681)) by [@ndyakov](https://github.com/ndyakov)
- **Redis 8.6**: Updated CI to use Redis 8.6-pre ([#3685](https://github.com/redis/go-redis/pull/3685)) by [@ndyakov](https://github.com/ndyakov)
## 🧰 Maintenance
- **Deprecation warnings**: Added deprecation warnings for commands based on Redis documentation ([#3673](https://github.com/redis/go-redis/pull/3673)) by [@ndyakov](https://github.com/ndyakov)
- **Use errors.Join()**: Replaced custom error join function with standard library `errors.Join()` ([#3653](https://github.com/redis/go-redis/pull/3653)) by [@cxljs](https://github.com/cxljs)
- **Use Go 1.21 min/max**: Use Go 1.21's built-in min/max functions ([#3656](https://github.com/redis/go-redis/pull/3656)) by [@cxljs](https://github.com/cxljs)
- **Proper formatting**: Code formatting improvements ([#3670](https://github.com/redis/go-redis/pull/3670)) by [@12ya](https://github.com/12ya)
- **Set commands documentation**: Added comprehensive documentation to all set command methods ([#3642](https://github.com/redis/go-redis/pull/3642)) by [@iamamirsalehi](https://github.com/iamamirsalehi)
- **MaxActiveConns docs**: Added default value documentation for `MaxActiveConns` ([#3674](https://github.com/redis/go-redis/pull/3674)) by [@codykaup](https://github.com/codykaup)
- **README example update**: Updated README example ([#3657](https://github.com/redis/go-redis/pull/3657)) by [@cxljs](https://github.com/cxljs)
- **Cluster maintnotif example**: Added example application for cluster maintenance notifications ([#3651](https://github.com/redis/go-redis/pull/3651)) by [@ndyakov](https://github.com/ndyakov)
## 👥 Contributors
We'd like to thank all the contributors who worked on this release!
[@12ya](https://github.com/12ya), [@Copilot](https://github.com/apps/copilot-swe-agent), [@codykaup](https://github.com/codykaup), [@cxljs](https://github.com/cxljs), [@cyningsun](https://github.com/cyningsun), [@feelshu](https://github.com/feelshu), [@feiguoL](https://github.com/feiguoL), [@iamamirsalehi](https://github.com/iamamirsalehi), [@naveenchander30](https://github.com/naveenchander30), [@ndyakov](https://github.com/ndyakov), [@ofekshenawa](https://github.com/ofekshenawa), [@veeceey](https://github.com/veeceey)
---
**Full Changelog**: https://github.com/redis/go-redis/compare/v9.17.0...v9.18.0
# 9.18.0-beta.2 (2025-12-09)
## 🚀 Highlights
### Go Version Update
This release updates the minimum required Go version to 1.21. This is part of a gradual migration strategy where the minimum supported Go version will be three versions behind the latest release. With each new Go version release, we will bump the minimum version by one, ensuring compatibility while staying current with the Go ecosystem.
### Stability Improvements
This release includes several important stability fixes:
- Fixed a critical panic in the handoff worker manager that could occur when handling nil errors
- Improved test reliability for Smart Client Handoff functionality
- Fixed logging format issues that could cause runtime errors
## ✨ New Features
- OpenTelemetry metrics improvements for nil response handling ([#3638](https://github.com/redis/go-redis/pull/3638)) by [@fengve](https://github.com/fengve)
## 🐛 Bug Fixes
- Fixed panic on nil error in handoffWorkerManager closeConnFromRequest ([#3633](https://github.com/redis/go-redis/pull/3633)) by [@ccoVeille](https://github.com/ccoVeille)
- Fixed bad sprintf syntax in logging ([#3632](https://github.com/redis/go-redis/pull/3632)) by [@ccoVeille](https://github.com/ccoVeille)
## 🧰 Maintenance
- Updated minimum Go version to 1.21 ([#3640](https://github.com/redis/go-redis/pull/3640)) by [@ndyakov](https://github.com/ndyakov)
- Use Go 1.20 idiomatic string<->byte conversion ([#3435](https://github.com/redis/go-redis/pull/3435)) by [@justinhwang](https://github.com/justinhwang)
- Reduce flakiness of Smart Client Handoff test ([#3641](https://github.com/redis/go-redis/pull/3641)) by [@kiryazovi-redis](https://github.com/kiryazovi-redis)
- Revert PR #3634 (Observability metrics phase1) ([#3635](https://github.com/redis/go-redis/pull/3635)) by [@ofekshenawa](https://github.com/ofekshenawa)
## 👥 Contributors
We'd like to thank all the contributors who worked on this release!
[@justinhwang](https://github.com/justinhwang), [@ndyakov](https://github.com/ndyakov), [@kiryazovi-redis](https://github.com/kiryazovi-redis), [@fengve](https://github.com/fengve), [@ccoVeille](https://github.com/ccoVeille), [@ofekshenawa](https://github.com/ofekshenawa)
---
**Full Changelog**: https://github.com/redis/go-redis/compare/v9.18.0-beta.1...v9.18.0-beta.2
# 9.18.0-beta.1 (2025-12-01)
## 🚀 Highlights
### Request and Response Policy Based Routing in Cluster Mode
This beta release introduces comprehensive support for Redis COMMAND-based request and response policy routing for cluster clients. This feature enables intelligent command routing and response aggregation based on Redis command metadata.
**Key Features:**
- **Command Policy Loader**: Automatically parses and caches COMMAND metadata with routing/aggregation hints
- **Enhanced Routing Engine**: Supports all request policies including:
- `default(keyless)` - Commands without keys
- `default(hashslot)` - Commands with hash slot routing
- `all_shards` - Commands that need to run on all shards
- `all_nodes` - Commands that need to run on all nodes
- `multi_shard` - Commands that span multiple shards
- `special` - Commands with custom routing logic
- **Response Aggregator**: Intelligently combines multi-shard replies based on response policies:
- `all_succeeded` - All shards must succeed
- `one_succeeded` - At least one shard must succeed
- `agg_sum` - Aggregate numeric responses
- `special` - Custom aggregation logic (e.g., FT.CURSOR)
- **Raw Command Support**: Policies are enforced on `Client.Do(ctx, args...)`
This feature is particularly useful for Redis Stack commands like RediSearch that need to operate across multiple shards in a cluster.
### Connection Pool Improvements
Fixed a critical defect in the connection pool's turn management mechanism that could lead to connection leaks under certain conditions. The fix ensures proper 1:1 correspondence between turns and connections.
## ✨ New Features
- Request and Response Policy Based Routing in Cluster Mode ([#3422](https://github.com/redis/go-redis/pull/3422)) by [@ofekshenawa](https://github.com/ofekshenawa)
## 🐛 Bug Fixes
- Fixed connection pool turn management to prevent connection leaks ([#3626](https://github.com/redis/go-redis/pull/3626)) by [@cyningsun](https://github.com/cyningsun)
## 🧰 Maintenance
- chore(deps): bump rojopolis/spellcheck-github-actions from 0.54.0 to 0.55.0 ([#3627](https://github.com/redis/go-redis/pull/3627))
## Contributors
## 👥 Contributors
We'd like to thank all the contributors who worked on this release!
[@cyningsun](https://github.com/cyningsun) and [@ndyakov](https://github.com/ndyakov)
[@cyningsun](https://github.com/cyningsun), [@ofekshenawa](https://github.com/ofekshenawa), [@ndyakov](https://github.com/ndyakov)
---
**Full Changelog**: https://github.com/redis/go-redis/compare/v9.17.1...v9.17.2
**Full Changelog**: https://github.com/redis/go-redis/compare/v9.17.1...v9.18.0-beta.1
# 9.17.1 (2025-11-25)
+7
View File
@@ -61,6 +61,13 @@ func (oa *optionsAdapter) GetAddr() string {
return oa.options.Addr
}
// GetNodeAddress returns the address of the Redis node as reported by the server.
// For cluster clients, this is the endpoint from CLUSTER SLOTS before any transformation.
// For standalone clients, this defaults to Addr.
func (oa *optionsAdapter) GetNodeAddress() string {
return oa.options.NodeAddress
}
// IsTLSEnabled returns true if TLS is enabled.
func (oa *optionsAdapter) IsTLSEnabled() bool {
return oa.options.TLSConfig != nil
@@ -44,4 +44,4 @@ func NewReAuthCredentialsListener(reAuth func(credentials Credentials) error, on
}
// Ensure ReAuthCredentialsListener implements the CredentialsListener interface.
var _ CredentialsListener = (*ReAuthCredentialsListener)(nil)
var _ CredentialsListener = (*ReAuthCredentialsListener)(nil)
+6
View File
@@ -42,6 +42,9 @@ func (c cmdable) ClusterMyID(ctx context.Context) *StringCmd {
return cmd
}
// ClusterSlots returns the mapping of cluster slots to nodes.
//
// Deprecated: Use ClusterShards instead as of Redis 7.0.0.
func (c cmdable) ClusterSlots(ctx context.Context) *ClusterSlotsCmd {
cmd := NewClusterSlotsCmd(ctx, "cluster", "slots")
_ = c(ctx, cmd)
@@ -153,6 +156,9 @@ func (c cmdable) ClusterSaveConfig(ctx context.Context) *StatusCmd {
return cmd
}
// ClusterSlaves lists the replica nodes of a master node.
//
// Deprecated: Use ClusterReplicas instead as of Redis 5.0.0.
func (c cmdable) ClusterSlaves(ctx context.Context, nodeID string) *StringSliceCmd {
cmd := NewStringSliceCmd(ctx, "cluster", "slaves", nodeID)
_ = c(ctx, cmd)
+2243 -127
View File
File diff suppressed because it is too large Load Diff
+209
View File
@@ -0,0 +1,209 @@
package redis
import (
"context"
"strings"
"github.com/redis/go-redis/v9/internal/routing"
)
type (
module = string
commandName = string
)
var defaultPolicies = map[module]map[commandName]*routing.CommandPolicy{
"ft": {
"create": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"search": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"aggregate": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"dictadd": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"dictdump": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"dictdel": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"suglen": {
Request: routing.ReqDefault,
Response: routing.RespDefaultHashSlot,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"cursor": {
Request: routing.ReqSpecial,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"sugadd": {
Request: routing.ReqDefault,
Response: routing.RespDefaultHashSlot,
},
"sugget": {
Request: routing.ReqDefault,
Response: routing.RespDefaultHashSlot,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"sugdel": {
Request: routing.ReqDefault,
Response: routing.RespDefaultHashSlot,
},
"spellcheck": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"explain": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"explaincli": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"aliasadd": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"aliasupdate": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"aliasdel": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"info": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"tagvals": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"syndump": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"synupdate": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"profile": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
Tips: map[string]string{
routing.ReadOnlyCMD: "",
},
},
"alter": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"dropindex": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
"drop": {
Request: routing.ReqDefault,
Response: routing.RespDefaultKeyless,
},
},
}
type CommandInfoResolveFunc func(ctx context.Context, cmd Cmder) *routing.CommandPolicy
type commandInfoResolver struct {
resolveFunc CommandInfoResolveFunc
fallBackResolver *commandInfoResolver
}
func NewCommandInfoResolver(resolveFunc CommandInfoResolveFunc) *commandInfoResolver {
return &commandInfoResolver{
resolveFunc: resolveFunc,
}
}
func NewDefaultCommandPolicyResolver() *commandInfoResolver {
return NewCommandInfoResolver(func(ctx context.Context, cmd Cmder) *routing.CommandPolicy {
module := "core"
command := cmd.Name()
cmdParts := strings.Split(command, ".")
if len(cmdParts) == 2 {
module = cmdParts[0]
command = cmdParts[1]
}
if policy, ok := defaultPolicies[module][command]; ok {
return policy
}
return nil
})
}
func (r *commandInfoResolver) GetCommandPolicy(ctx context.Context, cmd Cmder) *routing.CommandPolicy {
if r.resolveFunc == nil {
return nil
}
policy := r.resolveFunc(ctx, cmd)
if policy != nil {
return policy
}
if r.fallBackResolver != nil {
return r.fallBackResolver.GetCommandPolicy(ctx, cmd)
}
return nil
}
func (r *commandInfoResolver) SetFallbackResolver(fallbackResolver *commandInfoResolver) {
r.fallBackResolver = fallbackResolver
}
+11
View File
@@ -55,6 +55,11 @@ func appendArgs(dst, src []interface{}) []interface{} {
return appendArg(dst, src[0])
}
if cap(dst) < len(dst)+len(src) {
newDst := make([]interface{}, len(dst), len(dst)+len(src))
copy(newDst, dst)
dst = newDst
}
dst = append(dst, src...)
return dst
}
@@ -443,6 +448,9 @@ func (c cmdable) Do(ctx context.Context, args ...interface{}) *Cmd {
return cmd
}
// Quit closes the connection.
//
// Deprecated: Just close the connection instead as of Redis 7.2.0.
func (c cmdable) Quit(_ context.Context) *StatusCmd {
panic("not implemented")
}
@@ -665,6 +673,9 @@ func (c cmdable) ShutdownNoSave(ctx context.Context) *StatusCmd {
return c.shutdown(ctx, "nosave")
}
// SlaveOf sets a Redis server as a replica of another, or promotes it to being a master.
//
// Deprecated: Use ReplicaOf instead as of Redis 5.0.0.
func (c cmdable) SlaveOf(ctx context.Context, host, port string) *StatusCmd {
cmd := NewStatusCmd(ctx, "slaveof", host, port)
_ = c(ctx, cmd)
+75 -5
View File
@@ -1,12 +1,16 @@
---
x-default-image: &default-image ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.6.0}
services:
redis:
image: ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.4.0}
image: *default-image
platform: linux/amd64
container_name: redis-standalone
environment:
- TLS_ENABLED=yes
- TLS_CLIENT_CNS=testcertuser
- TLS_AUTH_CLIENTS_USER=CN
- REDIS_CLUSTER=no
- PORT=6379
- TLS_PORT=6666
@@ -21,9 +25,10 @@ services:
- sentinel
- all-stack
- all
- e2e
osscluster:
image: ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.4.0}
image: *default-image
platform: linux/amd64
container_name: redis-osscluster
environment:
@@ -39,14 +44,77 @@ services:
- all-stack
- all
cae-resp-proxy:
image: redislabs/client-resp-proxy:latest
container_name: cae-resp-proxy
environment:
- TARGET_HOST=redis
- TARGET_PORT=6379
- LISTEN_PORT=17000,17001,17002,17003 # 4 proxy nodes: initially show 3, swap in 4th during SMIGRATED
- LISTEN_HOST=0.0.0.0
- API_PORT=3000
- DEFAULT_INTERCEPTORS=cluster,hitless
ports:
- "17000:17000" # Proxy node 1 (host:container)
- "17001:17001" # Proxy node 2 (host:container)
- "17002:17002" # Proxy node 3 (host:container)
- "17003:17003" # Proxy node 4 (host:container) - hidden initially, swapped in during SMIGRATED
- "18100:3000" # HTTP API port (host:container)
depends_on:
- redis
profiles:
- e2e
- all
proxy-fault-injector:
build:
context: .
dockerfile: maintnotifications/e2e/cmd/proxy-fi-server/Dockerfile
container_name: proxy-fault-injector
ports:
- "15000:5000" # Fault injector API port (host:container)
depends_on:
- cae-resp-proxy
environment:
- PROXY_API_URL=http://cae-resp-proxy:3000
profiles:
- e2e
- all
osscluster-tls:
image: *default-image
platform: linux/amd64
container_name: redis-osscluster-tls
environment:
- NODES=6
- PORT=6430
- TLS_PORT=5430
- TLS_ENABLED=yes
- TLS_CLIENT_CNS=testcertuser
- TLS_AUTH_CLIENTS_USER=CN
- REDIS_CLUSTER=yes
- REPLICAS=1
command: "--tls-auth-clients optional --cluster-announce-ip 127.0.0.1"
ports:
- "6430-6435:6430-6435" # Regular ports
- "5430-5435:5430-5435" # TLS ports (set via TLS_PORT env var)
- "16430-16435:16430-16435" # Cluster bus ports (PORT + 10000)
volumes:
- "./dockers/osscluster-tls:/redis/work"
profiles:
- cluster-tls
- all
sentinel-cluster:
image: ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.4.0}
image: *default-image
platform: linux/amd64
container_name: redis-sentinel-cluster
network_mode: "host"
environment:
- NODES=3
- TLS_ENABLED=yes
- TLS_CLIENT_CNS=testcertuser
- TLS_AUTH_CLIENTS_USER=CN
- REDIS_CLUSTER=no
- PORT=9121
command: ${REDIS_EXTRA_ARGS:---enable-debug-command yes --enable-module-command yes --tls-auth-clients optional --save ""}
@@ -60,7 +128,7 @@ services:
- all
sentinel:
image: ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.4.0}
image: *default-image
platform: linux/amd64
container_name: redis-sentinel
depends_on:
@@ -84,12 +152,14 @@ services:
- all
ring-cluster:
image: ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.4.0}
image: *default-image
platform: linux/amd64
container_name: redis-ring-cluster
environment:
- NODES=3
- TLS_ENABLED=yes
- TLS_CLIENT_CNS=testcertuser
- TLS_AUTH_CLIENTS_USER=CN
- REDIS_CLUSTER=no
- PORT=6390
command: ${REDIS_EXTRA_ARGS:---enable-debug-command yes --enable-module-command yes --tls-auth-clients optional --save ""}
+14
View File
@@ -124,6 +124,9 @@ func shouldRetry(err error, retryTimeout bool) bool {
if proto.IsTryAgainError(err) {
return true
}
if proto.IsNoReplicasError(err) {
return true
}
// Fallback to string checking for backward compatibility with plain errors
s := err.Error()
@@ -145,6 +148,9 @@ func shouldRetry(err error, retryTimeout bool) bool {
if strings.HasPrefix(s, "MASTERDOWN ") {
return true
}
if strings.HasPrefix(s, "NOREPLICAS ") {
return true
}
return false
}
@@ -342,6 +348,14 @@ func IsOOMError(err error) bool {
return proto.IsOOMError(err)
}
// IsNoReplicasError checks if an error is a Redis NOREPLICAS error, even if wrapped.
// NOREPLICAS errors occur when not enough replicas acknowledge a write operation.
// This typically happens with WAIT/WAITAOF commands or CLUSTER SETSLOT with synchronous
// replication when the required number of replicas cannot confirm the write within the timeout.
func IsNoReplicasError(err error) bool {
return proto.IsNoReplicasError(err)
}
//------------------------------------------------------------------------------
type timeoutError interface {
+8 -2
View File
@@ -33,7 +33,10 @@ func (c cmdable) GeoAdd(ctx context.Context, key string, geoLocation ...*GeoLoca
return cmd
}
// GeoRadius is a read-only GEORADIUS_RO command.
// GeoRadius queries a geospatial index for members within a distance from a coordinate.
// This is a read-only variant that does not support Store or StoreDist options.
//
// Deprecated: Use GeoSearch with BYRADIUS argument instead as of Redis 6.2.0.
func (c cmdable) GeoRadius(
ctx context.Context, key string, longitude, latitude float64, query *GeoRadiusQuery,
) *GeoLocationCmd {
@@ -60,7 +63,10 @@ func (c cmdable) GeoRadiusStore(
return cmd
}
// GeoRadiusByMember is a read-only GEORADIUSBYMEMBER_RO command.
// GeoRadiusByMember queries a geospatial index for members within a distance from a member.
// This is a read-only variant that does not support Store or StoreDist options.
//
// Deprecated: Use GeoSearch with BYRADIUS and FROMMEMBER arguments instead as of Redis 6.2.0.
func (c cmdable) GeoRadiusByMember(
ctx context.Context, key, member string, query *GeoRadiusQuery,
) *GeoLocationCmd {
+122
View File
@@ -0,0 +1,122 @@
package redis
import (
"context"
"errors"
"strings"
)
// HOTKEYS commands are only available on standalone *Client instances.
// They are NOT available on ClusterClient, Ring, or UniversalClient because
// HOTKEYS is a stateful command requiring session affinity - all operations
// (START, GET, STOP, RESET) must be sent to the same Redis node.
//
// If you are using UniversalClient and need HOTKEYS functionality, you must
// type assert to *Client first:
//
// if client, ok := universalClient.(*redis.Client); ok {
// result, err := client.HotKeysStart(ctx, args)
// // ...
// }
// HotKeysMetric represents the metrics that can be tracked by the HOTKEYS command.
type HotKeysMetric string
const (
// HotKeysMetricCPU tracks CPU time spent on the key (in microseconds).
HotKeysMetricCPU HotKeysMetric = "CPU"
// HotKeysMetricNET tracks network bytes used by the key (ingress + egress + replication).
HotKeysMetricNET HotKeysMetric = "NET"
)
// HotKeysStartArgs contains the arguments for the HOTKEYS START command.
// This command is only available on standalone clients due to its stateful nature
// requiring session affinity. It must NOT be used on cluster or pooled clients.
type HotKeysStartArgs struct {
// Metrics to track. At least one must be specified.
Metrics []HotKeysMetric
// Count is the number of top keys to report.
// Default: 10, Min: 10, Max: 64
Count uint8
// Duration is the auto-stop tracking after this many seconds.
// Default: 0 (no auto-stop)
Duration int64
// Sample is the sample ratio - track keys with probability 1/sample.
// Default: 1 (track every key), Min: 1
Sample int64
// Slots specifies specific hash slots to track (0-16383).
// All specified slots must be hosted by the receiving node.
// If not specified, all slots are tracked.
Slots []uint16
}
// ErrHotKeysNoMetrics is returned when HotKeysStart is called without any metrics specified.
var ErrHotKeysNoMetrics = errors.New("redis: at least one metric must be specified for HOTKEYS START")
// HotKeysStart starts collecting hotkeys data.
// At least one metric must be specified in args.Metrics.
// This command is only available on standalone clients.
func (c *Client) HotKeysStart(ctx context.Context, args *HotKeysStartArgs) *StatusCmd {
cmdArgs := make([]interface{}, 0, 16)
cmdArgs = append(cmdArgs, "hotkeys", "start")
// Validate that at least one metric is specified
if len(args.Metrics) == 0 {
cmd := NewStatusCmd(ctx, cmdArgs...)
cmd.SetErr(ErrHotKeysNoMetrics)
return cmd
}
cmdArgs = append(cmdArgs, "metrics", len(args.Metrics))
for _, metric := range args.Metrics {
cmdArgs = append(cmdArgs, strings.ToLower(string(metric)))
}
if args.Count > 0 {
cmdArgs = append(cmdArgs, "count", args.Count)
}
if args.Duration > 0 {
cmdArgs = append(cmdArgs, "duration", args.Duration)
}
if args.Sample > 0 {
cmdArgs = append(cmdArgs, "sample", args.Sample)
}
if len(args.Slots) > 0 {
cmdArgs = append(cmdArgs, "slots", len(args.Slots))
for _, slot := range args.Slots {
cmdArgs = append(cmdArgs, slot)
}
}
cmd := NewStatusCmd(ctx, cmdArgs...)
_ = c.Process(ctx, cmd)
return cmd
}
// HotKeysStop stops the ongoing hotkeys collection session.
// This command is only available on standalone clients.
func (c *Client) HotKeysStop(ctx context.Context) *StatusCmd {
cmd := NewStatusCmd(ctx, "hotkeys", "stop")
_ = c.Process(ctx, cmd)
return cmd
}
// HotKeysReset discards the last hotkeys collection session results.
// Returns an error if tracking is currently active.
// This command is only available on standalone clients.
func (c *Client) HotKeysReset(ctx context.Context) *StatusCmd {
cmd := NewStatusCmd(ctx, "hotkeys", "reset")
_ = c.Process(ctx, cmd)
return cmd
}
// HotKeysGet retrieves the results of the ongoing or last hotkeys collection session.
// This command is only available on standalone clients.
func (c *Client) HotKeysGet(ctx context.Context) *HotKeysCmd {
cmd := NewHotKeysCmd(ctx, "hotkeys", "get")
_ = c.Process(ctx, cmd)
return cmd
}
@@ -40,6 +40,11 @@ type OptionsInterface interface {
// GetAddr returns the connection address.
GetAddr() string
// GetNodeAddress returns the address of the Redis node as reported by the server.
// For cluster clients, this is the endpoint from CLUSTER SLOTS before any transformation.
// For standalone clients, this defaults to Addr.
GetNodeAddress() string
// IsTLSEnabled returns true if TLS is enabled.
IsTLSEnabled() bool
@@ -121,6 +121,9 @@ const (
UnrelaxedTimeoutMessage = "clearing relaxed timeout"
ManagerNotInitializedMessage = "manager not initialized"
FailedToMarkForHandoffMessage = "failed to mark connection for handoff"
InvalidSeqIDInSMigratingNotificationMessage = "invalid SeqID in SMIGRATING notification"
InvalidSeqIDInSMigratedNotificationMessage = "invalid SeqID in SMIGRATED notification"
TriggeringClusterStateReloadMessage = "triggering cluster state reload"
// ========================================
// used in pool/conn
@@ -288,19 +291,29 @@ func OperationNotTracked(connID uint64, seqID int64) string {
// Connection pool functions
func RemovingConnectionFromPool(connID uint64, reason error) string {
message := fmt.Sprintf("conn[%d] %s due to: %v", connID, RemovingConnectionFromPoolMessage, reason)
return appendJSONIfDebug(message, map[string]interface{}{
metadata := map[string]interface{}{
"connID": connID,
"reason": reason.Error(),
})
"reason": "unknown", // this will be overwritten if reason is not nil
}
if reason != nil {
metadata["reason"] = reason.Error()
}
message := fmt.Sprintf("conn[%d] %s due to: %v", connID, RemovingConnectionFromPoolMessage, reason)
return appendJSONIfDebug(message, metadata)
}
func NoPoolProvidedCannotRemove(connID uint64, reason error) string {
message := fmt.Sprintf("conn[%d] %s due to: %v", connID, NoPoolProvidedMessageCannotRemoveMessage, reason)
return appendJSONIfDebug(message, map[string]interface{}{
metadata := map[string]interface{}{
"connID": connID,
"reason": reason.Error(),
})
"reason": "unknown", // this will be overwritten if reason is not nil
}
if reason != nil {
metadata["reason"] = reason.Error()
}
message := fmt.Sprintf("conn[%d] %s due to: %v", connID, NoPoolProvidedMessageCannotRemoveMessage, reason)
return appendJSONIfDebug(message, metadata)
}
// Circuit breaker functions
@@ -623,3 +636,28 @@ func ExtractDataFromLogMessage(logMessage string) map[string]interface{} {
// If JSON parsing fails, return empty map
return result
}
// Cluster notification functions
func InvalidSeqIDInSMigratingNotification(seqID interface{}) string {
message := fmt.Sprintf("%s: %v", InvalidSeqIDInSMigratingNotificationMessage, seqID)
return appendJSONIfDebug(message, map[string]interface{}{
"seqID": fmt.Sprintf("%v", seqID),
})
}
func InvalidSeqIDInSMigratedNotification(seqID interface{}) string {
message := fmt.Sprintf("%s: %v", InvalidSeqIDInSMigratedNotificationMessage, seqID)
return appendJSONIfDebug(message, map[string]interface{}{
"seqID": fmt.Sprintf("%v", seqID),
})
}
// TriggeringClusterStateReload logs when cluster state reload is triggered (deduplicated, once per seqID)
func TriggeringClusterStateReload(seqID int64, hostPort string, slotRanges []string) string {
message := fmt.Sprintf("%s seqID=%d host:port=%s slots=%v", TriggeringClusterStateReloadMessage, seqID, hostPort, slotRanges)
return appendJSONIfDebug(message, map[string]interface{}{
"seqID": seqID,
"hostPort": hostPort,
"slotRanges": slotRanges,
})
}
+279
View File
@@ -0,0 +1,279 @@
package otel
import (
"context"
"crypto/rand"
"encoding/hex"
"sync"
"time"
"github.com/redis/go-redis/v9/internal/pool"
)
// generateUniqueID generates a short unique identifier for pool names.
func generateUniqueID() string {
b := make([]byte, 4)
if _, err := rand.Read(b); err != nil {
return ""
}
return hex.EncodeToString(b)
}
// Cmder is a minimal interface for command information needed for metrics.
// This avoids circular dependencies with the main redis package.
type Cmder interface {
Name() string
FullName() string
Args() []interface{}
Err() error
}
// Recorder is the interface for recording metrics.
type Recorder interface {
// RecordOperationDuration records the total operation duration (including all retries)
// dbIndex is the Redis database index (0-15)
RecordOperationDuration(ctx context.Context, duration time.Duration, cmd Cmder, attempts int, err error, cn *pool.Conn, dbIndex int)
// RecordPipelineOperationDuration records the total pipeline/transaction duration.
// operationName should be "PIPELINE" for regular pipelines or "MULTI" for transactions.
// cmdCount is the number of commands in the pipeline.
// err is the error from the pipeline execution (can be nil).
// dbIndex is the Redis database index (0-15)
RecordPipelineOperationDuration(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn *pool.Conn, dbIndex int)
// RecordConnectionCreateTime records the time it took to create a new connection
RecordConnectionCreateTime(ctx context.Context, duration time.Duration, cn *pool.Conn)
// RecordConnectionRelaxedTimeout records when connection timeout is relaxed/unrelaxed
// delta: +1 for relaxed, -1 for unrelaxed
// poolName: name of the connection pool (e.g., "main", "pubsub")
// notificationType: the notification type that triggered the timeout relaxation (e.g., "MOVING")
RecordConnectionRelaxedTimeout(ctx context.Context, delta int, cn *pool.Conn, poolName, notificationType string)
// RecordConnectionHandoff records when a connection is handed off to another node
// poolName: name of the connection pool (e.g., "main", "pubsub")
RecordConnectionHandoff(ctx context.Context, cn *pool.Conn, poolName string)
// RecordError records client errors (ASK, MOVED, handshake failures, etc.)
// errorType: type of error (e.g., "ASK", "MOVED", "HANDSHAKE_FAILED")
// statusCode: Redis response status code if available (e.g., "MOVED", "ASK")
// isInternal: whether this is an internal error
// retryAttempts: number of retry attempts made
RecordError(ctx context.Context, errorType string, cn *pool.Conn, statusCode string, isInternal bool, retryAttempts int)
// RecordMaintenanceNotification records when a maintenance notification is received
// notificationType: the type of notification (e.g., "MOVING", "MIGRATING", etc.)
RecordMaintenanceNotification(ctx context.Context, cn *pool.Conn, notificationType string)
// RecordConnectionWaitTime records the time spent waiting for a connection from the pool
RecordConnectionWaitTime(ctx context.Context, duration time.Duration, cn *pool.Conn)
// RecordConnectionClosed records when a connection is closed
// reason: reason for closing (e.g., "idle", "max_lifetime", "error", "pool_closed")
// err: the error that caused the close (nil for non-error closures)
RecordConnectionClosed(ctx context.Context, cn *pool.Conn, reason string, err error)
// RecordPubSubMessage records a Pub/Sub message
// direction: "sent" or "received"
// channel: channel name (may be hidden for cardinality reduction)
// sharded: true for sharded pub/sub (SPUBLISH/SSUBSCRIBE)
RecordPubSubMessage(ctx context.Context, cn *pool.Conn, direction, channel string, sharded bool)
// RecordStreamLag records the lag for stream consumer group processing
// lag: time difference between message creation and consumption
// streamName: name of the stream (may be hidden for cardinality reduction)
// consumerGroup: name of the consumer group
// consumerName: name of the consumer
RecordStreamLag(ctx context.Context, lag time.Duration, cn *pool.Conn, streamName, consumerGroup, consumerName string)
}
type PubSubPooler interface {
Stats() *pool.PubSubStats
}
type PoolRegistrar interface {
// RegisterPool is called when a new client is created with its connection pools.
// poolName: identifier for the pool (e.g., "main_abc123")
// pool: the connection pool
RegisterPool(poolName string, pool pool.Pooler)
// UnregisterPool is called when a client is closed to remove its pool from the registry.
// pool: the connection pool to unregister
UnregisterPool(pool pool.Pooler)
// RegisterPubSubPool is called when a new client is created with a PubSub pool.
// poolName: identifier for the pool (e.g., "main_abc123_pubsub")
// pool: the PubSub connection pool
RegisterPubSubPool(poolName string, pool PubSubPooler)
// UnregisterPubSubPool is called when a PubSub client is closed to remove its pool.
// pool: the PubSub connection pool to unregister
UnregisterPubSubPool(pool PubSubPooler)
}
var (
// recorderMu protects globalRecorder and operation duration callbacks
recorderMu sync.RWMutex
// Global recorder instance (initialized by extra/redisotel-native)
globalRecorder Recorder = noopRecorder{}
// Callbacks for operation duration metrics
operationDurationCallback func(ctx context.Context, duration time.Duration, cmd Cmder, attempts int, err error, cn *pool.Conn, dbIndex int)
pipelineOperationDurationCallback func(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn *pool.Conn, dbIndex int)
)
// GetOperationDurationCallback returns the callback for operation duration.
func GetOperationDurationCallback() func(ctx context.Context, duration time.Duration, cmd Cmder, attempts int, err error, cn *pool.Conn, dbIndex int) {
recorderMu.RLock()
cb := operationDurationCallback
recorderMu.RUnlock()
return cb
}
// GetPipelineOperationDurationCallback returns the callback for pipeline operation duration.
func GetPipelineOperationDurationCallback() func(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn *pool.Conn, dbIndex int) {
recorderMu.RLock()
cb := pipelineOperationDurationCallback
recorderMu.RUnlock()
return cb
}
// getRecorder returns the current global recorder under a read lock.
func getRecorder() Recorder {
recorderMu.RLock()
r := globalRecorder
recorderMu.RUnlock()
return r
}
// SetGlobalRecorder sets the global recorder (called by Init() in extra/redisotel-native)
func SetGlobalRecorder(r Recorder) {
recorderMu.Lock()
if r == nil {
globalRecorder = noopRecorder{}
operationDurationCallback = nil
pipelineOperationDurationCallback = nil
recorderMu.Unlock()
// Unregister all pool metric callbacks atomically
pool.SetAllMetricCallbacks(nil)
return
}
globalRecorder = r
// Register operation duration callbacks
// These capture r directly since we want them to use the specific recorder
// that was set at this point in time
operationDurationCallback = func(ctx context.Context, duration time.Duration, cmd Cmder, attempts int, err error, cn *pool.Conn, dbIndex int) {
getRecorder().RecordOperationDuration(ctx, duration, cmd, attempts, err, cn, dbIndex)
}
pipelineOperationDurationCallback = func(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn *pool.Conn, dbIndex int) {
getRecorder().RecordPipelineOperationDuration(ctx, duration, operationName, cmdCount, attempts, err, cn, dbIndex)
}
recorderMu.Unlock()
// Register all pool metric callbacks atomically
// These use getRecorder() to safely access the current recorder
pool.SetAllMetricCallbacks(&pool.MetricCallbacks{
ConnectionCreateTime: func(ctx context.Context, duration time.Duration, cn *pool.Conn) {
getRecorder().RecordConnectionCreateTime(ctx, duration, cn)
},
ConnectionRelaxedTimeout: func(ctx context.Context, delta int, cn *pool.Conn, poolName, notificationType string) {
getRecorder().RecordConnectionRelaxedTimeout(ctx, delta, cn, poolName, notificationType)
},
ConnectionHandoff: func(ctx context.Context, cn *pool.Conn, poolName string) {
getRecorder().RecordConnectionHandoff(ctx, cn, poolName)
},
Error: func(ctx context.Context, errorType string, cn *pool.Conn, statusCode string, isInternal bool, retryAttempts int) {
getRecorder().RecordError(ctx, errorType, cn, statusCode, isInternal, retryAttempts)
},
MaintenanceNotification: func(ctx context.Context, cn *pool.Conn, notificationType string) {
getRecorder().RecordMaintenanceNotification(ctx, cn, notificationType)
},
ConnectionWaitTime: func(ctx context.Context, duration time.Duration, cn *pool.Conn) {
getRecorder().RecordConnectionWaitTime(ctx, duration, cn)
},
ConnectionClosed: func(ctx context.Context, cn *pool.Conn, reason string, err error) {
getRecorder().RecordConnectionClosed(ctx, cn, reason, err)
},
})
}
// RecordOperationDuration records the total operation duration.
// dbIndex is the Redis database index (0-15).
func RecordOperationDuration(ctx context.Context, duration time.Duration, cmd Cmder, attempts int, err error, cn *pool.Conn, dbIndex int) {
getRecorder().RecordOperationDuration(ctx, duration, cmd, attempts, err, cn, dbIndex)
}
// RecordPipelineOperationDuration records the total pipeline/transaction duration.
// This is called from redis.go after pipeline/transaction execution completes.
// operationName should be "PIPELINE" for regular pipelines or "MULTI" for transactions.
// err is the error from the pipeline execution (can be nil).
// dbIndex is the Redis database index (0-15).
func RecordPipelineOperationDuration(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn *pool.Conn, dbIndex int) {
getRecorder().RecordPipelineOperationDuration(ctx, duration, operationName, cmdCount, attempts, err, cn, dbIndex)
}
// RecordConnectionCreateTime records the time it took to create a new connection.
func RecordConnectionCreateTime(ctx context.Context, duration time.Duration, cn *pool.Conn) {
getRecorder().RecordConnectionCreateTime(ctx, duration, cn)
}
// RecordPubSubMessage records a Pub/Sub message sent or received.
func RecordPubSubMessage(ctx context.Context, cn *pool.Conn, direction, channel string, sharded bool) {
getRecorder().RecordPubSubMessage(ctx, cn, direction, channel, sharded)
}
// RecordStreamLag records the lag between message creation and consumption in a stream.
func RecordStreamLag(ctx context.Context, lag time.Duration, cn *pool.Conn, streamName, consumerGroup, consumerName string) {
getRecorder().RecordStreamLag(ctx, lag, cn, streamName, consumerGroup, consumerName)
}
type noopRecorder struct{}
func (noopRecorder) RecordOperationDuration(context.Context, time.Duration, Cmder, int, error, *pool.Conn, int) {
}
func (noopRecorder) RecordPipelineOperationDuration(context.Context, time.Duration, string, int, int, error, *pool.Conn, int) {
}
func (noopRecorder) RecordConnectionCreateTime(context.Context, time.Duration, *pool.Conn) {}
func (noopRecorder) RecordConnectionRelaxedTimeout(context.Context, int, *pool.Conn, string, string) {
}
func (noopRecorder) RecordConnectionHandoff(context.Context, *pool.Conn, string) {}
func (noopRecorder) RecordError(context.Context, string, *pool.Conn, string, bool, int) {}
func (noopRecorder) RecordMaintenanceNotification(context.Context, *pool.Conn, string) {}
func (noopRecorder) RecordConnectionWaitTime(context.Context, time.Duration, *pool.Conn) {}
func (noopRecorder) RecordConnectionClosed(context.Context, *pool.Conn, string, error) {}
func (noopRecorder) RecordPubSubMessage(context.Context, *pool.Conn, string, string, bool) {}
func (noopRecorder) RecordStreamLag(context.Context, time.Duration, *pool.Conn, string, string, string) {
}
// RegisterPools registers connection pools with the global recorder.
func RegisterPools(connPool pool.Pooler, pubSubPool PubSubPooler, addr string) {
// Check if the global recorder implements PoolRegistrar
if registrar, ok := globalRecorder.(PoolRegistrar); ok {
// Generate a unique ID for this client's pools
uniqueID := generateUniqueID()
if connPool != nil {
poolName := addr + "_" + uniqueID
registrar.RegisterPool(poolName, connPool)
}
if pubSubPool != nil {
poolName := addr + "_" + uniqueID + "_pubsub"
registrar.RegisterPubSubPool(poolName, pubSubPool)
}
}
}
// UnregisterPools removes connection pools from the global recorder
func UnregisterPools(connPool pool.Pooler, pubSubPool PubSubPooler) {
// Check if the global recorder implements PoolRegistrar
if registrar, ok := globalRecorder.(PoolRegistrar); ok {
if connPool != nil {
registrar.UnregisterPool(connPool)
}
if pubSubPool != nil {
registrar.UnregisterPubSubPool(pubSubPool)
}
}
}
+29 -2
View File
@@ -69,8 +69,9 @@ type Conn struct {
// Connection identifier for unique tracking
id uint64
usedAt atomic.Int64
lastPutAt atomic.Int64
usedAt atomic.Int64
lastPutAt atomic.Int64
dialStartNs atomic.Int64 // Time when dial started (for connection create time metric)
// Lock-free netConn access using atomic.Value
// Contains *atomicNetConn wrapper, accessed atomically for better performance
@@ -104,6 +105,7 @@ type Conn struct {
closed atomic.Bool
createdAt time.Time
expiresAt time.Time
poolName string // Name of the pool this connection belongs to (for metrics)
// maintenanceNotifications upgrade support: relaxed timeouts during migrations/failovers
@@ -184,6 +186,24 @@ func (cn *Conn) SetLastPutAtNs(ns int64) {
cn.lastPutAt.Store(ns)
}
// GetDialStartNs returns the time when the dial started (in nanoseconds since epoch).
// This is used to calculate the full connection creation time (TCP + handshake).
func (cn *Conn) GetDialStartNs() int64 {
return cn.dialStartNs.Load()
}
// PoolName returns the name of the pool this connection belongs to.
// This is used for metrics to identify which pool a connection is from.
func (cn *Conn) PoolName() string {
return cn.poolName
}
// SetPoolName sets the name of the pool this connection belongs to.
// This should be called when the connection is added to a pool.
func (cn *Conn) SetPoolName(name string) {
cn.poolName = name
}
// Backward-compatible wrapper methods for state machine
// These maintain the existing API while using the new state machine internally
@@ -418,6 +438,8 @@ func (cn *Conn) IsPubSub() bool {
// SetRelaxedTimeout sets relaxed timeouts for this connection during maintenanceNotifications upgrades.
// These timeouts will be used for all subsequent commands until the deadline expires.
// Uses atomic operations for lock-free access.
// Note: Metrics should be recorded by the caller (notification handler) which has context about
// the notification type and pool name.
func (cn *Conn) SetRelaxedTimeout(readTimeout, writeTimeout time.Duration) {
cn.relaxedCounter.Add(1)
cn.relaxedReadTimeoutNs.Store(int64(readTimeout))
@@ -452,6 +474,11 @@ func (cn *Conn) clearRelaxedTimeout() {
cn.relaxedWriteTimeoutNs.Store(0)
cn.relaxedDeadlineNs.Store(0)
cn.relaxedCounter.Store(0)
// Note: Metrics for timeout unrelaxing are not recorded here because we don't have
// context about which notification type or pool triggered the relaxation.
// In practice, relaxed timeouts expire automatically via deadline, so explicit
// unrelaxing metrics are less critical than the initial relaxation metrics.
}
// HasRelaxedTimeout returns true if relaxed timeouts are currently active on this connection.
+7 -7
View File
@@ -13,11 +13,12 @@ import (
// States are designed to be lightweight and fast to check.
//
// State Transitions:
// CREATED → INITIALIZING → IDLE ⇄ IN_USE
//
// UNUSABLE (handoff/reauth)
//
// IDLE/CLOSED
//
// CREATED → INITIALIZING → IDLE ⇄ IN_USE
//
// UNUSABLE (handoff/reauth)
//
// IDLE/CLOSED
type ConnState uint32
const (
@@ -120,7 +121,7 @@ type ConnStateMachine struct {
// FIFO queue for waiters - only locked during waiter add/remove/notify
mu sync.Mutex
waiters *list.List // List of *waiter
waiters *list.List // List of *waiter
waiterCount atomic.Int32 // Fast lock-free check for waiters (avoids mutex in hot path)
}
@@ -340,4 +341,3 @@ func (sm *ConnStateMachine) notifyWaiters() {
}
}
}
+354 -31
View File
@@ -10,7 +10,7 @@ import (
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/proto"
"github.com/redis/go-redis/v9/internal/util"
"github.com/redis/go-redis/v9/internal/rand"
)
var (
@@ -32,6 +32,45 @@ var (
// errConnNotPooled is returned when trying to return a non-pooled connection to the pool.
errConnNotPooled = errors.New("connection not pooled")
// metricCallbackMu protects all global metric callback functions for thread-safe access.
metricCallbackMu sync.RWMutex
// Global metric callbacks for connection state changes
metricConnectionStateChangeCallback func(ctx context.Context, cn *Conn, fromState, toState string)
// Global metric callback for connection creation time
metricConnectionCreateTimeCallback func(ctx context.Context, duration time.Duration, cn *Conn)
// Global metric callback for connection relaxed timeout changes
// Parameters: ctx, delta (+1/-1), cn, poolName, notificationType
metricConnectionRelaxedTimeoutCallback func(ctx context.Context, delta int, cn *Conn, poolName, notificationType string)
// Global metric callback for connection handoff
// Parameters: ctx, cn, poolName
metricConnectionHandoffCallback func(ctx context.Context, cn *Conn, poolName string)
// Global metric callback for error tracking
// Parameters: ctx, errorType, cn, statusCode, isInternal, retryAttempts
metricErrorCallback func(ctx context.Context, errorType string, cn *Conn, statusCode string, isInternal bool, retryAttempts int)
// Global metric callback for maintenance notifications
// Parameters: ctx, cn, notificationType
metricMaintenanceNotificationCallback func(ctx context.Context, cn *Conn, notificationType string)
// Global metric callback for connection wait time
// Parameters: ctx, duration, cn
metricConnectionWaitTimeCallback func(ctx context.Context, duration time.Duration, cn *Conn)
// Global metric callback for connection timeouts
// Parameters: ctx, cn, timeoutType
metricConnectionTimeoutCallback func(ctx context.Context, cn *Conn, timeoutType string)
// Global metric callback for connection closed
// Parameters: ctx, cn, reason, err
metricConnectionClosedCallback func(ctx context.Context, cn *Conn, reason string, err error)
// errPanicInDial is returned when a panic occurs in the dial function.
errPanicInQueuedNewConn = errors.New("panic in queuedNewConn")
// popAttempts is the maximum number of attempts to find a usable connection
// when popping from the idle connection pool. This handles cases where connections
@@ -51,6 +90,139 @@ var (
noExpiration = maxTime
)
// MetricCallbacks holds all metric callback functions.
// Use SetAllMetricCallbacks to register all callbacks atomically.
type MetricCallbacks struct {
// ConnectionCreateTime is called when a new connection is created
ConnectionCreateTime func(ctx context.Context, duration time.Duration, cn *Conn)
// ConnectionRelaxedTimeout is called when connection timeout is relaxed/unrelaxed
// delta: +1 for relaxed, -1 for unrelaxed
ConnectionRelaxedTimeout func(ctx context.Context, delta int, cn *Conn, poolName, notificationType string)
// ConnectionHandoff is called when a connection is handed off to another node
ConnectionHandoff func(ctx context.Context, cn *Conn, poolName string)
// Error is called when an error occurs
Error func(ctx context.Context, errorType string, cn *Conn, statusCode string, isInternal bool, retryAttempts int)
// MaintenanceNotification is called when a maintenance notification is received
MaintenanceNotification func(ctx context.Context, cn *Conn, notificationType string)
// ConnectionWaitTime is called to record time spent waiting for a connection
ConnectionWaitTime func(ctx context.Context, duration time.Duration, cn *Conn)
// ConnectionClosed is called when a connection is closed
ConnectionClosed func(ctx context.Context, cn *Conn, reason string, err error)
}
// SetAllMetricCallbacks sets all metric callbacks atomically.
// Pass nil to clear all callbacks (disable metrics).
// This ensures all callbacks are set together under a single lock,
// preventing inconsistent state during registration.
//
// Note on thread safety: After returning, there is a small window where
// concurrent getMetric* calls may return the old callback value. This is
// acceptable for metrics - at most one event may go to the old recorder
// or be missed during the transition. The callbacks themselves are immutable
// function pointers, so calling an "old" callback is safe.
func SetAllMetricCallbacks(callbacks *MetricCallbacks) {
metricCallbackMu.Lock()
defer metricCallbackMu.Unlock()
if callbacks == nil {
metricConnectionCreateTimeCallback = nil
metricConnectionRelaxedTimeoutCallback = nil
metricConnectionHandoffCallback = nil
metricErrorCallback = nil
metricMaintenanceNotificationCallback = nil
metricConnectionWaitTimeCallback = nil
metricConnectionClosedCallback = nil
return
}
metricConnectionCreateTimeCallback = callbacks.ConnectionCreateTime
metricConnectionRelaxedTimeoutCallback = callbacks.ConnectionRelaxedTimeout
metricConnectionHandoffCallback = callbacks.ConnectionHandoff
metricErrorCallback = callbacks.Error
metricMaintenanceNotificationCallback = callbacks.MaintenanceNotification
metricConnectionWaitTimeCallback = callbacks.ConnectionWaitTime
metricConnectionClosedCallback = callbacks.ConnectionClosed
}
// getMetricConnectionStateChangeCallback returns the metric callback for connection state changes.
func getMetricConnectionStateChangeCallback() func(ctx context.Context, cn *Conn, fromState, toState string) {
metricCallbackMu.RLock()
cb := metricConnectionStateChangeCallback
metricCallbackMu.RUnlock()
return cb
}
// GetMetricConnectionCreateTimeCallback returns the metric callback for connection creation time.
func GetMetricConnectionCreateTimeCallback() func(ctx context.Context, duration time.Duration, cn *Conn) {
metricCallbackMu.RLock()
cb := metricConnectionCreateTimeCallback
metricCallbackMu.RUnlock()
return cb
}
// GetMetricConnectionRelaxedTimeoutCallback returns the metric callback for connection relaxed timeout changes.
// This is used by maintnotifications to record relaxed timeout metrics.
func GetMetricConnectionRelaxedTimeoutCallback() func(ctx context.Context, delta int, cn *Conn, poolName, notificationType string) {
metricCallbackMu.RLock()
cb := metricConnectionRelaxedTimeoutCallback
metricCallbackMu.RUnlock()
return cb
}
// GetMetricConnectionHandoffCallback returns the metric callback for connection handoffs.
// This is used by maintnotifications to record handoff metrics.
func GetMetricConnectionHandoffCallback() func(ctx context.Context, cn *Conn, poolName string) {
metricCallbackMu.RLock()
cb := metricConnectionHandoffCallback
metricCallbackMu.RUnlock()
return cb
}
// GetMetricErrorCallback returns the metric callback for error tracking.
// This is used by cluster and client code to record error metrics.
func GetMetricErrorCallback() func(ctx context.Context, errorType string, cn *Conn, statusCode string, isInternal bool, retryAttempts int) {
metricCallbackMu.RLock()
cb := metricErrorCallback
metricCallbackMu.RUnlock()
return cb
}
// GetMetricMaintenanceNotificationCallback returns the metric callback for maintenance notifications.
// This is used by maintnotifications to record notification metrics.
func GetMetricMaintenanceNotificationCallback() func(ctx context.Context, cn *Conn, notificationType string) {
metricCallbackMu.RLock()
cb := metricMaintenanceNotificationCallback
metricCallbackMu.RUnlock()
return cb
}
func getMetricConnectionWaitTimeCallback() func(ctx context.Context, duration time.Duration, cn *Conn) {
metricCallbackMu.RLock()
cb := metricConnectionWaitTimeCallback
metricCallbackMu.RUnlock()
return cb
}
func getMetricConnectionTimeoutCallback() func(ctx context.Context, cn *Conn, timeoutType string) {
metricCallbackMu.RLock()
cb := metricConnectionTimeoutCallback
metricCallbackMu.RUnlock()
return cb
}
func getMetricConnectionClosedCallback() func(ctx context.Context, cn *Conn, reason string, err error) {
metricCallbackMu.RLock()
cb := metricConnectionClosedCallback
metricCallbackMu.RUnlock()
return cb
}
// Stats contains pool state information and accumulated stats.
type Stats struct {
Hits uint32 // number of times free connection was found in the pool
@@ -60,9 +232,10 @@ type Stats struct {
Unusable uint32 // number of times a connection was found to be unusable
WaitDurationNs int64 // total time spent for waiting a connection in nanoseconds
TotalConns uint32 // number of total connections in the pool
IdleConns uint32 // number of idle connections in the pool
StaleConns uint32 // number of stale connections removed from the pool
TotalConns uint32 // number of total connections in the pool
IdleConns uint32 // number of idle connections in the pool
StaleConns uint32 // number of stale connections removed from the pool
PendingRequests uint32 // number of pending requests waiting for a connection
PubSubStats PubSubStats
}
@@ -110,6 +283,7 @@ type Options struct {
MaxActiveConns int32
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
ConnMaxLifetimeJitter time.Duration
PushNotificationsEnabled bool
// DialerRetries is the maximum number of retry attempts when dialing fails.
@@ -119,6 +293,10 @@ type Options struct {
// DialerRetryTimeout is the backoff duration between retry attempts.
// Default: 100ms
DialerRetryTimeout time.Duration
// Name is a unique identifier for this pool, used in metrics.
// Format: addr_uniqueID (e.g., "localhost:6379_a1b2c3d4")
Name string
}
type lastDialErrorWrap struct {
@@ -240,9 +418,9 @@ func (p *ConnPool) checkMinIdleConns() {
for p.poolSize.Load() < p.cfg.PoolSize && p.idleConnsLen.Load() < p.cfg.MinIdleConns {
// Try to acquire a semaphore token
if !p.semaphore.TryAcquire() {
// Semaphore is full, can't create more connections
p.idleCheckInProgress.Store(false)
return
// Semaphore is full, can't create more connections right now
// Break out of inner loop to check if we need to retry
break
}
p.poolSize.Add(1)
@@ -321,6 +499,12 @@ func (p *ConnPool) newConn(ctx context.Context, pooled bool) (*Conn, error) {
return nil, ErrPoolExhausted
}
// Protect against nil context due to race condition in queuedNewConn
// where the context can be set to nil after timeout/cancellation
if ctx == nil {
ctx = context.Background()
}
dialCtx, cancel := context.WithTimeout(ctx, p.cfg.DialTimeout)
defer cancel()
cn, err := p.dialConn(dialCtx, pooled)
@@ -359,6 +543,11 @@ func (p *ConnPool) newConn(ctx context.Context, pooled bool) (*Conn, error) {
}
}
// Notify metrics: new connection created and idle
if cb := getMetricConnectionStateChangeCallback(); cb != nil {
cb(ctx, cn, "", "idle")
}
return cn, nil
}
@@ -371,6 +560,14 @@ func (p *ConnPool) dialConn(ctx context.Context, pooled bool) (*Conn, error) {
return nil, p.getLastDialError()
}
// Record dial start time for connection creation metric
// This will be used after handshake completes in redis.go _getConn()
// Only call time.Now() if callback is registered to avoid overhead
var dialStartNs int64
if GetMetricConnectionCreateTimeCallback() != nil {
dialStartNs = time.Now().UnixNano()
}
// Retry dialing with backoff
// the context timeout is already handled by the context passed in
// so we may never reach the max retries, higher values don't hurt
@@ -404,14 +601,15 @@ func (p *ConnPool) dialConn(ctx context.Context, pooled bool) (*Conn, error) {
continue
}
// Success - create connection
cn := NewConnWithBufferSize(netConn, p.cfg.ReadBufferSize, p.cfg.WriteBufferSize)
cn.pooled = pooled
if p.cfg.ConnMaxLifetime > 0 {
cn.expiresAt = time.Now().Add(p.cfg.ConnMaxLifetime)
} else {
cn.expiresAt = noExpiration
// Store dial start time only if we recorded it
if dialStartNs > 0 {
cn.dialStartNs.Store(dialStartNs)
}
cn.expiresAt = p.calcConnExpiresAt()
// Set pool name for metrics
cn.SetPoolName(p.cfg.Name)
return cn, nil
}
@@ -425,6 +623,25 @@ func (p *ConnPool) dialConn(ctx context.Context, pooled bool) (*Conn, error) {
return nil, lastErr
}
// calcConnExpiresAt calculates the expiration time for a connection.
// It applies random jitter to prevent all connections from expiring simultaneously,
// avoiding the "thundering herd" problem where all connections expire at once.
// Returns noExpiration if ConnMaxLifetime is not set.
func (p *ConnPool) calcConnExpiresAt() time.Time {
if p.cfg.ConnMaxLifetime <= 0 {
return noExpiration
}
if p.cfg.ConnMaxLifetimeJitter <= 0 {
return time.Now().Add(p.cfg.ConnMaxLifetime)
}
jitter := p.cfg.ConnMaxLifetimeJitter
jitterRange := jitter.Nanoseconds() * 2
jitterNs := rand.Int63n(jitterRange) - jitter.Nanoseconds()
return time.Now().Add(p.cfg.ConnMaxLifetime + time.Duration(jitterNs))
}
func (p *ConnPool) tryDial() {
for {
if p.closed() {
@@ -466,17 +683,44 @@ func (p *ConnPool) Get(ctx context.Context) (*Conn, error) {
}
// getConn returns a connection from the pool.
func (p *ConnPool) getConn(ctx context.Context) (*Conn, error) {
var cn *Conn
var err error
func (p *ConnPool) getConn(ctx context.Context) (cn *Conn, err error) {
if p.closed() {
return nil, ErrClosed
}
if err := p.waitTurn(ctx); err != nil {
// Track pending requests in pool stats
// NOTE: We only track in stats, not via callback. The AsyncGauge reads stats directly.
atomic.AddUint32(&p.stats.PendingRequests, 1)
defer func() {
if err != nil {
// Failed to get connection, decrement pending requests
atomic.AddUint32(&p.stats.PendingRequests, ^uint32(0)) // -1
}
}()
// Track wait time - only call time.Now() if callback is registered
var waitStart time.Time
waitTimeCallback := getMetricConnectionWaitTimeCallback()
if waitTimeCallback != nil {
waitStart = time.Now()
}
if err = p.waitTurn(ctx); err != nil {
// Record timeout if applicable
if err == ErrPoolTimeout {
if cb := getMetricConnectionTimeoutCallback(); cb != nil {
cb(ctx, nil, "pool")
}
// Record general error metric for pool timeout
if cb := GetMetricErrorCallback(); cb != nil {
cb(ctx, "POOL_TIMEOUT", nil, "POOL_TIMEOUT", true, 0)
}
}
return nil, err
}
var waitDuration time.Duration
if waitTimeCallback != nil {
waitDuration = time.Since(waitStart)
}
// Use cached time for health checks (max 50ms staleness is acceptable)
nowNs := getCachedTimeNs()
@@ -507,10 +751,10 @@ func (p *ConnPool) getConn(ctx context.Context) (*Conn, error) {
// Process connection using the hooks system
// Combine error and rejection checks to reduce branches
if hookManager != nil {
acceptConn, err := hookManager.ProcessOnGet(ctx, cn, false)
if err != nil || !acceptConn {
if err != nil {
internal.Logger.Printf(ctx, "redis: connection pool: failed to process idle connection by hook: %v", err)
acceptConn, hookErr := hookManager.ProcessOnGet(ctx, cn, false)
if hookErr != nil || !acceptConn {
if hookErr != nil {
internal.Logger.Printf(ctx, "redis: connection pool: failed to process idle connection by hook: %v", hookErr)
_ = p.CloseConn(cn)
} else {
internal.Logger.Printf(ctx, "redis: connection pool: conn[%d] rejected by hook, returning to pool", cn.GetID())
@@ -524,19 +768,37 @@ func (p *ConnPool) getConn(ctx context.Context) (*Conn, error) {
}
atomic.AddUint32(&p.stats.Hits, 1)
// Notify metrics: connection moved from idle to used
if cb := getMetricConnectionStateChangeCallback(); cb != nil {
cb(ctx, cn, "idle", "used")
}
// Record wait time (use cached callback from above)
if waitTimeCallback != nil {
waitTimeCallback(ctx, waitDuration, cn)
}
// Decrement pending requests (connection acquired successfully)
// NOTE: We only track in stats, not via callback. The AsyncGauge reads stats directly.
atomic.AddUint32(&p.stats.PendingRequests, ^uint32(0)) // -1
return cn, nil
}
atomic.AddUint32(&p.stats.Misses, 1)
newcn, err := p.queuedNewConn(ctx)
var newcn *Conn
newcn, err = p.queuedNewConn(ctx)
if err != nil {
return nil, err
}
// Process connection using the hooks system
// This includes the handshake (HELLO/AUTH) via initConn hook
if hookManager != nil {
acceptConn, err := hookManager.ProcessOnGet(ctx, newcn, true)
var acceptConn bool
acceptConn, err = hookManager.ProcessOnGet(ctx, newcn, true)
// both errors and accept=false mean a hook rejected the connection
// this should not happen with a new connection, but we handle it gracefully
if err != nil || !acceptConn {
@@ -546,6 +808,21 @@ func (p *ConnPool) getConn(ctx context.Context) (*Conn, error) {
return nil, err
}
}
// Notify metrics: new connection is created and used
if cb := getMetricConnectionStateChangeCallback(); cb != nil {
cb(ctx, newcn, "", "used")
}
// Record wait time (use cached callback from above)
if waitTimeCallback != nil {
waitTimeCallback(ctx, waitDuration, newcn)
}
// Decrement pending requests (connection acquired successfully)
// NOTE: We only track in stats, not via callback. The AsyncGauge reads stats directly.
atomic.AddUint32(&p.stats.PendingRequests, ^uint32(0)) // -1
return newcn, nil
}
@@ -574,12 +851,15 @@ func (p *ConnPool) queuedNewConn(ctx context.Context) (*Conn, error) {
}
}()
p.dialsQueue.discardDoneAtFront()
p.dialsQueue.enqueue(w)
go func(w *wantConn) {
var freeTurnCalled bool
defer func() {
if err := recover(); err != nil {
w.tryDeliver(nil, errPanicInQueuedNewConn)
p.dialsQueue.discardDoneAtFront()
if !freeTurnCalled {
p.freeTurn()
}
@@ -594,12 +874,14 @@ func (p *ConnPool) queuedNewConn(ctx context.Context) (*Conn, error) {
cn, cnErr := p.newConn(dialCtx, true)
if cnErr != nil {
w.tryDeliver(nil, cnErr) // deliver error to caller, notify connection creation failed
p.dialsQueue.discardDoneAtFront()
p.freeTurn()
freeTurnCalled = true
return
}
delivered := w.tryDeliver(cn, cnErr)
p.dialsQueue.discardDoneAtFront()
if !delivered && p.putIdleConn(dialCtx, cn) {
p.freeTurn()
freeTurnCalled = true
@@ -695,7 +977,7 @@ func (p *ConnPool) popIdle() (*Conn, error) {
var cn *Conn
attempts := 0
maxAttempts := util.Min(popAttempts, n)
maxAttempts := min(popAttempts, n)
for attempts < maxAttempts {
if len(p.idleConns) == 0 {
return nil, nil
@@ -756,6 +1038,15 @@ func (p *ConnPool) putConnWithoutTurn(ctx context.Context, cn *Conn) {
// putConn is the internal implementation of Put that optionally frees a turn.
func (p *ConnPool) putConn(ctx context.Context, cn *Conn, freeTurn bool) {
// Guard against nil connection
if cn == nil {
internal.Logger.Printf(ctx, "putConn called with nil connection")
if freeTurn {
p.freeTurn()
}
return
}
// Process connection using the hooks system
shouldPool := true
shouldRemove := false
@@ -806,7 +1097,14 @@ func (p *ConnPool) putConn(ctx context.Context, cn *Conn, freeTurn bool) {
if !transitionedToIdle {
// Fast path failed - hook might have changed state (e.g., to UNUSABLE for handoff)
// Keep the state set by the hook and pool the connection anyway
currentState := cn.GetStateMachine().GetState()
sm := cn.GetStateMachine()
if sm == nil {
// State machine is nil - connection is in an invalid state, remove it
internal.Logger.Printf(ctx, "conn[%d] has nil state machine, removing it", cn.GetID())
p.removeConnInternal(ctx, cn, errConnNotPooled, freeTurn)
return
}
currentState := sm.GetState()
switch currentState {
case StateUnusable:
// expected state, don't log it
@@ -840,9 +1138,19 @@ func (p *ConnPool) putConn(ctx context.Context, cn *Conn, freeTurn bool) {
p.connsMu.Unlock()
p.idleConnsLen.Add(1)
}
// Notify metrics: connection moved from used to idle
if cb := getMetricConnectionStateChangeCallback(); cb != nil {
cb(ctx, cn, "used", "idle")
}
} else {
shouldCloseConn = true
p.removeConnWithLock(cn)
// Notify metrics: connection removed (used -> nothing)
if cb := getMetricConnectionStateChangeCallback(); cb != nil {
cb(ctx, cn, "used", "")
}
}
if freeTurn {
@@ -883,6 +1191,20 @@ func (p *ConnPool) removeConnInternal(ctx context.Context, cn *Conn, reason erro
p.freeTurn()
}
// Notify metrics: connection removed (assume from used state)
if cb := getMetricConnectionStateChangeCallback(); cb != nil {
cb(ctx, cn, "used", "")
}
// Record connection closed
if cb := getMetricConnectionClosedCallback(); cb != nil {
reasonStr := "unknown"
if reason != nil {
reasonStr = reason.Error()
}
cb(ctx, cn, reasonStr, reason)
}
_ = p.closeConn(cn)
// Check if we need to create new idle connections to maintain MinIdleConns
@@ -949,12 +1271,13 @@ func (p *ConnPool) Size() int {
func (p *ConnPool) Stats() *Stats {
return &Stats{
Hits: atomic.LoadUint32(&p.stats.Hits),
Misses: atomic.LoadUint32(&p.stats.Misses),
Timeouts: atomic.LoadUint32(&p.stats.Timeouts),
WaitCount: atomic.LoadUint32(&p.stats.WaitCount),
Unusable: atomic.LoadUint32(&p.stats.Unusable),
WaitDurationNs: p.waitDurationNs.Load(),
Hits: atomic.LoadUint32(&p.stats.Hits),
Misses: atomic.LoadUint32(&p.stats.Misses),
Timeouts: atomic.LoadUint32(&p.stats.Timeouts),
WaitCount: atomic.LoadUint32(&p.stats.WaitCount),
Unusable: atomic.LoadUint32(&p.stats.Unusable),
WaitDurationNs: p.waitDurationNs.Load(),
PendingRequests: atomic.LoadUint32(&p.stats.PendingRequests),
TotalConns: uint32(p.Len()),
IdleConns: uint32(p.IdleLen()),
+2 -1
View File
@@ -44,9 +44,10 @@ func (p *PubSubPool) NewConn(ctx context.Context, network string, addr string, c
}
cn := NewConnWithBufferSize(netConn, p.opt.ReadBufferSize, p.opt.WriteBufferSize)
cn.pubsub = true
// Set pool name for metrics
cn.SetPoolName(p.opt.Name)
atomic.AddUint32(&p.stats.Created, 1)
return cn, nil
}
func (p *PubSubPool) TrackConn(cn *Conn) {
+25 -3
View File
@@ -6,7 +6,7 @@ import (
)
type wantConn struct {
mu sync.Mutex // protects ctx, done and sending of the result
mu sync.RWMutex // protects ctx, done and sending of the result
ctx context.Context // context for dial, cleared after delivered or canceled
cancelCtx context.CancelFunc
done bool // true after delivered or canceled
@@ -15,8 +15,8 @@ type wantConn struct {
// getCtxForDial returns context for dial or nil if connection was delivered or canceled.
func (w *wantConn) getCtxForDial() context.Context {
w.mu.Lock()
defer w.mu.Unlock()
w.mu.RLock()
defer w.mu.RUnlock()
return w.ctx
}
@@ -57,6 +57,12 @@ func (w *wantConn) cancel() *Conn {
return cn
}
func (w *wantConn) isOngoing() bool {
w.mu.RLock()
defer w.mu.RUnlock()
return !w.done
}
type wantConnResult struct {
cn *Conn
err error
@@ -91,3 +97,19 @@ func (q *wantConnQueue) dequeue() (*wantConn, bool) {
q.items = q.items[1:]
return item, true
}
func (q *wantConnQueue) discardDoneAtFront() int {
q.mu.Lock()
defer q.mu.Unlock()
count := 0
for len(q.items) > 0 {
if q.items[0].isOngoing() {
break
}
q.items = q.items[1:]
count++
}
return count
}
@@ -212,6 +212,25 @@ func NewOOMError(msg string) *OOMError {
return &OOMError{msg: msg}
}
// NoReplicasError is returned when not enough replicas acknowledge a write.
// This error occurs when using WAIT/WAITAOF commands or CLUSTER SETSLOT with
// synchronous replication, and the required number of replicas cannot confirm
// the write within the timeout period.
type NoReplicasError struct {
msg string
}
func (e *NoReplicasError) Error() string {
return e.msg
}
func (e *NoReplicasError) RedisError() {}
// NewNoReplicasError creates a new NoReplicasError with the given message.
func NewNoReplicasError(msg string) *NoReplicasError {
return &NoReplicasError{msg: msg}
}
// parseTypedRedisError parses a Redis error message and returns a typed error if applicable.
// This function maintains backward compatibility by keeping the same error messages.
func parseTypedRedisError(msg string) error {
@@ -235,6 +254,8 @@ func parseTypedRedisError(msg string) error {
return NewTryAgainError(msg)
case strings.HasPrefix(msg, "MASTERDOWN "):
return NewMasterDownError(msg)
case strings.HasPrefix(msg, "NOREPLICAS "):
return NewNoReplicasError(msg)
case msg == "ERR max number of clients reached":
return NewMaxClientsError(msg)
case strings.HasPrefix(msg, "NOAUTH "), strings.HasPrefix(msg, "WRONGPASS "), strings.Contains(msg, "unauthenticated"):
@@ -486,3 +507,21 @@ func IsOOMError(err error) bool {
// Fallback to string checking for backward compatibility
return strings.HasPrefix(err.Error(), "OOM ")
}
// IsNoReplicasError checks if an error is a NoReplicasError, even if wrapped.
func IsNoReplicasError(err error) bool {
if err == nil {
return false
}
var noReplicasErr *NoReplicasError
if errors.As(err, &noReplicasErr) {
return true
}
// Check if wrapped error is a RedisError with NOREPLICAS prefix
var redisErr RedisError
if errors.As(err, &redisErr) && strings.HasPrefix(redisErr.Error(), "NOREPLICAS ") {
return true
}
// Fallback to string checking for backward compatibility
return strings.HasPrefix(err.Error(), "NOREPLICAS ")
}
File diff suppressed because it is too large Load Diff
+144
View File
@@ -0,0 +1,144 @@
package routing
import (
"fmt"
"strings"
)
type RequestPolicy uint8
const (
ReqDefault RequestPolicy = iota
ReqAllNodes
ReqAllShards
ReqMultiShard
ReqSpecial
)
const (
ReadOnlyCMD string = "readonly"
)
func (p RequestPolicy) String() string {
switch p {
case ReqDefault:
return "default"
case ReqAllNodes:
return "all_nodes"
case ReqAllShards:
return "all_shards"
case ReqMultiShard:
return "multi_shard"
case ReqSpecial:
return "special"
default:
return fmt.Sprintf("unknown_request_policy(%d)", p)
}
}
func ParseRequestPolicy(raw string) (RequestPolicy, error) {
switch strings.ToLower(raw) {
case "", "default", "none":
return ReqDefault, nil
case "all_nodes":
return ReqAllNodes, nil
case "all_shards":
return ReqAllShards, nil
case "multi_shard":
return ReqMultiShard, nil
case "special":
return ReqSpecial, nil
default:
return ReqDefault, fmt.Errorf("routing: unknown request_policy %q", raw)
}
}
type ResponsePolicy uint8
const (
RespDefaultKeyless ResponsePolicy = iota
RespDefaultHashSlot
RespAllSucceeded
RespOneSucceeded
RespAggSum
RespAggMin
RespAggMax
RespAggLogicalAnd
RespAggLogicalOr
RespSpecial
)
func (p ResponsePolicy) String() string {
switch p {
case RespDefaultKeyless:
return "default(keyless)"
case RespDefaultHashSlot:
return "default(hashslot)"
case RespAllSucceeded:
return "all_succeeded"
case RespOneSucceeded:
return "one_succeeded"
case RespAggSum:
return "agg_sum"
case RespAggMin:
return "agg_min"
case RespAggMax:
return "agg_max"
case RespAggLogicalAnd:
return "agg_logical_and"
case RespAggLogicalOr:
return "agg_logical_or"
case RespSpecial:
return "special"
default:
return "all_succeeded"
}
}
func ParseResponsePolicy(raw string) (ResponsePolicy, error) {
switch strings.ToLower(raw) {
case "default(keyless)":
return RespDefaultKeyless, nil
case "default(hashslot)":
return RespDefaultHashSlot, nil
case "all_succeeded":
return RespAllSucceeded, nil
case "one_succeeded":
return RespOneSucceeded, nil
case "agg_sum":
return RespAggSum, nil
case "agg_min":
return RespAggMin, nil
case "agg_max":
return RespAggMax, nil
case "agg_logical_and":
return RespAggLogicalAnd, nil
case "agg_logical_or":
return RespAggLogicalOr, nil
case "special":
return RespSpecial, nil
default:
return RespDefaultKeyless, fmt.Errorf("routing: unknown response_policy %q", raw)
}
}
type CommandPolicy struct {
Request RequestPolicy
Response ResponsePolicy
// Tips that are not request_policy or response_policy
// e.g nondeterministic_output, nondeterministic_output_order.
Tips map[string]string
}
func (p *CommandPolicy) CanBeUsedInPipeline() bool {
return p.Request != ReqAllNodes && p.Request != ReqAllShards && p.Request != ReqMultiShard
}
func (p *CommandPolicy) IsReadOnly() bool {
_, readOnly := p.Tips[ReadOnlyCMD]
return readOnly
}
@@ -0,0 +1,57 @@
package routing
import (
"math/rand"
"sync/atomic"
)
// ShardPicker chooses “one arbitrary shard” when the request_policy is
// ReqDefault and the command has no keys.
type ShardPicker interface {
Next(total int) int // returns an index in [0,total)
}
// StaticShardPicker always returns the same shard index.
type StaticShardPicker struct {
index int
}
func NewStaticShardPicker(index int) *StaticShardPicker {
return &StaticShardPicker{index: index}
}
func (p *StaticShardPicker) Next(total int) int {
if total == 0 || p.index >= total {
return 0
}
return p.index
}
/*
Round-robin (default)
*/
type RoundRobinPicker struct {
cnt atomic.Uint32
}
func (p *RoundRobinPicker) Next(total int) int {
if total == 0 {
return 0
}
i := p.cnt.Add(1)
return int(i-1) % total
}
/*
Random
*/
type RandomPicker struct{}
func (RandomPicker) Next(total int) int {
if total == 0 {
return 0
}
return rand.Intn(total)
}
+1 -1
View File
@@ -190,4 +190,4 @@ func (s *FIFOSemaphore) Close() {
// Len returns the current number of acquired tokens.
func (s *FIFOSemaphore) Len() int32 {
return s.max - int32(len(s.tokens))
}
}
@@ -0,0 +1,97 @@
/*
© 2023present Harald Rudell <harald.rudell@gmail.com> (https://haraldrudell.github.io/haraldrudell/)
ISC License
Modified by htemelski-redis
Removed the treshold, adapted it to work with float64
*/
package util
import (
"math"
"go.uber.org/atomic"
)
// AtomicMax is a thread-safe max container
// - hasValue indicator true if a value was equal to or greater than threshold
// - optional threshold for minimum accepted max value
// - if threshold is not used, initialization-free
// - —
// - wait-free CompareAndSwap mechanic
type AtomicMax struct {
// value is current max
value atomic.Float64
// whether [AtomicMax.Value] has been invoked
// with value equal or greater to threshold
hasValue atomic.Bool
}
// NewAtomicMax returns a thread-safe max container
// - if threshold is not used, AtomicMax is initialization-free
func NewAtomicMax() (atomicMax *AtomicMax) {
m := AtomicMax{}
m.value.Store((-math.MaxFloat64))
return &m
}
// Value updates the container with a possible max value
// - isNewMax is true if:
// - — value is equal to or greater than any threshold and
// - — invocation recorded the first 0 or
// - — a new max
// - upon return, Max and Max1 are guaranteed to reflect the invocation
// - the return order of concurrent Value invocations is not guaranteed
// - Thread-safe
func (m *AtomicMax) Value(value float64) (isNewMax bool) {
// -math.MaxFloat64 as max case
var hasValue0 = m.hasValue.Load()
if value == (-math.MaxFloat64) {
if !hasValue0 {
isNewMax = m.hasValue.CompareAndSwap(false, true)
}
return // -math.MaxFloat64 as max: isNewMax true for first 0 writer
}
// check against present value
var current = m.value.Load()
if isNewMax = value > current; !isNewMax {
return // not a new max return: isNewMax false
}
// store the new max
for {
// try to write value to *max
if isNewMax = m.value.CompareAndSwap(current, value); isNewMax {
if !hasValue0 {
// may be rarely written multiple times
// still faster than CompareAndSwap
m.hasValue.Store(true)
}
return // new max written return: isNewMax true
}
if current = m.value.Load(); current >= value {
return // no longer a need to write return: isNewMax false
}
}
}
// Max returns current max and value-present flag
// - hasValue true indicates that value reflects a Value invocation
// - hasValue false: value is zero-value
// - Thread-safe
func (m *AtomicMax) Max() (value float64, hasValue bool) {
if hasValue = m.hasValue.Load(); !hasValue {
return
}
value = m.value.Load()
return
}
// Max1 returns current maximum whether zero-value or set by Value
// - threshold is ignored
// - Thread-safe
func (m *AtomicMax) Max1() (value float64) { return m.value.Load() }
@@ -0,0 +1,96 @@
package util
/*
© 2023present Harald Rudell <harald.rudell@gmail.com> (https://haraldrudell.github.io/haraldrudell/)
ISC License
Modified by htemelski-redis
Adapted from the modified atomic_max, but with inverted logic
*/
import (
"math"
"go.uber.org/atomic"
)
// AtomicMin is a thread-safe Min container
// - hasValue indicator true if a value was equal to or greater than threshold
// - optional threshold for minimum accepted Min value
// - —
// - wait-free CompareAndSwap mechanic
type AtomicMin struct {
// value is current Min
value atomic.Float64
// whether [AtomicMin.Value] has been invoked
// with value equal or greater to threshold
hasValue atomic.Bool
}
// NewAtomicMin returns a thread-safe Min container
// - if threshold is not used, AtomicMin is initialization-free
func NewAtomicMin() (atomicMin *AtomicMin) {
m := AtomicMin{}
m.value.Store(math.MaxFloat64)
return &m
}
// Value updates the container with a possible Min value
// - isNewMin is true if:
// - — value is equal to or greater than any threshold and
// - — invocation recorded the first 0 or
// - — a new Min
// - upon return, Min and Min1 are guaranteed to reflect the invocation
// - the return order of concurrent Value invocations is not guaranteed
// - Thread-safe
func (m *AtomicMin) Value(value float64) (isNewMin bool) {
// math.MaxFloat64 as Min case
var hasValue0 = m.hasValue.Load()
if value == math.MaxFloat64 {
if !hasValue0 {
isNewMin = m.hasValue.CompareAndSwap(false, true)
}
return // math.MaxFloat64 as Min: isNewMin true for first 0 writer
}
// check against present value
var current = m.value.Load()
if isNewMin = value < current; !isNewMin {
return // not a new Min return: isNewMin false
}
// store the new Min
for {
// try to write value to *Min
if isNewMin = m.value.CompareAndSwap(current, value); isNewMin {
if !hasValue0 {
// may be rarely written multiple times
// still faster than CompareAndSwap
m.hasValue.Store(true)
}
return // new Min written return: isNewMin true
}
if current = m.value.Load(); current <= value {
return // no longer a need to write return: isNewMin false
}
}
}
// Min returns current min and value-present flag
// - hasValue true indicates that value reflects a Value invocation
// - hasValue false: value is zero-value
// - Thread-safe
func (m *AtomicMin) Min() (value float64, hasValue bool) {
if hasValue = m.hasValue.Load(); !hasValue {
return
}
value = m.value.Load()
return
}
// Min1 returns current Minimum whether zero-value or set by Value
// - threshold is ignored
// - Thread-safe
func (m *AtomicMin) Min1() (value float64) { return m.value.Load() }
-17
View File
@@ -1,17 +0,0 @@
package util
// Max returns the maximum of two integers
func Max(a, b int) int {
if a > b {
return a
}
return b
}
// Min returns the minimum of two integers
func Min(a, b int) int {
if a < b {
return a
}
return b
}
+2 -7
View File
@@ -8,15 +8,10 @@ import (
// BytesToString converts byte slice to string.
func BytesToString(b []byte) string {
return *(*string)(unsafe.Pointer(&b))
return unsafe.String(unsafe.SliceData(b), len(b))
}
// StringToBytes converts string to byte slice.
func StringToBytes(s string) []byte {
return *(*[]byte)(unsafe.Pointer(
&struct {
string
Cap int
}{s, len(s)},
))
return unsafe.Slice(unsafe.StringData(s), len(s))
}
+41 -6
View File
@@ -68,8 +68,9 @@ var _ Cmder = (*JSONCmd)(nil)
func newJSONCmd(ctx context.Context, args ...interface{}) *JSONCmd {
return &JSONCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeJSON,
},
}
}
@@ -165,6 +166,14 @@ func (cmd *JSONCmd) readReply(rd *proto.Reader) error {
return nil
}
func (cmd *JSONCmd) Clone() Cmder {
return &JSONCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val,
expanded: cmd.expanded, // interface{} can be shared as it should be immutable after parsing
}
}
// -------------------------------------------
type JSONSliceCmd struct {
@@ -175,8 +184,9 @@ type JSONSliceCmd struct {
func NewJSONSliceCmd(ctx context.Context, args ...interface{}) *JSONSliceCmd {
return &JSONSliceCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeJSONSlice,
},
}
}
@@ -233,6 +243,18 @@ func (cmd *JSONSliceCmd) readReply(rd *proto.Reader) error {
return nil
}
func (cmd *JSONSliceCmd) Clone() Cmder {
var val []interface{}
if cmd.val != nil {
val = make([]interface{}, len(cmd.val))
copy(val, cmd.val)
}
return &JSONSliceCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
/*******************************************************************************
*
* IntPointerSliceCmd
@@ -249,8 +271,9 @@ type IntPointerSliceCmd struct {
func NewIntPointerSliceCmd(ctx context.Context, args ...interface{}) *IntPointerSliceCmd {
return &IntPointerSliceCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeIntPointerSlice,
},
}
}
@@ -290,6 +313,18 @@ func (cmd *IntPointerSliceCmd) readReply(rd *proto.Reader) error {
return nil
}
func (cmd *IntPointerSliceCmd) Clone() Cmder {
var val []*int64
if cmd.val != nil {
val = make([]*int64, len(cmd.val))
copy(val, cmd.val)
}
return &IntPointerSliceCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
//------------------------------------------------------------------------------
// JSONArrAppend adds the provided JSON values to the end of the array at the given path.
+8
View File
@@ -77,6 +77,10 @@ func (c cmdable) BRPop(ctx context.Context, timeout time.Duration, keys ...strin
return cmd
}
// BRPopLPush pops an element from a list, pushes it to another list and returns it.
// Blocks until an element is available or timeout is reached.
//
// Deprecated: Use BLMove with RIGHT and LEFT arguments instead as of Redis 6.2.0.
func (c cmdable) BRPopLPush(ctx context.Context, source, destination string, timeout time.Duration) *StringCmd {
cmd := NewStringCmd(
ctx,
@@ -247,6 +251,10 @@ func (c cmdable) RPopCount(ctx context.Context, key string, count int) *StringSl
return cmd
}
// RPopLPush atomically returns and removes the last element of the source list,
// and pushes the element as the first element of the destination list.
//
// Deprecated: Use LMove with RIGHT and LEFT arguments instead as of Redis 6.2.0.
func (c cmdable) RPopLPush(ctx context.Context, source, destination string) *StringCmd {
cmd := NewStringCmd(ctx, "rpoplpush", source, destination)
_ = c(ctx, cmd)
+29 -12
View File
@@ -156,18 +156,16 @@ Capped by: min(MaxActiveConns + 1, 5 × PoolSize)
### Client Support
#### Currently Supported
- **Standalone Client** (`redis.NewClient`)
- **Standalone Client** (`redis.NewClient`) - Full support for MOVING, MIGRATING, MIGRATED, FAILING_OVER, FAILED_OVER notifications
- **Cluster Client** (`redis.NewClusterClient`) - Support for SMIGRATING and SMIGRATED notifications for hitless slot migrations
#### Planned Support
- **Cluster Client** (not yet supported)
#### Will Not Support
- **Failover Client** (no planned support)
- **Ring Client** (no planned support)
## Migration Guide
### Enabling Maintenance Notifications
### Enabling Maintenance Notifications (Standalone Client)
**Before:**
```go
@@ -188,6 +186,26 @@ client := redis.NewClient(&redis.Options{
})
```
### Enabling Hitless Upgrades (Cluster Client)
For Redis Cluster with hitless slot migration support:
```go
client := redis.NewClusterClient(&redis.ClusterOptions{
Addrs: []string{"localhost:7000", "localhost:7001", "localhost:7002"},
Protocol: 3, // RESP3 required for push notifications
MaintNotificationsConfig: &maintnotifications.Config{
Mode: maintnotifications.ModeAuto,
RelaxedTimeout: 10 * time.Second, // Extended timeout during slot migrations
},
})
```
The cluster client automatically handles:
- **SMIGRATING**: Relaxes timeouts when slots are being migrated
- **SMIGRATED**: Triggers lazy cluster state reload when migration completes
- **SeqID Deduplication**: Same notification from multiple nodes triggers only one reload
### Adding Monitoring
```go
@@ -206,13 +224,12 @@ if manager != nil {
## Known Limitations
1. **Standalone Only**: Currently only supported in standalone Redis clients
2. **RESP3 Required**: Push notifications require RESP3 protocol
3. **Server Support**: Requires Redis Enterprise or compatible Redis with maintenance notifications
4. **Single Connection Commands**: Some commands (MULTI/EXEC, WATCH) may need special handling
5. **No Failover/Ring Client Support**: Failover and Ring clients are not supported and there are no plans to add support
1. **RESP3 Required**: Push notifications require RESP3 protocol
2. **Server Support**: Requires Redis Enterprise or compatible Redis with maintenance notifications
3. **Single Connection Commands**: Some commands (MULTI/EXEC, WATCH) may need special handling
4. **No Failover/Ring Client Support**: Failover and Ring clients are not supported and there are no plans to add support
## Future Enhancements
- Cluster client support
- Enhanced metrics and observability
- Enhanced metrics and observability
- TTL-based cleanup for SeqID deduplication map
+8 -2
View File
@@ -2,8 +2,14 @@
Seamless Redis connection handoffs during cluster maintenance operations without dropping connections.
## ⚠️ **Important Note**
**Maintenance notifications are currently supported only in standalone Redis clients.** Cluster clients (ClusterClient, FailoverClient, etc.) do not yet support this functionality.
## Cluster Support
**Cluster notifications are now supported for ClusterClient!**
- **SMIGRATING**: `["SMIGRATING", SeqID, slot/range, ...]` - Relaxes timeouts when slots are being migrated
- **SMIGRATED**: `["SMIGRATED", SeqID, src host:port, dst host:port, slot/range, ...]` - Reloads cluster state when slot migration completes
**Note:** Other maintenance notifications (MOVING, MIGRATING, MIGRATED, FAILING_OVER, FAILED_OVER) are supported only in standalone Redis clients. Cluster clients support SMIGRATING and SMIGRATED for cluster-specific slot migration handling.
## Quick Start
+5 -6
View File
@@ -9,7 +9,6 @@ import (
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/maintnotifications/logs"
"github.com/redis/go-redis/v9/internal/util"
)
// Mode represents the maintenance notifications mode
@@ -261,10 +260,10 @@ func (c *Config) ApplyDefaultsWithPoolConfig(poolSize int, maxActiveConns int) *
// Default: max(20x workers, PoolSize), capped by maxActiveConns or 5x pool size
workerBasedSize := result.MaxWorkers * 20
poolBasedSize := poolSize
result.HandoffQueueSize = util.Max(workerBasedSize, poolBasedSize)
result.HandoffQueueSize = max(workerBasedSize, poolBasedSize)
if c.HandoffQueueSize > 0 {
// When explicitly set: enforce minimum of 200
result.HandoffQueueSize = util.Max(200, c.HandoffQueueSize)
result.HandoffQueueSize = max(200, c.HandoffQueueSize)
}
// Cap queue size: use maxActiveConns+1 if set, otherwise 5x pool size
@@ -278,7 +277,7 @@ func (c *Config) ApplyDefaultsWithPoolConfig(poolSize int, maxActiveConns int) *
} else {
queueCap = poolSize * 5
}
result.HandoffQueueSize = util.Min(result.HandoffQueueSize, queueCap)
result.HandoffQueueSize = min(result.HandoffQueueSize, queueCap)
// Ensure minimum queue size of 2 (fallback for very small pools)
if result.HandoffQueueSize < 2 {
@@ -353,10 +352,10 @@ func (c *Config) applyWorkerDefaults(poolSize int) {
// When not set: min(poolSize/2, max(10, poolSize/3)) - balanced scaling approach
originalMaxWorkers := c.MaxWorkers
c.MaxWorkers = util.Min(poolSize/2, util.Max(10, poolSize/3))
c.MaxWorkers = min(poolSize/2, max(10, poolSize/3))
if originalMaxWorkers != 0 {
// When explicitly set: max(poolSize/2, set_value) - ensure at least poolSize/2 workers
c.MaxWorkers = util.Max(poolSize/2, originalMaxWorkers)
c.MaxWorkers = max(poolSize/2, originalMaxWorkers)
}
// Ensure minimum of 1 worker (fallback for very small pools)
@@ -13,6 +13,9 @@ import (
"github.com/redis/go-redis/v9/internal/pool"
)
// PoolNameMain is the name used for the main connection pool in metrics.
const PoolNameMain = "main"
// handoffWorkerManager manages background workers and queue for connection handoffs
type handoffWorkerManager struct {
// Event-driven handoff support
@@ -434,6 +437,11 @@ func (hwm *handoffWorkerManager) performHandoffInternal(
deadline := time.Now().Add(hwm.config.PostHandoffRelaxedDuration)
conn.SetRelaxedTimeoutWithDeadline(relaxedTimeout, relaxedTimeout, deadline)
// Record relaxed timeout metric (post-handoff)
if relaxedTimeoutCallback := pool.GetMetricConnectionRelaxedTimeoutCallback(); relaxedTimeoutCallback != nil {
relaxedTimeoutCallback(ctx, 1, conn, PoolNameMain, "HANDOFF")
}
if internal.LogLevel.InfoOrAbove() {
internal.Logger.Printf(context.Background(), logs.ApplyingRelaxedTimeoutDueToPostHandoff(connID, relaxedTimeout, deadline.Format("15:04:05.000")))
}
@@ -462,6 +470,11 @@ func (hwm *handoffWorkerManager) performHandoffInternal(
internal.Logger.Printf(ctx, logs.HandoffSucceeded(connID, newEndpoint))
// successfully completed the handoff, no retry needed and no error
// Notify metrics: connection handoff succeeded
if handoffCallback := pool.GetMetricConnectionHandoffCallback(); handoffCallback != nil {
handoffCallback(ctx, conn, PoolNameMain)
}
return false, nil
}
@@ -501,9 +514,9 @@ func (hwm *handoffWorkerManager) closeConnFromRequest(ctx context.Context, reque
internal.Logger.Printf(ctx, logs.RemovingConnectionFromPool(conn.GetID(), err))
}
} else {
err := conn.Close() // Close the connection if no pool provided
if err != nil {
internal.Logger.Printf(ctx, "redis: failed to close connection: %v", err)
errClose := conn.Close() // Close the connection if no pool provided
if errClose != nil {
internal.Logger.Printf(ctx, "redis: failed to close connection: %v", errClose)
}
if internal.LogLevel.WarnOrAbove() {
internal.Logger.Printf(ctx, logs.NoPoolProvidedCannotRemove(conn.GetID(), err))
+47 -5
View File
@@ -18,11 +18,13 @@ import (
// Push notification type constants for maintenance
const (
NotificationMoving = "MOVING"
NotificationMigrating = "MIGRATING"
NotificationMigrated = "MIGRATED"
NotificationFailingOver = "FAILING_OVER"
NotificationFailedOver = "FAILED_OVER"
NotificationMoving = "MOVING" // Per-connection handoff notification
NotificationMigrating = "MIGRATING" // Per-connection migration start notification - relaxes timeouts
NotificationMigrated = "MIGRATED" // Per-connection migration complete notification - clears relaxed timeouts
NotificationFailingOver = "FAILING_OVER" // Per-connection failover start notification - relaxes timeouts
NotificationFailedOver = "FAILED_OVER" // Per-connection failover complete notification - clears relaxed timeouts
NotificationSMigrating = "SMIGRATING" // Cluster slot migrating notification - relaxes timeouts
NotificationSMigrated = "SMIGRATED" // Cluster slot migrated notification - unrelaxes timeouts and triggers cluster state reload
)
// maintenanceNotificationTypes contains all notification types that maintenance handles
@@ -32,6 +34,8 @@ var maintenanceNotificationTypes = []string{
NotificationMigrated,
NotificationFailingOver,
NotificationFailedOver,
NotificationSMigrating,
NotificationSMigrated,
}
// NotificationHook is called before and after notification processing
@@ -65,6 +69,10 @@ type Manager struct {
// MOVING operation tracking - using sync.Map for better concurrent performance
activeMovingOps sync.Map // map[MovingOperationKey]*MovingOperation
// SMIGRATED notification deduplication - tracks processed SeqIDs
// Multiple connections may receive the same SMIGRATED notification
processedSMigratedSeqIDs sync.Map // map[int64]bool
// Atomic state tracking - no locks needed for state queries
activeOperationCount atomic.Int64 // Number of active operations
closed atomic.Bool // Manager closed state
@@ -73,6 +81,9 @@ type Manager struct {
hooks []NotificationHook
hooksMu sync.RWMutex // Protects hooks slice
poolHooksRef *PoolHook
// Cluster state reload callback for SMIGRATED notifications
clusterStateReloadCallback ClusterStateReloadCallback
}
// MovingOperation tracks an active MOVING operation.
@@ -83,6 +94,14 @@ type MovingOperation struct {
Deadline time.Time
}
// ClusterStateReloadCallback is a callback function that triggers cluster state reload.
// This is used by node clients to notify their parent ClusterClient about SMIGRATED notifications.
// The hostPort parameter indicates the destination node (e.g., "127.0.0.1:6379").
// The slotRanges parameter contains the migrated slots (e.g., ["1234", "5000-6000"]).
// Currently, implementations typically reload the entire cluster state, but in the future
// this could be optimized to reload only the specific slots.
type ClusterStateReloadCallback func(ctx context.Context, hostPort string, slotRanges []string)
// NewManager creates a new simplified manager.
func NewManager(client interfaces.ClientInterface, pool pool.Pooler, config *Config) (*Manager, error) {
if client == nil {
@@ -223,6 +242,15 @@ func (hm *Manager) GetActiveOperationCount() int64 {
return hm.activeOperationCount.Load()
}
// MarkSMigratedSeqIDProcessed attempts to mark a SMIGRATED SeqID as processed.
// Returns true if this is the first time processing this SeqID (should process),
// false if it was already processed (should skip).
// This prevents duplicate processing when multiple connections receive the same notification.
func (hm *Manager) MarkSMigratedSeqIDProcessed(seqID int64) bool {
_, alreadyProcessed := hm.processedSMigratedSeqIDs.LoadOrStore(seqID, true)
return !alreadyProcessed // Return true if NOT already processed
}
// Close closes the manager.
func (hm *Manager) Close() error {
// Use atomic operation for thread-safe close check
@@ -318,3 +346,17 @@ func (hm *Manager) AddNotificationHook(notificationHook NotificationHook) {
defer hm.hooksMu.Unlock()
hm.hooks = append(hm.hooks, notificationHook)
}
// SetClusterStateReloadCallback sets the callback function that will be called when a SMIGRATED notification is received.
// This allows node clients to notify their parent ClusterClient to reload cluster state.
func (hm *Manager) SetClusterStateReloadCallback(callback ClusterStateReloadCallback) {
hm.clusterStateReloadCallback = callback
}
// TriggerClusterStateReload calls the cluster state reload callback if it's set.
// This is called when a SMIGRATED notification is received.
func (hm *Manager) TriggerClusterStateReload(ctx context.Context, hostPort string, slotRanges []string) {
if hm.clusterStateReloadCallback != nil {
hm.clusterStateReloadCallback(ctx, hostPort, slotRanges)
}
}
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/redis/go-redis/v9/internal"
@@ -49,11 +50,22 @@ func (snh *NotificationHandler) HandlePushNotification(ctx context.Context, hand
err = snh.handleFailingOver(ctx, handlerCtx, modifiedNotification)
case NotificationFailedOver:
err = snh.handleFailedOver(ctx, handlerCtx, modifiedNotification)
case NotificationSMigrating:
err = snh.handleSMigrating(ctx, handlerCtx, modifiedNotification)
case NotificationSMigrated:
err = snh.handleSMigrated(ctx, handlerCtx, modifiedNotification)
default:
// Ignore other notification types (e.g., pub/sub messages)
err = nil
}
// Record maintenance notification metric
if maintenanceCallback := pool.GetMetricMaintenanceNotificationCallback(); maintenanceCallback != nil {
if conn, ok := handlerCtx.Conn.(*pool.Conn); ok {
maintenanceCallback(ctx, conn, notificationType)
}
}
// Process post-hooks with the result
snh.manager.processPostHooks(ctx, handlerCtx, notificationType, modifiedNotification, err)
@@ -61,7 +73,9 @@ func (snh *NotificationHandler) HandlePushNotification(ctx context.Context, hand
}
// handleMoving processes MOVING notifications.
// ["MOVING", seqNum, timeS, endpoint] - per-connection handoff
// MOVING indicates that a connection should be handed off to a new endpoint.
// This is a per-connection notification that triggers connection handoff.
// Expected format: ["MOVING", seqNum, timeS, endpoint]
func (snh *NotificationHandler) handleMoving(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
if len(notification) < 3 {
internal.Logger.Printf(ctx, logs.InvalidNotification("MOVING", notification))
@@ -140,7 +154,28 @@ func (snh *NotificationHandler) handleMoving(ctx context.Context, handlerCtx pus
if err := snh.markConnForHandoff(poolConn, newEndpoint, seqID, deadline); err != nil {
// Log error but don't fail the goroutine - use background context since original may be cancelled
internal.Logger.Printf(context.Background(), logs.FailedToMarkForHandoff(poolConn.GetID(), err))
return
}
// Queue the handoff immediately if the connection is idle in the pool.
// If the connection is in use (StateInUse), it will be queued when returned to the pool via OnPut.
// This handles the case where the connection is idle and might never be retrieved again.
if poolConn.GetStateMachine().GetState() == pool.StateIdle {
if snh.manager.poolHooksRef != nil && snh.manager.poolHooksRef.workerManager != nil {
if err := snh.manager.poolHooksRef.workerManager.queueHandoff(poolConn); err != nil {
internal.Logger.Printf(context.Background(), logs.FailedToQueueHandoff(poolConn.GetID(), err))
} else {
// Mark the connection as queued for handoff to prevent it from being retrieved
// This transitions the connection to StateUnusable
if err := poolConn.MarkQueuedForHandoff(); err != nil {
internal.Logger.Printf(context.Background(), logs.FailedToMarkForHandoff(poolConn.GetID(), err))
} else {
internal.Logger.Printf(context.Background(), logs.MarkedForHandoff(poolConn.GetID()))
}
}
}
}
// If connection is StateInUse, the handoff will be queued when it's returned to the pool
})
return nil
}
@@ -167,9 +202,10 @@ func (snh *NotificationHandler) markConnForHandoff(conn *pool.Conn, newEndpoint
}
// handleMigrating processes MIGRATING notifications.
// MIGRATING indicates that a connection migration is starting.
// This is a per-connection notification that applies relaxed timeouts.
// Expected format: ["MIGRATING", ...]
func (snh *NotificationHandler) handleMigrating(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
// MIGRATING notifications indicate that a connection is about to be migrated
// Apply relaxed timeouts to the specific connection that received this notification
if len(notification) < 2 {
internal.Logger.Printf(ctx, logs.InvalidNotification("MIGRATING", notification))
return ErrInvalidNotification
@@ -191,13 +227,20 @@ func (snh *NotificationHandler) handleMigrating(ctx context.Context, handlerCtx
internal.Logger.Printf(ctx, logs.RelaxedTimeoutDueToNotification(conn.GetID(), "MIGRATING", snh.manager.config.RelaxedTimeout))
}
conn.SetRelaxedTimeout(snh.manager.config.RelaxedTimeout, snh.manager.config.RelaxedTimeout)
// Record relaxed timeout metric
if relaxedTimeoutCallback := pool.GetMetricConnectionRelaxedTimeoutCallback(); relaxedTimeoutCallback != nil {
relaxedTimeoutCallback(ctx, 1, conn, PoolNameMain, "MIGRATING")
}
return nil
}
// handleMigrated processes MIGRATED notifications.
// MIGRATED indicates that a connection migration has completed.
// This is a per-connection notification that clears relaxed timeouts.
// Expected format: ["MIGRATED", ...]
func (snh *NotificationHandler) handleMigrated(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
// MIGRATED notifications indicate that a connection migration has completed
// Restore normal timeouts for the specific connection that received this notification
if len(notification) < 2 {
internal.Logger.Printf(ctx, logs.InvalidNotification("MIGRATED", notification))
return ErrInvalidNotification
@@ -224,9 +267,10 @@ func (snh *NotificationHandler) handleMigrated(ctx context.Context, handlerCtx p
}
// handleFailingOver processes FAILING_OVER notifications.
// FAILING_OVER indicates that a failover is starting.
// This is a per-connection notification that applies relaxed timeouts.
// Expected format: ["FAILING_OVER", ...]
func (snh *NotificationHandler) handleFailingOver(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
// FAILING_OVER notifications indicate that a connection is about to failover
// Apply relaxed timeouts to the specific connection that received this notification
if len(notification) < 2 {
internal.Logger.Printf(ctx, logs.InvalidNotification("FAILING_OVER", notification))
return ErrInvalidNotification
@@ -249,13 +293,20 @@ func (snh *NotificationHandler) handleFailingOver(ctx context.Context, handlerCt
internal.Logger.Printf(ctx, logs.RelaxedTimeoutDueToNotification(connID, "FAILING_OVER", snh.manager.config.RelaxedTimeout))
}
conn.SetRelaxedTimeout(snh.manager.config.RelaxedTimeout, snh.manager.config.RelaxedTimeout)
// Record relaxed timeout metric
if relaxedTimeoutCallback := pool.GetMetricConnectionRelaxedTimeoutCallback(); relaxedTimeoutCallback != nil {
relaxedTimeoutCallback(ctx, 1, conn, PoolNameMain, "FAILING_OVER")
}
return nil
}
// handleFailedOver processes FAILED_OVER notifications.
// FAILED_OVER indicates that a failover has completed.
// This is a per-connection notification that clears relaxed timeouts.
// Expected format: ["FAILED_OVER", ...]
func (snh *NotificationHandler) handleFailedOver(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
// FAILED_OVER notifications indicate that a connection failover has completed
// Restore normal timeouts for the specific connection that received this notification
if len(notification) < 2 {
internal.Logger.Printf(ctx, logs.InvalidNotification("FAILED_OVER", notification))
return ErrInvalidNotification
@@ -280,3 +331,194 @@ func (snh *NotificationHandler) handleFailedOver(ctx context.Context, handlerCtx
conn.ClearRelaxedTimeout()
return nil
}
// handleSMigrating processes SMIGRATING notifications.
// SMIGRATING indicates that a cluster slot is in the process of migrating to a different node.
// This is a per-connection notification that applies relaxed timeouts during slot migration.
// Expected format: ["SMIGRATING", SeqID, slot/range1-range2, ...]
func (snh *NotificationHandler) handleSMigrating(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
if len(notification) < 3 {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATING", notification))
return ErrInvalidNotification
}
// Validate SeqID (position 1)
if _, ok := notification[1].(int64); !ok {
internal.Logger.Printf(ctx, logs.InvalidSeqIDInSMigratingNotification(notification[1]))
return ErrInvalidNotification
}
if handlerCtx.Conn == nil {
internal.Logger.Printf(ctx, logs.NoConnectionInHandlerContext("SMIGRATING"))
return ErrInvalidNotification
}
conn, ok := handlerCtx.Conn.(*pool.Conn)
if !ok {
internal.Logger.Printf(ctx, logs.InvalidConnectionTypeInHandlerContext("SMIGRATING", handlerCtx.Conn, handlerCtx))
return ErrInvalidNotification
}
// Apply relaxed timeout to this specific connection
if internal.LogLevel.InfoOrAbove() {
internal.Logger.Printf(ctx, logs.RelaxedTimeoutDueToNotification(conn.GetID(), "SMIGRATING", snh.manager.config.RelaxedTimeout))
}
conn.SetRelaxedTimeout(snh.manager.config.RelaxedTimeout, snh.manager.config.RelaxedTimeout)
return nil
}
// handleSMigrated processes SMIGRATED notifications.
// SMIGRATED indicates that a cluster slot has finished migrating to a different node.
// This is a cluster-level notification that triggers cluster state reload.
//
// Expected RESP3 format:
//
// >3
// +SMIGRATED
// :SeqID
// *<num_entries> <- array of triplet arrays
// *3 <- each triplet is a 3-element array
// +<source> <- node from which slots are migrating FROM
// +<destination> <- node to which slots are migrating TO
// +<slots> <- comma-separated slots and/or ranges (e.g., "123,789-1000")
//
// A source and target endpoint may appear in multiple triplets.
// The notification is only processed if the connection's NodeAddress matches one of the source endpoints.
//
// Note: Multiple connections may receive the same notification, so we deduplicate by SeqID before triggering reload.
// but we still process the notification on each connection to clear the relaxed timeout.
// In the case when the connection is from MOVED/ASK, the connection's original endpoint is not set,
// so we will not be able to match the source endpoint. In such case, we will trigger the reload callback with the first target endpoint.
func (snh *NotificationHandler) handleSMigrated(ctx context.Context, handlerCtx push.NotificationHandlerContext, notification []interface{}) error {
// Expected: ["SMIGRATED", SeqID, [[source, target, slots], ...]]
// Minimum 3 elements: SMIGRATED, SeqID, and the array of triplets
if len(notification) < 3 {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED", notification))
return ErrInvalidNotification
}
// Extract SeqID (position 1)
seqID, ok := notification[1].(int64)
if !ok {
internal.Logger.Printf(ctx, logs.InvalidSeqIDInSMigratedNotification(notification[1]))
return ErrInvalidNotification
}
// Extract the array of triplets (position 2)
triplets, ok := notification[2].([]interface{})
if !ok {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED (triplets array)", notification[2]))
return ErrInvalidNotification
}
if len(triplets) == 0 {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED (empty triplets)", notification))
return ErrInvalidNotification
}
// Get the connection's endpoints to check if this notification is relevant
// We check against both nodeAddress (from CLUSTER SLOTS) and addr (after resolution)
// since we cannot be certain which format the notification source will use
var connectionNodeAddress string
var connectionAddr string
if snh.manager.options != nil {
connectionNodeAddress = snh.manager.options.GetNodeAddress()
connectionAddr = snh.manager.options.GetAddr()
}
// Helper function to check if source matches either of our endpoints
// notification source can be either the node address or the addr after resolution
sourceMatchesConnection := func(source string) bool {
if source == connectionNodeAddress {
return true
}
if source == connectionAddr {
return true
}
return false
}
// Parse triplets and check if any source matches our connection's endpoints
var matchingTriplets []struct {
source string
target string
slots string
}
var allSlotRanges []string
for _, tripletInterface := range triplets {
// Each triplet should be a 3-element array: [source, target, slots]
triplet, ok := tripletInterface.([]interface{})
if !ok || len(triplet) != 3 {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED (triplet format)", tripletInterface))
continue
}
// Extract source endpoint
source, ok := triplet[0].(string)
if !ok {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED (source)", triplet[0]))
continue
}
// Extract target endpoint
target, ok := triplet[1].(string)
if !ok {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED (target)", triplet[1]))
continue
}
// Extract slots
slots, ok := triplet[2].(string)
if !ok {
internal.Logger.Printf(ctx, logs.InvalidNotification("SMIGRATED (slots)", triplet[2]))
continue
}
// Check if this triplet's source matches our connection's endpoints
if sourceMatchesConnection(source) {
matchingTriplets = append(matchingTriplets, struct {
source string
target string
slots string
}{source, target, slots})
slotRanges := strings.Split(slots, ",")
allSlotRanges = append(allSlotRanges, slotRanges...)
}
}
var connID uint64
// Reset relaxed timeout for this specific connection
if handlerCtx.Conn != nil {
conn, ok := handlerCtx.Conn.(*pool.Conn)
if ok {
if internal.LogLevel.InfoOrAbove() {
connID = conn.GetID()
internal.Logger.Printf(ctx, logs.UnrelaxedTimeout(connID))
}
conn.ClearRelaxedTimeout()
}
}
// If no matching triplets, this notification is not relevant to this connection
if len(matchingTriplets) == 0 {
return nil
}
// Deduplicate by SeqID - multiple connections may receive the same notification
// Only trigger cluster state reload once per seqID
if snh.manager.MarkSMigratedSeqIDProcessed(seqID) {
// Use the first matching triplet
target := matchingTriplets[0].target
slotsForLog := allSlotRanges
if internal.LogLevel.InfoOrAbove() {
internal.Logger.Printf(ctx, logs.TriggeringClusterStateReload(seqID, target, slotsForLog))
}
// Trigger cluster state reload via callback
snh.manager.TriggerClusterStateReload(ctx, target, slotsForLog)
}
return nil
}
+57 -1
View File
@@ -11,6 +11,7 @@ import (
"sort"
"strconv"
"strings"
"sync/atomic"
"time"
"github.com/redis/go-redis/v9/auth"
@@ -21,6 +22,16 @@ import (
"github.com/redis/go-redis/v9/push"
)
// poolIDCounter is a global auto-increment counter for generating unique pool IDs.
var poolIDCounter atomic.Uint64
// generateUniqueID generates a short unique identifier for pool names using auto-increment.
// This makes it easier to identify and track pools in order of creation.
func generateUniqueID() string {
id := poolIDCounter.Add(1)
return strconv.FormatUint(id, 10)
}
// Limiter is the interface of a rate limiter or a circuit breaker.
type Limiter interface {
// Allow returns nil if operation is allowed or an error otherwise.
@@ -42,6 +53,17 @@ type Options struct {
// Addr is the address formated as host:port
Addr string
// NodeAddress is the address of the Redis node as reported by the server.
// For cluster clients, this is the exact endpoint string returned by CLUSTER SLOTS
// before any resolution or transformation (e.g., loopback replacement).
// For standalone clients, this defaults to Addr.
//
// This is used to match the source endpoint in maintenance notifications
// (e.g. SMIGRATED).
//
// Use Client.NodeAddress() to access this value.
NodeAddress string
// ClientName will execute the `CLIENT SETNAME ClientName` command for each conn.
ClientName string
@@ -200,6 +222,8 @@ type Options struct {
// MaxActiveConns is the maximum number of connections allocated by the pool at a given time.
// When zero, there is no limit on the number of connections in the pool.
// If the pool is full, the next call to Get() will block until a connection is released.
//
// default: 0
MaxActiveConns int
// ConnMaxIdleTime is the maximum amount of time a connection may be idle.
@@ -220,6 +244,19 @@ type Options struct {
// default: 0
ConnMaxLifetime time.Duration
// ConnMaxLifetimeJitter is the absolute jitter duration applied to ConnMaxLifetime
// to prevent all connections from expiring simultaneously.
//
// The jitter is applied as a random offset in the range [-jitter, +jitter].
// For example, if ConnMaxLifetime is 1 hour and ConnMaxLifetimeJitter is 6 minutes,
// connections will expire between 54 minutes and 66 minutes.
//
// If <= 0, no jitter is applied.
// If > ConnMaxLifetime, it will be capped at ConnMaxLifetime.
//
// default: 0
ConnMaxLifetimeJitter time.Duration
// TLSConfig to use. When set, TLS will be negotiated.
TLSConfig *tls.Config
@@ -280,6 +317,12 @@ func (opt *Options) init() {
opt.Network = "tcp"
}
}
// For standalone clients, default NodeAddress to Addr if not set.
// This ensures maintenance notifications (SMIGRATED, etc.) can match
// the connection's endpoint even for non-cluster clients.
if opt.NodeAddress == "" {
opt.NodeAddress = opt.Addr
}
if opt.Protocol < 2 {
opt.Protocol = 3
}
@@ -336,6 +379,8 @@ func (opt *Options) init() {
opt.ConnMaxIdleTime = 30 * time.Minute
}
opt.ConnMaxLifetimeJitter = min(opt.ConnMaxLifetimeJitter, opt.ConnMaxLifetime)
switch opt.MaxRetries {
case -1:
opt.MaxRetries = 0
@@ -645,6 +690,9 @@ func setupConnParams(u *url.URL, o *Options) (*Options, error) {
} else {
o.ConnMaxLifetime = q.duration("max_conn_age")
}
if q.has("conn_max_lifetime_jitter") {
o.ConnMaxLifetimeJitter = min(q.duration("conn_max_lifetime_jitter"), o.ConnMaxLifetime)
}
if q.err != nil {
return nil, q.err
}
@@ -674,6 +722,7 @@ func getUserPassword(u *url.URL) (string, string) {
func newConnPool(
opt *Options,
dialer func(ctx context.Context, network, addr string) (net.Conn, error),
poolName string,
) (*pool.ConnPool, error) {
poolSize, err := util.SafeIntToInt32(opt.PoolSize, "PoolSize")
if err != nil {
@@ -711,13 +760,18 @@ func newConnPool(
MaxActiveConns: maxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ConnMaxLifetimeJitter: opt.ConnMaxLifetimeJitter,
ReadBufferSize: opt.ReadBufferSize,
WriteBufferSize: opt.WriteBufferSize,
PushNotificationsEnabled: opt.Protocol == 3,
Name: poolName,
}), nil
}
func newPubSubPool(opt *Options, dialer func(ctx context.Context, network, addr string) (net.Conn, error),
func newPubSubPool(
opt *Options,
dialer func(ctx context.Context, network, addr string) (net.Conn, error),
poolName string,
) (*pool.PubSubPool, error) {
poolSize, err := util.SafeIntToInt32(opt.PoolSize, "PoolSize")
if err != nil {
@@ -752,8 +806,10 @@ func newPubSubPool(opt *Options, dialer func(ctx context.Context, network, addr
MaxActiveConns: maxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ConnMaxLifetimeJitter: opt.ConnMaxLifetimeJitter,
ReadBufferSize: 32 * 1024,
WriteBufferSize: 32 * 1024,
PushNotificationsEnabled: opt.Protocol == 3,
Name: poolName,
}, dialer), nil
}
+406 -73
View File
@@ -3,6 +3,7 @@ package redis
import (
"context"
"crypto/tls"
"errors"
"fmt"
"math"
"net"
@@ -17,9 +18,11 @@ import (
"github.com/redis/go-redis/v9/auth"
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/hashtag"
"github.com/redis/go-redis/v9/internal/otel"
"github.com/redis/go-redis/v9/internal/pool"
"github.com/redis/go-redis/v9/internal/proto"
"github.com/redis/go-redis/v9/internal/rand"
"github.com/redis/go-redis/v9/internal/routing"
"github.com/redis/go-redis/v9/maintnotifications"
"github.com/redis/go-redis/v9/push"
)
@@ -28,7 +31,11 @@ const (
minLatencyMeasurementInterval = 10 * time.Second
)
var errClusterNoNodes = fmt.Errorf("redis: cluster has no nodes")
var (
errClusterNoNodes = errors.New("redis: cluster has no nodes")
errNoWatchKeys = errors.New("redis: Watch requires at least one key")
errWatchCrosslot = errors.New("redis: Watch requires all keys to be in the same slot")
)
// ClusterOptions are used to configure a cluster client and should be
// passed to NewClusterClient.
@@ -85,19 +92,35 @@ type ClusterOptions struct {
MinRetryBackoff time.Duration
MaxRetryBackoff time.Duration
DialTimeout time.Duration
DialTimeout time.Duration
// DialerRetries is the maximum number of retry attempts when dialing fails.
//
// default: 5
DialerRetries int
// DialerRetryTimeout is the backoff duration between retry attempts.
//
// default: 100 milliseconds
DialerRetryTimeout time.Duration
ReadTimeout time.Duration
WriteTimeout time.Duration
ContextTimeoutEnabled bool
PoolFIFO bool
PoolSize int // applies per cluster node and not for the whole cluster
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int // applies per cluster node and not for the whole cluster
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
// MaxConcurrentDials is the maximum number of concurrent connection creation goroutines.
// If <= 0, defaults to PoolSize. If > PoolSize, it will be capped at PoolSize.
MaxConcurrentDials int
PoolFIFO bool
PoolSize int // applies per cluster node and not for the whole cluster
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int // applies per cluster node and not for the whole cluster
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
ConnMaxLifetimeJitter time.Duration
// ReadBufferSize is the size of the bufio.Reader buffer for each connection.
// Larger buffers can improve performance for commands that return large responses.
@@ -115,6 +138,11 @@ type ClusterOptions struct {
TLSConfig *tls.Config
// DisableRoutingPolicies disables the request/response policy routing system.
// When disabled, all commands use the legacy routing behavior.
// Experimental. Will be removed when shard picker is fully implemented.
DisableRoutingPolicies bool
// DisableIndentity - Disable set-lib on connect.
//
// default: false
@@ -146,8 +174,16 @@ type ClusterOptions struct {
// cluster upgrade notifications gracefully and manage connection/pool state
// transitions seamlessly. Requires Protocol: 3 (RESP3) for push notifications.
// If nil, maintnotifications upgrades are in "auto" mode and will be enabled if the server supports it.
// The ClusterClient does not directly work with maintnotifications, it is up to the clients in the Nodes map to work with maintnotifications.
// The ClusterClient supports SMIGRATING and SMIGRATED notifications for cluster state management.
// Individual node clients handle other maintenance notifications (MOVING, MIGRATING, etc.).
MaintNotificationsConfig *maintnotifications.Config
// ShardPicker is used to pick a shard when the request_policy is
// ReqDefault and the command has no keys.
ShardPicker routing.ShardPicker
// ClusterStateReloadInterval is the interval for reloading the cluster state.
// Default is 10 seconds.
ClusterStateReloadInterval time.Duration
}
func (opt *ClusterOptions) init() {
@@ -162,9 +198,24 @@ func (opt *ClusterOptions) init() {
opt.ReadOnly = true
}
if opt.DialTimeout == 0 {
opt.DialTimeout = 5 * time.Second
}
if opt.DialerRetries == 0 {
opt.DialerRetries = 5
}
if opt.DialerRetryTimeout == 0 {
opt.DialerRetryTimeout = 100 * time.Millisecond
}
if opt.PoolSize == 0 {
opt.PoolSize = 5 * runtime.GOMAXPROCS(0)
}
if opt.MaxConcurrentDials <= 0 {
opt.MaxConcurrentDials = opt.PoolSize
} else if opt.MaxConcurrentDials > opt.PoolSize {
opt.MaxConcurrentDials = opt.PoolSize
}
if opt.ReadBufferSize == 0 {
opt.ReadBufferSize = proto.DefaultBufferSize
}
@@ -208,6 +259,14 @@ func (opt *ClusterOptions) init() {
if opt.FailingTimeoutSeconds == 0 {
opt.FailingTimeoutSeconds = 15
}
if opt.ShardPicker == nil {
opt.ShardPicker = &routing.RoundRobinPicker{}
}
if opt.ClusterStateReloadInterval == 0 {
opt.ClusterStateReloadInterval = 10 * time.Second
}
}
// ParseClusterURL parses a URL into ClusterOptions that can be used to connect to Redis.
@@ -302,15 +361,21 @@ func setupClusterQueryParams(u *url.URL, o *ClusterOptions) (*ClusterOptions, er
o.MinRetryBackoff = q.duration("min_retry_backoff")
o.MaxRetryBackoff = q.duration("max_retry_backoff")
o.DialTimeout = q.duration("dial_timeout")
o.DialerRetries = q.int("dialer_retries")
o.DialerRetryTimeout = q.duration("dialer_retry_timeout")
o.ReadTimeout = q.duration("read_timeout")
o.WriteTimeout = q.duration("write_timeout")
o.PoolFIFO = q.bool("pool_fifo")
o.PoolSize = q.int("pool_size")
o.MaxConcurrentDials = q.int("max_concurrent_dials")
o.MinIdleConns = q.int("min_idle_conns")
o.MaxIdleConns = q.int("max_idle_conns")
o.MaxActiveConns = q.int("max_active_conns")
o.PoolTimeout = q.duration("pool_timeout")
o.ConnMaxLifetime = q.duration("conn_max_lifetime")
if q.has("conn_max_lifetime_jitter") {
o.ConnMaxLifetimeJitter = min(q.duration("conn_max_lifetime_jitter"), o.ConnMaxLifetime)
}
o.ConnMaxIdleTime = q.duration("conn_max_idle_time")
o.FailingTimeoutSeconds = q.int("failing_timeout_seconds")
@@ -361,19 +426,24 @@ func (opt *ClusterOptions) clientOptions() *Options {
MinRetryBackoff: opt.MinRetryBackoff,
MaxRetryBackoff: opt.MaxRetryBackoff,
DialTimeout: opt.DialTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
DialTimeout: opt.DialTimeout,
DialerRetries: opt.DialerRetries,
DialerRetryTimeout: opt.DialerRetryTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
ContextTimeoutEnabled: opt.ContextTimeoutEnabled,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
MaxConcurrentDials: opt.MaxConcurrentDials,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ConnMaxLifetimeJitter: opt.ConnMaxLifetimeJitter,
ReadBufferSize: opt.ReadBufferSize,
WriteBufferSize: opt.WriteBufferSize,
DisableIdentity: opt.DisableIdentity,
@@ -407,9 +477,10 @@ type clusterNode struct {
lastLatencyMeasurement int64 // atomic
}
func newClusterNode(clOpt *ClusterOptions, addr string) *clusterNode {
func newClusterNodeWithNodeAddress(clOpt *ClusterOptions, addr, nodeAddress string) *clusterNode {
opt := clOpt.clientOptions()
opt.Addr = addr
opt.NodeAddress = nodeAddress
node := clusterNode{
Client: clOpt.NewClient(opt),
}
@@ -637,6 +708,10 @@ func (c *clusterNodes) GC(generation uint32) {
}
func (c *clusterNodes) GetOrCreate(addr string) (*clusterNode, error) {
return c.GetOrCreateWithNodeAddress(addr, "")
}
func (c *clusterNodes) GetOrCreateWithNodeAddress(addr, nodeAddress string) (*clusterNode, error) {
node, err := c.get(addr)
if err != nil {
return nil, err
@@ -657,7 +732,7 @@ func (c *clusterNodes) GetOrCreate(addr string) (*clusterNode, error) {
return node, nil
}
node = newClusterNode(c.opt, addr)
node = newClusterNodeWithNodeAddress(c.opt, addr, nodeAddress)
for _, fn := range c.onNewNode {
fn(node.Client)
}
@@ -754,12 +829,14 @@ func newClusterState(
for _, slot := range slots {
var nodes []*clusterNode
for i, slotNode := range slot.Nodes {
addr := slotNode.Addr
// slotNode.Addr is the node address from CLUSTER SLOTS
nodeAddress := slotNode.Addr
addr := nodeAddress
if !isLoopbackOrigin {
addr = replaceLoopbackHost(addr, originHost)
}
node, err := c.nodes.GetOrCreate(addr)
node, err := c.nodes.GetOrCreateWithNodeAddress(addr, nodeAddress)
if err != nil {
return nil, err
}
@@ -926,6 +1003,29 @@ func (c *clusterState) slotRandomNode(slot int) (*clusterNode, error) {
return nodes[randomNodes[0]], nil
}
func (c *clusterState) slotShardPickerSlaveNode(slot int, shardPicker routing.ShardPicker) (*clusterNode, error) {
nodes := c.slotNodes(slot)
if len(nodes) == 0 {
return c.nodes.Random()
}
// nodes[0] is master, nodes[1:] are slaves
// First, try all slave nodes for this slot using ShardPicker order
slaves := nodes[1:]
if len(slaves) > 0 {
for i := 0; i < len(slaves); i++ {
idx := shardPicker.Next(len(slaves))
slave := slaves[idx]
if !slave.Failing() && !slave.Loading() {
return slave, nil
}
}
}
// All slaves are failing or loading - return master
return nodes[0], nil
}
func (c *clusterState) slotNodes(slot int) []*clusterNode {
i := sort.Search(len(c.slots), func(i int) bool {
return c.slots[i].end >= slot
@@ -945,13 +1045,16 @@ func (c *clusterState) slotNodes(slot int) []*clusterNode {
type clusterStateHolder struct {
load func(ctx context.Context) (*clusterState, error)
state atomic.Value
reloading uint32 // atomic
reloadInterval time.Duration
state atomic.Value
reloading uint32 // atomic
reloadPending uint32 // atomic - set to 1 when reload is requested during active reload
}
func newClusterStateHolder(fn func(ctx context.Context) (*clusterState, error)) *clusterStateHolder {
func newClusterStateHolder(load func(ctx context.Context) (*clusterState, error), reloadInterval time.Duration) *clusterStateHolder {
return &clusterStateHolder{
load: fn,
load: load,
reloadInterval: reloadInterval,
}
}
@@ -965,17 +1068,37 @@ func (c *clusterStateHolder) Reload(ctx context.Context) (*clusterState, error)
}
func (c *clusterStateHolder) LazyReload() {
// If already reloading, mark that another reload is pending
if !atomic.CompareAndSwapUint32(&c.reloading, 0, 1) {
atomic.StoreUint32(&c.reloadPending, 1)
return
}
go func() {
defer atomic.StoreUint32(&c.reloading, 0)
_, err := c.Reload(context.Background())
if err != nil {
return
go func() {
for {
_, err := c.Reload(context.Background())
if err != nil {
atomic.StoreUint32(&c.reloadPending, 0)
atomic.StoreUint32(&c.reloading, 0)
return
}
// Clear pending flag after reload completes, before cooldown
// This captures notifications that arrived during the reload
atomic.StoreUint32(&c.reloadPending, 0)
// Wait cooldown period
time.Sleep(200 * time.Millisecond)
// Check if another reload was requested during cooldown
if atomic.LoadUint32(&c.reloadPending) == 0 {
// No pending reload, we're done
atomic.StoreUint32(&c.reloading, 0)
return
}
// Pending reload requested, loop to reload again
}
time.Sleep(200 * time.Millisecond)
}()
}
@@ -986,7 +1109,7 @@ func (c *clusterStateHolder) Get(ctx context.Context) (*clusterState, error) {
}
state := v.(*clusterState)
if time.Since(state.createdAt) > 10*time.Second {
if time.Since(state.createdAt) > c.reloadInterval {
c.LazyReload()
}
return state, nil
@@ -1006,10 +1129,11 @@ func (c *clusterStateHolder) ReloadOrGet(ctx context.Context) (*clusterState, er
// or more underlying connections. It's safe for concurrent use by
// multiple goroutines.
type ClusterClient struct {
opt *ClusterOptions
nodes *clusterNodes
state *clusterStateHolder
cmdsInfoCache *cmdsInfoCache
opt *ClusterOptions
nodes *clusterNodes
state *clusterStateHolder
cmdsInfoCache *cmdsInfoCache
cmdInfoResolver *commandInfoResolver
cmdable
hooksMixin
}
@@ -1017,9 +1141,6 @@ type ClusterClient struct {
// NewClusterClient returns a Redis Cluster client as described in
// http://redis.io/topics/cluster-spec.
func NewClusterClient(opt *ClusterOptions) *ClusterClient {
if opt == nil {
panic("redis: NewClusterClient nil options")
}
opt.init()
c := &ClusterClient{
@@ -1027,10 +1148,13 @@ func NewClusterClient(opt *ClusterOptions) *ClusterClient {
nodes: newClusterNodes(opt),
}
c.state = newClusterStateHolder(c.loadState)
c.cmdsInfoCache = newCmdsInfoCache(c.cmdsInfo)
c.cmdable = c.Process
c.state = newClusterStateHolder(c.loadState, opt.ClusterStateReloadInterval)
c.SetCommandInfoResolver(NewDefaultCommandPolicyResolver())
c.cmdable = c.Process
c.initHooks(hooks{
dial: nil,
process: c.process,
@@ -1038,6 +1162,26 @@ func NewClusterClient(opt *ClusterOptions) *ClusterClient {
txPipeline: c.processTxPipeline,
})
// Set up SMIGRATED notification handling for cluster state reload
// When a node client receives a SMIGRATED notification, it should trigger
// cluster state reload on the parent ClusterClient
if opt.MaintNotificationsConfig != nil {
c.nodes.OnNewNode(func(nodeClient *Client) {
manager := nodeClient.GetMaintNotificationsManager()
if manager != nil {
manager.SetClusterStateReloadCallback(func(ctx context.Context, hostPort string, slotRanges []string) {
// Log the migration details for now
if internal.LogLevel.InfoOrAbove() {
internal.Logger.Printf(ctx, "cluster: slots %v migrated to %s, reloading cluster state", slotRanges, hostPort)
}
// Currently we reload the entire cluster state
// In the future, this could be optimized to reload only the specific slots
c.state.LazyReload()
})
}
})
}
return c
}
@@ -1083,7 +1227,11 @@ func (c *ClusterClient) process(ctx context.Context, cmd Cmder) error {
if node == nil {
var err error
node, err = c.cmdNode(ctx, cmd.Name(), slot)
if !c.opt.DisableRoutingPolicies && c.opt.ShardPicker != nil {
node, err = c.cmdNodeWithShardPicker(ctx, cmd.Name(), slot, c.opt.ShardPicker)
} else {
node, err = c.cmdNode(ctx, cmd.Name(), slot)
}
if err != nil {
return err
}
@@ -1091,13 +1239,16 @@ func (c *ClusterClient) process(ctx context.Context, cmd Cmder) error {
if ask {
ask = false
pipe := node.Client.Pipeline()
_ = pipe.Process(ctx, NewCmd(ctx, "asking"))
_ = pipe.Process(ctx, cmd)
_, lastErr = pipe.Exec(ctx)
} else {
lastErr = node.Client.Process(ctx, cmd)
if !c.opt.DisableRoutingPolicies {
lastErr = c.routeAndRun(ctx, cmd, node)
} else {
lastErr = node.Client.Process(ctx, cmd)
}
}
// If there is no error - we are done.
@@ -1124,6 +1275,18 @@ func (c *ClusterClient) process(ctx context.Context, cmd Cmder) error {
if moved || ask {
c.state.LazyReload()
// Record error metrics
if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
errorType := "MOVED"
statusCode := "MOVED"
if ask {
errorType = "ASK"
statusCode = "ASK"
}
// MOVED/ASK are not internal errors, and this is the first attempt (retry count = 0)
errorCallback(ctx, errorType, nil, statusCode, false, 0)
}
var err error
node, err = c.nodes.GetOrCreate(addr)
if err != nil {
@@ -1371,17 +1534,35 @@ func (c *ClusterClient) Pipelined(ctx context.Context, fn func(Pipeliner) error)
}
func (c *ClusterClient) processPipeline(ctx context.Context, cmds []Cmder) error {
// Only call time.Now() if pipeline operation duration callback is set to avoid overhead
var operationStart time.Time
pipelineOpDurationCallback := otel.GetPipelineOperationDurationCallback()
if pipelineOpDurationCallback != nil {
operationStart = time.Now()
}
totalAttempts := 0
cmdsMap := newCmdsMap()
if err := c.mapCmdsByNode(ctx, cmdsMap, cmds); err != nil {
setCmdsErr(cmds, err)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, "PIPELINE", len(cmds), 1, err, nil, 0)
}
return err
}
var lastErr error
for attempt := 0; attempt <= c.opt.MaxRedirects; attempt++ {
totalAttempts++
if attempt > 0 {
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
setCmdsErr(cmds, err)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, "PIPELINE", len(cmds), totalAttempts, err, nil, 0)
}
return err
}
}
@@ -1402,6 +1583,17 @@ func (c *ClusterClient) processPipeline(ctx context.Context, cmds []Cmder) error
break
}
cmdsMap = failedCmds
lastErr = cmdsFirstErr(cmds)
}
// Record pipeline operation duration
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
finalErr := cmdsFirstErr(cmds)
if finalErr == nil {
finalErr = lastErr
}
pipelineOpDurationCallback(ctx, operationDuration, "PIPELINE", len(cmds), totalAttempts, finalErr, nil, 0)
}
return cmdsFirstErr(cmds)
@@ -1413,16 +1605,33 @@ func (c *ClusterClient) mapCmdsByNode(ctx context.Context, cmdsMap *cmdsMap, cmd
return err
}
preferredRandomSlot := -1
if c.opt.ReadOnly && c.cmdsAreReadOnly(ctx, cmds) {
for _, cmd := range cmds {
slot := c.cmdSlot(cmd, preferredRandomSlot)
if preferredRandomSlot == -1 {
preferredRandomSlot = slot
var policy *routing.CommandPolicy
if c.cmdInfoResolver != nil {
policy = c.cmdInfoResolver.GetCommandPolicy(ctx, cmd)
}
node, err := c.slotReadOnlyNode(state, slot)
if err != nil {
return err
if policy != nil && !policy.CanBeUsedInPipeline() {
return fmt.Errorf(
"redis: cannot pipeline command %q with request policy ReqAllNodes/ReqAllShards/ReqMultiShard; Note: This behavior is subject to change in the future", cmd.Name(),
)
}
slot := c.cmdSlot(cmd, -1)
var node *clusterNode
// For keyless commands (slot == -1), use ShardPicker if routing policies are enabled
if slot == -1 && !c.opt.DisableRoutingPolicies && c.opt.ShardPicker != nil {
if len(state.Masters) == 0 {
return errClusterNoNodes
}
// For read-only keyless commands, pick from all nodes (masters + slaves)
allNodes := append(state.Masters, state.Slaves...)
idx := c.opt.ShardPicker.Next(len(allNodes))
node = allNodes[idx]
} else {
node, err = c.slotReadOnlyNode(state, slot)
if err != nil {
return err
}
}
cmdsMap.Add(node, cmd)
}
@@ -1430,13 +1639,29 @@ func (c *ClusterClient) mapCmdsByNode(ctx context.Context, cmdsMap *cmdsMap, cmd
}
for _, cmd := range cmds {
slot := c.cmdSlot(cmd, preferredRandomSlot)
if preferredRandomSlot == -1 {
preferredRandomSlot = slot
var policy *routing.CommandPolicy
if c.cmdInfoResolver != nil {
policy = c.cmdInfoResolver.GetCommandPolicy(ctx, cmd)
}
node, err := state.slotMasterNode(slot)
if err != nil {
return err
if policy != nil && !policy.CanBeUsedInPipeline() {
return fmt.Errorf(
"redis: cannot pipeline command %q with request policy ReqAllNodes/ReqAllShards/ReqMultiShard; Note: This behavior is subject to change in the future", cmd.Name(),
)
}
slot := c.cmdSlot(cmd, -1)
var node *clusterNode
// For keyless commands (slot == -1), use ShardPicker if routing policies are enabled
if slot == -1 && !c.opt.DisableRoutingPolicies && c.opt.ShardPicker != nil {
if len(state.Masters) == 0 {
return errClusterNoNodes
}
idx := c.opt.ShardPicker.Next(len(state.Masters))
node = state.Masters[idx]
} else {
node, err = state.slotMasterNode(slot)
if err != nil {
return err
}
}
cmdsMap.Add(node, cmd)
}
@@ -1582,6 +1807,14 @@ func (c *ClusterClient) TxPipelined(ctx context.Context, fn func(Pipeliner) erro
}
func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) error {
// Only call time.Now() if pipeline operation duration callback is set to avoid overhead
var operationStart time.Time
pipelineOpDurationCallback := otel.GetPipelineOperationDurationCallback()
if pipelineOpDurationCallback != nil {
operationStart = time.Now()
}
totalAttempts := 0
// Trim multi .. exec.
cmds = cmds[1 : len(cmds)-1]
@@ -1592,10 +1825,14 @@ func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) err
state, err := c.state.Get(ctx)
if err != nil {
setCmdsErr(cmds, err)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), 1, err, nil, 0)
}
return err
}
keyedCmdsBySlot := c.slottedKeyedCommands(cmds)
keyedCmdsBySlot := c.slottedKeyedCommands(ctx, cmds)
slot := -1
switch len(keyedCmdsBySlot) {
case 0:
@@ -1608,20 +1845,34 @@ func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) err
default:
// TxPipeline does not support cross slot transaction.
setCmdsErr(cmds, ErrCrossSlot)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), 1, ErrCrossSlot, nil, 0)
}
return ErrCrossSlot
}
node, err := state.slotMasterNode(slot)
if err != nil {
setCmdsErr(cmds, err)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), 1, err, nil, 0)
}
return err
}
var lastErr error
cmdsMap := map[*clusterNode][]Cmder{node: cmds}
for attempt := 0; attempt <= c.opt.MaxRedirects; attempt++ {
totalAttempts++
if attempt > 0 {
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
setCmdsErr(cmds, err)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), totalAttempts, err, nil, 0)
}
return err
}
}
@@ -1642,6 +1893,16 @@ func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) err
break
}
cmdsMap = failedCmds.m
lastErr = cmdsFirstErr(cmds)
}
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
finalErr := cmdsFirstErr(cmds)
if finalErr == nil {
finalErr = lastErr
}
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), totalAttempts, finalErr, nil, 0)
}
return cmdsFirstErr(cmds)
@@ -1649,18 +1910,18 @@ func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) err
// slottedKeyedCommands returns a map of slot to commands taking into account
// only commands that have keys.
func (c *ClusterClient) slottedKeyedCommands(cmds []Cmder) map[int][]Cmder {
func (c *ClusterClient) slottedKeyedCommands(ctx context.Context, cmds []Cmder) map[int][]Cmder {
cmdsSlots := map[int][]Cmder{}
preferredRandomSlot := -1
prefferedRandomSlot := -1
for _, cmd := range cmds {
if cmdFirstKeyPos(cmd) == 0 {
continue
}
slot := c.cmdSlot(cmd, preferredRandomSlot)
if preferredRandomSlot == -1 {
preferredRandomSlot = slot
slot := c.cmdSlot(cmd, prefferedRandomSlot)
if prefferedRandomSlot == -1 {
prefferedRandomSlot = slot
}
cmdsSlots[slot] = append(cmdsSlots[slot], cmd)
@@ -1819,14 +2080,13 @@ func (c *ClusterClient) cmdsMoved(
func (c *ClusterClient) Watch(ctx context.Context, fn func(*Tx) error, keys ...string) error {
if len(keys) == 0 {
return fmt.Errorf("redis: Watch requires at least one key")
return errNoWatchKeys
}
slot := hashtag.Slot(keys[0])
for _, key := range keys[1:] {
if hashtag.Slot(key) != slot {
err := fmt.Errorf("redis: Watch requires all keys to be in the same slot")
return err
return errWatchCrosslot
}
}
@@ -1995,7 +2255,6 @@ func (c *ClusterClient) cmdsInfo(ctx context.Context) (map[string]*CommandInfo,
for _, idx := range perm {
addr := addrs[idx]
node, err := c.nodes.GetOrCreate(addr)
if err != nil {
if firstErr == nil {
@@ -2008,6 +2267,7 @@ func (c *ClusterClient) cmdsInfo(ctx context.Context) (map[string]*CommandInfo,
if err == nil {
return info, nil
}
if firstErr == nil {
firstErr = err
}
@@ -2019,35 +2279,48 @@ func (c *ClusterClient) cmdsInfo(ctx context.Context) (map[string]*CommandInfo,
return nil, firstErr
}
// cmdInfo will fetch and cache the command policies after the first execution
func (c *ClusterClient) cmdInfo(ctx context.Context, name string) *CommandInfo {
cmdsInfo, err := c.cmdsInfoCache.Get(ctx)
// Use a separate context that won't be canceled to ensure command info lookup
// doesn't fail due to original context cancellation
cmdInfoCtx := c.context(ctx)
if c.opt.ContextTimeoutEnabled && ctx != nil {
// If context timeout is enabled, still use a reasonable timeout
var cancel context.CancelFunc
cmdInfoCtx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
}
cmdsInfo, err := c.cmdsInfoCache.Get(cmdInfoCtx)
if err != nil {
internal.Logger.Printf(context.TODO(), "getting command info: %s", err)
internal.Logger.Printf(cmdInfoCtx, "getting command info: %s", err)
return nil
}
info := cmdsInfo[name]
if info == nil {
internal.Logger.Printf(context.TODO(), "info for cmd=%s not found", name)
internal.Logger.Printf(cmdInfoCtx, "info for cmd=%s not found", name)
}
return info
}
func (c *ClusterClient) cmdSlot(cmd Cmder, preferredRandomSlot int) int {
func (c *ClusterClient) cmdSlot(cmd Cmder, prefferedSlot int) int {
args := cmd.Args()
if args[0] == "cluster" && (args[1] == "getkeysinslot" || args[1] == "countkeysinslot") {
return args[2].(int)
}
return cmdSlot(cmd, cmdFirstKeyPos(cmd), preferredRandomSlot)
return cmdSlot(cmd, cmdFirstKeyPos(cmd), prefferedSlot)
}
func cmdSlot(cmd Cmder, pos int, preferredRandomSlot int) int {
func cmdSlot(cmd Cmder, pos int, prefferedRandomSlot int) int {
if pos == 0 {
if preferredRandomSlot != -1 {
return preferredRandomSlot
if prefferedRandomSlot != -1 {
return prefferedRandomSlot
}
return hashtag.RandomSlot()
// Return -1 for keyless commands to signal that ShardPicker should be used
return -1
}
firstKey := cmd.stringArg(pos)
return hashtag.Slot(firstKey)
@@ -2072,6 +2345,36 @@ func (c *ClusterClient) cmdNode(
return state.slotMasterNode(slot)
}
func (c *ClusterClient) cmdNodeWithShardPicker(
ctx context.Context,
cmdName string,
slot int,
shardPicker routing.ShardPicker,
) (*clusterNode, error) {
state, err := c.state.Get(ctx)
if err != nil {
return nil, err
}
// For keyless commands (slot == -1), use ShardPicker to select a shard
// This respects the user's configured ShardPicker policy
if slot == -1 {
if len(state.Masters) == 0 {
return nil, errClusterNoNodes
}
idx := shardPicker.Next(len(state.Masters))
return state.Masters[idx], nil
}
if c.opt.ReadOnly {
cmdInfo := c.cmdInfo(ctx, cmdName)
if cmdInfo != nil && cmdInfo.ReadOnly {
return c.slotReadOnlyNode(state, slot)
}
}
return state.slotMasterNode(slot)
}
func (c *ClusterClient) slotReadOnlyNode(state *clusterState, slot int) (*clusterNode, error) {
if c.opt.RouteByLatency {
return state.slotClosestNode(slot)
@@ -2079,6 +2382,11 @@ func (c *ClusterClient) slotReadOnlyNode(state *clusterState, slot int) (*cluste
if c.opt.RouteRandomly {
return state.slotRandomNode(slot)
}
if c.opt.ShardPicker != nil {
return state.slotShardPickerSlaveNode(slot, c.opt.ShardPicker)
}
return state.slotSlaveNode(slot)
}
@@ -2126,6 +2434,31 @@ func (c *ClusterClient) context(ctx context.Context) context.Context {
return context.Background()
}
func (c *ClusterClient) GetResolver() *commandInfoResolver {
return c.cmdInfoResolver
}
func (c *ClusterClient) SetCommandInfoResolver(cmdInfoResolver *commandInfoResolver) {
c.cmdInfoResolver = cmdInfoResolver
}
// extractCommandInfo retrieves the routing policy for a command
func (c *ClusterClient) extractCommandInfo(ctx context.Context, cmd Cmder) *routing.CommandPolicy {
if cmdInfo := c.cmdInfo(ctx, cmd.Name()); cmdInfo != nil && cmdInfo.CommandPolicy != nil {
return cmdInfo.CommandPolicy
}
return nil
}
// NewDynamicResolver returns a CommandInfoResolver
// that uses the underlying cmdInfo cache to resolve the policies
func (c *ClusterClient) NewDynamicResolver() *commandInfoResolver {
return &commandInfoResolver{
resolveFunc: c.extractCommandInfo,
}
}
func appendIfNotExist[T comparable](vals []T, newVal T) []T {
for _, v := range vals {
if v == newVal {
+992
View File
@@ -0,0 +1,992 @@
package redis
import (
"context"
"errors"
"fmt"
"reflect"
"sync"
"time"
"github.com/redis/go-redis/v9/internal/hashtag"
"github.com/redis/go-redis/v9/internal/routing"
)
var (
errInvalidCmdPointer = errors.New("redis: invalid command pointer")
errNoCmdsToAggregate = errors.New("redis: no commands to aggregate")
errNoResToAggregate = errors.New("redis: no results to aggregate")
errInvalidCursorCmdArgsCount = errors.New("redis: FT.CURSOR command requires at least 3 arguments")
errInvalidCursorIdType = errors.New("redis: invalid cursor ID type")
)
// slotResult represents the result of executing a command on a specific slot
type slotResult struct {
cmd Cmder
keys []string
err error
}
// routeAndRun routes a command to the appropriate cluster nodes and executes it
func (c *ClusterClient) routeAndRun(ctx context.Context, cmd Cmder, node *clusterNode) error {
var policy *routing.CommandPolicy
if c.cmdInfoResolver != nil {
policy = c.cmdInfoResolver.GetCommandPolicy(ctx, cmd)
}
// Set stepCount from cmdInfo if not already set
if cmd.stepCount() == 0 {
if cmdInfo := c.cmdInfo(ctx, cmd.Name()); cmdInfo != nil && cmdInfo.StepCount > 0 {
cmd.SetStepCount(cmdInfo.StepCount)
}
}
if policy == nil {
return c.executeDefault(ctx, cmd, policy, node)
}
switch policy.Request {
case routing.ReqAllNodes:
return c.executeOnAllNodes(ctx, cmd, policy)
case routing.ReqAllShards:
return c.executeOnAllShards(ctx, cmd, policy)
case routing.ReqMultiShard:
return c.executeMultiShard(ctx, cmd, policy)
case routing.ReqSpecial:
return c.executeSpecialCommand(ctx, cmd, policy, node)
default:
return c.executeDefault(ctx, cmd, policy, node)
}
}
// executeDefault handles standard command routing based on keys
func (c *ClusterClient) executeDefault(ctx context.Context, cmd Cmder, policy *routing.CommandPolicy, node *clusterNode) error {
if policy != nil && !c.hasKeys(cmd) {
if c.readOnlyEnabled() && policy.IsReadOnly() {
return c.executeOnArbitraryNode(ctx, cmd)
}
}
return node.Client.Process(ctx, cmd)
}
// executeOnArbitraryNode routes command to an arbitrary node
func (c *ClusterClient) executeOnArbitraryNode(ctx context.Context, cmd Cmder) error {
node := c.pickArbitraryNode(ctx)
if node == nil {
return errClusterNoNodes
}
return node.Client.Process(ctx, cmd)
}
// executeOnAllNodes executes command on all nodes (masters and replicas)
func (c *ClusterClient) executeOnAllNodes(ctx context.Context, cmd Cmder, policy *routing.CommandPolicy) error {
state, err := c.state.Get(ctx)
if err != nil {
return err
}
nodes := append(state.Masters, state.Slaves...)
if len(nodes) == 0 {
return errClusterNoNodes
}
return c.executeParallel(ctx, cmd, nodes, policy)
}
// executeOnAllShards executes command on all master shards
func (c *ClusterClient) executeOnAllShards(ctx context.Context, cmd Cmder, policy *routing.CommandPolicy) error {
state, err := c.state.Get(ctx)
if err != nil {
return err
}
if len(state.Masters) == 0 {
return errClusterNoNodes
}
return c.executeParallel(ctx, cmd, state.Masters, policy)
}
// executeMultiShard handles commands that operate on multiple keys across shards
func (c *ClusterClient) executeMultiShard(ctx context.Context, cmd Cmder, policy *routing.CommandPolicy) error {
args := cmd.Args()
firstKeyPos := int(cmdFirstKeyPos(cmd))
stepCount := int(cmd.stepCount())
if stepCount == 0 {
stepCount = 1 // Default to 1 if not set
}
if firstKeyPos == 0 || firstKeyPos >= len(args) {
return fmt.Errorf("redis: multi-shard command %s has no key arguments", cmd.Name())
}
// Group keys by slot
slotMap := make(map[int][]string)
keyOrder := make([]string, 0)
for i := firstKeyPos; i < len(args); i += stepCount {
key, ok := args[i].(string)
if !ok {
return fmt.Errorf("redis: non-string key at position %d: %v", i, args[i])
}
slot := hashtag.Slot(key)
slotMap[slot] = append(slotMap[slot], key)
for j := 1; j < stepCount; j++ {
if i+j >= len(args) {
break
}
slotMap[slot] = append(slotMap[slot], args[i+j].(string))
}
keyOrder = append(keyOrder, key)
}
return c.executeMultiSlot(ctx, cmd, slotMap, keyOrder, policy)
}
// executeMultiSlot executes commands across multiple slots concurrently
func (c *ClusterClient) executeMultiSlot(ctx context.Context, cmd Cmder, slotMap map[int][]string, keyOrder []string, policy *routing.CommandPolicy) error {
results := make(chan slotResult, len(slotMap))
var wg sync.WaitGroup
// Execute on each slot concurrently
for slot, keys := range slotMap {
wg.Add(1)
go func(slot int, keys []string) {
defer wg.Done()
node, err := c.cmdNodeWithShardPicker(ctx, cmd.Name(), slot, c.opt.ShardPicker)
if err != nil {
results <- slotResult{nil, keys, err}
return
}
// Create a command for this specific slot's keys
subCmd := c.createSlotSpecificCommand(ctx, cmd, keys)
err = node.Client.Process(ctx, subCmd)
results <- slotResult{subCmd, keys, err}
}(slot, keys)
}
go func() {
wg.Wait()
close(results)
}()
return c.aggregateMultiSlotResults(ctx, cmd, results, keyOrder, policy)
}
// createSlotSpecificCommand creates a new command for a specific slot's keys
func (c *ClusterClient) createSlotSpecificCommand(ctx context.Context, originalCmd Cmder, keys []string) Cmder {
originalArgs := originalCmd.Args()
firstKeyPos := int(cmdFirstKeyPos(originalCmd))
// Build new args with only the specified keys
newArgs := make([]interface{}, 0, firstKeyPos+len(keys))
// Copy command name and arguments before the keys
newArgs = append(newArgs, originalArgs[:firstKeyPos]...)
// Add the slot-specific keys
for _, key := range keys {
newArgs = append(newArgs, key)
}
// Create a new command of the same type using the helper function
return createCommandByType(ctx, originalCmd.GetCmdType(), newArgs...)
}
// createCommandByType creates a new command of the specified type with the given arguments
func createCommandByType(ctx context.Context, cmdType CmdType, args ...interface{}) Cmder {
switch cmdType {
case CmdTypeString:
return NewStringCmd(ctx, args...)
case CmdTypeInt:
return NewIntCmd(ctx, args...)
case CmdTypeBool:
return NewBoolCmd(ctx, args...)
case CmdTypeFloat:
return NewFloatCmd(ctx, args...)
case CmdTypeStringSlice:
return NewStringSliceCmd(ctx, args...)
case CmdTypeIntSlice:
return NewIntSliceCmd(ctx, args...)
case CmdTypeFloatSlice:
return NewFloatSliceCmd(ctx, args...)
case CmdTypeBoolSlice:
return NewBoolSliceCmd(ctx, args...)
case CmdTypeStatus:
return NewStatusCmd(ctx, args...)
case CmdTypeTime:
return NewTimeCmd(ctx, args...)
case CmdTypeMapStringString:
return NewMapStringStringCmd(ctx, args...)
case CmdTypeMapStringInt:
return NewMapStringIntCmd(ctx, args...)
case CmdTypeMapStringInterface:
return NewMapStringInterfaceCmd(ctx, args...)
case CmdTypeMapStringInterfaceSlice:
return NewMapStringInterfaceSliceCmd(ctx, args...)
case CmdTypeSlice:
return NewSliceCmd(ctx, args...)
case CmdTypeStringStructMap:
return NewStringStructMapCmd(ctx, args...)
case CmdTypeXMessageSlice:
return NewXMessageSliceCmd(ctx, args...)
case CmdTypeXStreamSlice:
return NewXStreamSliceCmd(ctx, args...)
case CmdTypeXPending:
return NewXPendingCmd(ctx, args...)
case CmdTypeXPendingExt:
return NewXPendingExtCmd(ctx, args...)
case CmdTypeXAutoClaim:
return NewXAutoClaimCmd(ctx, args...)
case CmdTypeXAutoClaimJustID:
return NewXAutoClaimJustIDCmd(ctx, args...)
case CmdTypeXInfoStreamFull:
return NewXInfoStreamFullCmd(ctx, args...)
case CmdTypeZSlice:
return NewZSliceCmd(ctx, args...)
case CmdTypeZWithKey:
return NewZWithKeyCmd(ctx, args...)
case CmdTypeClusterSlots:
return NewClusterSlotsCmd(ctx, args...)
case CmdTypeGeoPos:
return NewGeoPosCmd(ctx, args...)
case CmdTypeCommandsInfo:
return NewCommandsInfoCmd(ctx, args...)
case CmdTypeSlowLog:
return NewSlowLogCmd(ctx, args...)
case CmdTypeKeyValues:
return NewKeyValuesCmd(ctx, args...)
case CmdTypeZSliceWithKey:
return NewZSliceWithKeyCmd(ctx, args...)
case CmdTypeFunctionList:
return NewFunctionListCmd(ctx, args...)
case CmdTypeFunctionStats:
return NewFunctionStatsCmd(ctx, args...)
case CmdTypeKeyFlags:
return NewKeyFlagsCmd(ctx, args...)
case CmdTypeDuration:
return NewDurationCmd(ctx, time.Millisecond, args...)
}
return NewCmd(ctx, args...)
}
// executeSpecialCommand handles commands with special routing requirements
func (c *ClusterClient) executeSpecialCommand(ctx context.Context, cmd Cmder, policy *routing.CommandPolicy, node *clusterNode) error {
switch cmd.Name() {
case "ft.cursor":
return c.executeCursorCommand(ctx, cmd)
default:
return c.executeDefault(ctx, cmd, policy, node)
}
}
// executeCursorCommand handles FT.CURSOR commands with sticky routing
func (c *ClusterClient) executeCursorCommand(ctx context.Context, cmd Cmder) error {
args := cmd.Args()
if len(args) < 4 {
return errInvalidCursorCmdArgsCount
}
cursorID, ok := args[3].(string)
if !ok {
return errInvalidCursorIdType
}
// Route based on cursor ID to maintain stickiness
slot := hashtag.Slot(cursorID)
node, err := c.cmdNodeWithShardPicker(ctx, cmd.Name(), slot, c.opt.ShardPicker)
if err != nil {
return err
}
return node.Client.Process(ctx, cmd)
}
// executeParallel executes a command on multiple nodes concurrently
func (c *ClusterClient) executeParallel(ctx context.Context, cmd Cmder, nodes []*clusterNode, policy *routing.CommandPolicy) error {
if len(nodes) == 0 {
return errClusterNoNodes
}
if len(nodes) == 1 {
return nodes[0].Client.Process(ctx, cmd)
}
type nodeResult struct {
cmd Cmder
err error
}
results := make(chan nodeResult, len(nodes))
var wg sync.WaitGroup
for _, node := range nodes {
wg.Add(1)
go func(n *clusterNode) {
defer wg.Done()
cmdCopy := cmd.Clone()
err := n.Client.Process(ctx, cmdCopy)
results <- nodeResult{cmdCopy, err}
}(node)
}
go func() {
wg.Wait()
close(results)
}()
// Collect results and check for errors
cmds := make([]Cmder, 0, len(nodes))
var firstErr error
for result := range results {
if result.err != nil && firstErr == nil {
firstErr = result.err
}
cmds = append(cmds, result.cmd)
}
// If there was an error and no policy specified, fail fast
if firstErr != nil && (policy == nil || policy.Response == routing.RespDefaultKeyless) {
cmd.SetErr(firstErr)
return firstErr
}
return c.aggregateResponses(cmd, cmds, policy)
}
// aggregateMultiSlotResults aggregates results from multi-slot execution
func (c *ClusterClient) aggregateMultiSlotResults(ctx context.Context, cmd Cmder, results <-chan slotResult, keyOrder []string, policy *routing.CommandPolicy) error {
keyedResults := make(map[string]routing.AggregatorResErr)
var firstErr error
for result := range results {
if result.err != nil && firstErr == nil {
firstErr = result.err
}
if result.cmd != nil && result.err == nil {
value, err := ExtractCommandValue(result.cmd)
// Check if the result is a slice (e.g., from MGET)
if sliceValue, ok := value.([]interface{}); ok {
// Map each element to its corresponding key
for i, key := range result.keys {
if i < len(sliceValue) {
keyedResults[key] = routing.AggregatorResErr{Result: sliceValue[i], Err: err}
} else {
keyedResults[key] = routing.AggregatorResErr{Result: nil, Err: err}
}
}
} else {
// For non-slice results, map the entire result to each key
for _, key := range result.keys {
keyedResults[key] = routing.AggregatorResErr{Result: value, Err: err}
}
}
}
// TODO: return multiple errors by order when we will implement multiple errors returning
if result.err != nil {
firstErr = result.err
}
}
return c.aggregateKeyedValues(cmd, keyedResults, keyOrder, policy)
}
// aggregateKeyedValues aggregates individual key-value pairs while preserving key order
func (c *ClusterClient) aggregateKeyedValues(cmd Cmder, keyedResults map[string]routing.AggregatorResErr, keyOrder []string, policy *routing.CommandPolicy) error {
if len(keyedResults) == 0 {
return errNoResToAggregate
}
aggregator := c.createAggregator(policy, cmd, true)
// Set key order for keyed aggregators
var keyedAgg *routing.DefaultKeyedAggregator
var isKeyedAgg bool
var err error
if keyedAgg, isKeyedAgg = aggregator.(*routing.DefaultKeyedAggregator); isKeyedAgg {
err = keyedAgg.BatchAddWithKeyOrder(keyedResults, keyOrder)
} else {
err = aggregator.BatchAdd(keyedResults)
}
if err != nil {
return err
}
return c.finishAggregation(cmd, aggregator)
}
// aggregateResponses aggregates multiple shard responses
func (c *ClusterClient) aggregateResponses(cmd Cmder, cmds []Cmder, policy *routing.CommandPolicy) error {
if len(cmds) == 0 {
return errNoCmdsToAggregate
}
if len(cmds) == 1 {
shardCmd := cmds[0]
if err := shardCmd.Err(); err != nil {
cmd.SetErr(err)
return err
}
value, _ := ExtractCommandValue(shardCmd)
return c.setCommandValue(cmd, value)
}
aggregator := c.createAggregator(policy, cmd, false)
batchWithErrs := []routing.AggregatorResErr{}
// Add all results to aggregator
for _, shardCmd := range cmds {
value, err := ExtractCommandValue(shardCmd)
batchWithErrs = append(batchWithErrs, routing.AggregatorResErr{
Result: value,
Err: err,
})
}
err := aggregator.BatchSlice(batchWithErrs)
if err != nil {
return err
}
return c.finishAggregation(cmd, aggregator)
}
// createAggregator creates the appropriate response aggregator
func (c *ClusterClient) createAggregator(policy *routing.CommandPolicy, cmd Cmder, isKeyed bool) routing.ResponseAggregator {
if policy != nil {
return routing.NewResponseAggregator(policy.Response, cmd.Name())
}
if !isKeyed {
firstKeyPos := cmdFirstKeyPos(cmd)
isKeyed = firstKeyPos > 0
}
return routing.NewDefaultAggregator(isKeyed)
}
// finishAggregation completes the aggregation process and sets the result
func (c *ClusterClient) finishAggregation(cmd Cmder, aggregator routing.ResponseAggregator) error {
finalValue, finalErr := aggregator.Result()
if finalErr != nil {
cmd.SetErr(finalErr)
return finalErr
}
return c.setCommandValue(cmd, finalValue)
}
// pickArbitraryNode selects a master or slave shard using the configured ShardPicker
func (c *ClusterClient) pickArbitraryNode(ctx context.Context) *clusterNode {
state, err := c.state.Get(ctx)
if err != nil || len(state.Masters) == 0 {
return nil
}
allNodes := append(state.Masters, state.Slaves...)
idx := c.opt.ShardPicker.Next(len(allNodes))
return allNodes[idx]
}
// hasKeys checks if a command operates on keys
func (c *ClusterClient) hasKeys(cmd Cmder) bool {
firstKeyPos := cmdFirstKeyPos(cmd)
return firstKeyPos > 0
}
func (c *ClusterClient) readOnlyEnabled() bool {
return c.opt.ReadOnly
}
// setCommandValue sets the aggregated value on a command using the enum-based approach
func (c *ClusterClient) setCommandValue(cmd Cmder, value interface{}) error {
// If value is nil, it might mean ExtractCommandValue couldn't extract the value
// but the command might have executed successfully. In this case, don't set an error.
if value == nil {
// ExtractCommandValue returned nil - this means the command type is not supported
// in the aggregation flow. This is a programming error, not a runtime error.
if cmd.Err() != nil {
// Command already has an error, preserve it
return cmd.Err()
}
// Command executed successfully but we can't extract/set the aggregated value
// This indicates the command type needs to be added to ExtractCommandValue
return fmt.Errorf("redis: cannot aggregate command %s: unsupported command type %d",
cmd.Name(), cmd.GetCmdType())
}
switch cmd.GetCmdType() {
case CmdTypeGeneric:
if c, ok := cmd.(*Cmd); ok {
c.SetVal(value)
}
case CmdTypeString:
if c, ok := cmd.(*StringCmd); ok {
if v, ok := value.(string); ok {
c.SetVal(v)
}
}
case CmdTypeInt:
if c, ok := cmd.(*IntCmd); ok {
if v, ok := value.(int64); ok {
c.SetVal(v)
} else if v, ok := value.(float64); ok {
c.SetVal(int64(v))
}
}
case CmdTypeBool:
if c, ok := cmd.(*BoolCmd); ok {
if v, ok := value.(bool); ok {
c.SetVal(v)
}
}
case CmdTypeFloat:
if c, ok := cmd.(*FloatCmd); ok {
if v, ok := value.(float64); ok {
c.SetVal(v)
}
}
case CmdTypeStringSlice:
if c, ok := cmd.(*StringSliceCmd); ok {
if v, ok := value.([]string); ok {
c.SetVal(v)
}
}
case CmdTypeIntSlice:
if c, ok := cmd.(*IntSliceCmd); ok {
if v, ok := value.([]int64); ok {
c.SetVal(v)
} else if v, ok := value.([]float64); ok {
els := len(v)
intSlc := make([]int, els)
for i := range v {
intSlc[i] = int(v[i])
}
}
}
case CmdTypeFloatSlice:
if c, ok := cmd.(*FloatSliceCmd); ok {
if v, ok := value.([]float64); ok {
c.SetVal(v)
}
}
case CmdTypeBoolSlice:
if c, ok := cmd.(*BoolSliceCmd); ok {
if v, ok := value.([]bool); ok {
c.SetVal(v)
}
}
case CmdTypeMapStringString:
if c, ok := cmd.(*MapStringStringCmd); ok {
if v, ok := value.(map[string]string); ok {
c.SetVal(v)
}
}
case CmdTypeMapStringInt:
if c, ok := cmd.(*MapStringIntCmd); ok {
if v, ok := value.(map[string]int64); ok {
c.SetVal(v)
}
}
case CmdTypeMapStringInterface:
if c, ok := cmd.(*MapStringInterfaceCmd); ok {
if v, ok := value.(map[string]interface{}); ok {
c.SetVal(v)
}
}
case CmdTypeSlice:
if c, ok := cmd.(*SliceCmd); ok {
if v, ok := value.([]interface{}); ok {
c.SetVal(v)
}
}
case CmdTypeStatus:
if c, ok := cmd.(*StatusCmd); ok {
if v, ok := value.(string); ok {
c.SetVal(v)
}
}
case CmdTypeDuration:
if c, ok := cmd.(*DurationCmd); ok {
if v, ok := value.(time.Duration); ok {
c.SetVal(v)
}
}
case CmdTypeTime:
if c, ok := cmd.(*TimeCmd); ok {
if v, ok := value.(time.Time); ok {
c.SetVal(v)
}
}
case CmdTypeKeyValueSlice:
if c, ok := cmd.(*KeyValueSliceCmd); ok {
if v, ok := value.([]KeyValue); ok {
c.SetVal(v)
}
}
case CmdTypeStringStructMap:
if c, ok := cmd.(*StringStructMapCmd); ok {
if v, ok := value.(map[string]struct{}); ok {
c.SetVal(v)
}
}
case CmdTypeXMessageSlice:
if c, ok := cmd.(*XMessageSliceCmd); ok {
if v, ok := value.([]XMessage); ok {
c.SetVal(v)
}
}
case CmdTypeXStreamSlice:
if c, ok := cmd.(*XStreamSliceCmd); ok {
if v, ok := value.([]XStream); ok {
c.SetVal(v)
}
}
case CmdTypeXPending:
if c, ok := cmd.(*XPendingCmd); ok {
if v, ok := value.(*XPending); ok {
c.SetVal(v)
}
}
case CmdTypeXPendingExt:
if c, ok := cmd.(*XPendingExtCmd); ok {
if v, ok := value.([]XPendingExt); ok {
c.SetVal(v)
}
}
case CmdTypeXAutoClaim:
if c, ok := cmd.(*XAutoClaimCmd); ok {
if v, ok := value.(CmdTypeXAutoClaimValue); ok {
c.SetVal(v.messages, v.start)
}
}
case CmdTypeXAutoClaimJustID:
if c, ok := cmd.(*XAutoClaimJustIDCmd); ok {
if v, ok := value.(CmdTypeXAutoClaimJustIDValue); ok {
c.SetVal(v.ids, v.start)
}
}
case CmdTypeXInfoConsumers:
if c, ok := cmd.(*XInfoConsumersCmd); ok {
if v, ok := value.([]XInfoConsumer); ok {
c.SetVal(v)
}
}
case CmdTypeXInfoGroups:
if c, ok := cmd.(*XInfoGroupsCmd); ok {
if v, ok := value.([]XInfoGroup); ok {
c.SetVal(v)
}
}
case CmdTypeXInfoStream:
if c, ok := cmd.(*XInfoStreamCmd); ok {
if v, ok := value.(*XInfoStream); ok {
c.SetVal(v)
}
}
case CmdTypeXInfoStreamFull:
if c, ok := cmd.(*XInfoStreamFullCmd); ok {
if v, ok := value.(*XInfoStreamFull); ok {
c.SetVal(v)
}
}
case CmdTypeZSlice:
if c, ok := cmd.(*ZSliceCmd); ok {
if v, ok := value.([]Z); ok {
c.SetVal(v)
}
}
case CmdTypeZWithKey:
if c, ok := cmd.(*ZWithKeyCmd); ok {
if v, ok := value.(*ZWithKey); ok {
c.SetVal(v)
}
}
case CmdTypeScan:
if c, ok := cmd.(*ScanCmd); ok {
if v, ok := value.(CmdTypeScanValue); ok {
c.SetVal(v.keys, v.cursor)
}
}
case CmdTypeClusterSlots:
if c, ok := cmd.(*ClusterSlotsCmd); ok {
if v, ok := value.([]ClusterSlot); ok {
c.SetVal(v)
}
}
case CmdTypeGeoLocation:
if c, ok := cmd.(*GeoLocationCmd); ok {
if v, ok := value.([]GeoLocation); ok {
c.SetVal(v)
}
}
case CmdTypeGeoSearchLocation:
if c, ok := cmd.(*GeoSearchLocationCmd); ok {
if v, ok := value.([]GeoLocation); ok {
c.SetVal(v)
}
}
case CmdTypeGeoPos:
if c, ok := cmd.(*GeoPosCmd); ok {
if v, ok := value.([]*GeoPos); ok {
c.SetVal(v)
}
}
case CmdTypeCommandsInfo:
if c, ok := cmd.(*CommandsInfoCmd); ok {
if v, ok := value.(map[string]*CommandInfo); ok {
c.SetVal(v)
}
}
case CmdTypeSlowLog:
if c, ok := cmd.(*SlowLogCmd); ok {
if v, ok := value.([]SlowLog); ok {
c.SetVal(v)
}
}
case CmdTypeMapStringStringSlice:
if c, ok := cmd.(*MapStringStringSliceCmd); ok {
if v, ok := value.([]map[string]string); ok {
c.SetVal(v)
}
}
case CmdTypeMapMapStringInterface:
if c, ok := cmd.(*MapMapStringInterfaceCmd); ok {
if v, ok := value.(map[string]interface{}); ok {
c.SetVal(v)
}
}
case CmdTypeMapStringInterfaceSlice:
if c, ok := cmd.(*MapStringInterfaceSliceCmd); ok {
if v, ok := value.([]map[string]interface{}); ok {
c.SetVal(v)
}
}
case CmdTypeKeyValues:
if c, ok := cmd.(*KeyValuesCmd); ok {
// KeyValuesCmd needs a key string and values slice
if v, ok := value.(CmdTypeKeyValuesValue); ok {
c.SetVal(v.key, v.values)
}
}
case CmdTypeZSliceWithKey:
if c, ok := cmd.(*ZSliceWithKeyCmd); ok {
// ZSliceWithKeyCmd needs a key string and Z slice
if v, ok := value.(CmdTypeZSliceWithKeyValue); ok {
c.SetVal(v.key, v.zSlice)
}
}
case CmdTypeFunctionList:
if c, ok := cmd.(*FunctionListCmd); ok {
if v, ok := value.([]Library); ok {
c.SetVal(v)
}
}
case CmdTypeFunctionStats:
if c, ok := cmd.(*FunctionStatsCmd); ok {
if v, ok := value.(FunctionStats); ok {
c.SetVal(v)
}
}
case CmdTypeLCS:
if c, ok := cmd.(*LCSCmd); ok {
if v, ok := value.(*LCSMatch); ok {
c.SetVal(v)
}
}
case CmdTypeKeyFlags:
if c, ok := cmd.(*KeyFlagsCmd); ok {
if v, ok := value.([]KeyFlags); ok {
c.SetVal(v)
}
}
case CmdTypeClusterLinks:
if c, ok := cmd.(*ClusterLinksCmd); ok {
if v, ok := value.([]ClusterLink); ok {
c.SetVal(v)
}
}
case CmdTypeClusterShards:
if c, ok := cmd.(*ClusterShardsCmd); ok {
if v, ok := value.([]ClusterShard); ok {
c.SetVal(v)
}
}
case CmdTypeRankWithScore:
if c, ok := cmd.(*RankWithScoreCmd); ok {
if v, ok := value.(RankScore); ok {
c.SetVal(v)
}
}
case CmdTypeClientInfo:
if c, ok := cmd.(*ClientInfoCmd); ok {
if v, ok := value.(*ClientInfo); ok {
c.SetVal(v)
}
}
case CmdTypeACLLog:
if c, ok := cmd.(*ACLLogCmd); ok {
if v, ok := value.([]*ACLLogEntry); ok {
c.SetVal(v)
}
}
case CmdTypeInfo:
if c, ok := cmd.(*InfoCmd); ok {
if v, ok := value.(map[string]map[string]string); ok {
c.SetVal(v)
}
}
case CmdTypeMonitor:
// MonitorCmd doesn't have SetVal method
// Skip setting value for MonitorCmd
case CmdTypeJSON:
if c, ok := cmd.(*JSONCmd); ok {
if v, ok := value.(string); ok {
c.SetVal(v)
}
}
case CmdTypeJSONSlice:
if c, ok := cmd.(*JSONSliceCmd); ok {
if v, ok := value.([]interface{}); ok {
c.SetVal(v)
}
}
case CmdTypeIntPointerSlice:
if c, ok := cmd.(*IntPointerSliceCmd); ok {
if v, ok := value.([]*int64); ok {
c.SetVal(v)
}
}
case CmdTypeScanDump:
if c, ok := cmd.(*ScanDumpCmd); ok {
if v, ok := value.(ScanDump); ok {
c.SetVal(v)
}
}
case CmdTypeBFInfo:
if c, ok := cmd.(*BFInfoCmd); ok {
if v, ok := value.(BFInfo); ok {
c.SetVal(v)
}
}
case CmdTypeCFInfo:
if c, ok := cmd.(*CFInfoCmd); ok {
if v, ok := value.(CFInfo); ok {
c.SetVal(v)
}
}
case CmdTypeCMSInfo:
if c, ok := cmd.(*CMSInfoCmd); ok {
if v, ok := value.(CMSInfo); ok {
c.SetVal(v)
}
}
case CmdTypeTopKInfo:
if c, ok := cmd.(*TopKInfoCmd); ok {
if v, ok := value.(TopKInfo); ok {
c.SetVal(v)
}
}
case CmdTypeTDigestInfo:
if c, ok := cmd.(*TDigestInfoCmd); ok {
if v, ok := value.(TDigestInfo); ok {
c.SetVal(v)
}
}
case CmdTypeFTSynDump:
if c, ok := cmd.(*FTSynDumpCmd); ok {
if v, ok := value.([]FTSynDumpResult); ok {
c.SetVal(v)
}
}
case CmdTypeAggregate:
if c, ok := cmd.(*AggregateCmd); ok {
if v, ok := value.(*FTAggregateResult); ok {
c.SetVal(v)
}
}
case CmdTypeFTInfo:
if c, ok := cmd.(*FTInfoCmd); ok {
if v, ok := value.(FTInfoResult); ok {
c.SetVal(v)
}
}
case CmdTypeFTSpellCheck:
if c, ok := cmd.(*FTSpellCheckCmd); ok {
if v, ok := value.([]SpellCheckResult); ok {
c.SetVal(v)
}
}
case CmdTypeFTSearch:
if c, ok := cmd.(*FTSearchCmd); ok {
if v, ok := value.(FTSearchResult); ok {
c.SetVal(v)
}
}
case CmdTypeTSTimestampValue:
if c, ok := cmd.(*TSTimestampValueCmd); ok {
if v, ok := value.(TSTimestampValue); ok {
c.SetVal(v)
}
}
case CmdTypeTSTimestampValueSlice:
if c, ok := cmd.(*TSTimestampValueSliceCmd); ok {
if v, ok := value.([]TSTimestampValue); ok {
c.SetVal(v)
}
}
default:
// Fallback to reflection for unknown types
return c.setCommandValueReflection(cmd, value)
}
return nil
}
// setCommandValueReflection is a fallback function that uses reflection
func (c *ClusterClient) setCommandValueReflection(cmd Cmder, value interface{}) error {
cmdValue := reflect.ValueOf(cmd)
if cmdValue.Kind() != reflect.Ptr || cmdValue.IsNil() {
return errInvalidCmdPointer
}
setValMethod := cmdValue.MethodByName("SetVal")
if !setValMethod.IsValid() {
return fmt.Errorf("redis: command %T does not have SetVal method", cmd)
}
args := []reflect.Value{reflect.ValueOf(value)}
switch cmd.(type) {
case *XAutoClaimCmd, *XAutoClaimJustIDCmd:
args = append(args, reflect.ValueOf(""))
case *ScanCmd:
args = append(args, reflect.ValueOf(uint64(0)))
case *KeyValuesCmd, *ZSliceWithKeyCmd:
if key, ok := value.(string); ok {
args = []reflect.Value{reflect.ValueOf(key)}
if _, ok := cmd.(*ZSliceWithKeyCmd); ok {
args = append(args, reflect.ValueOf([]Z{}))
} else {
args = append(args, reflect.ValueOf([]string{}))
}
}
}
defer func() {
if r := recover(); r != nil {
cmd.SetErr(fmt.Errorf("redis: failed to set command value: %v", r))
}
}()
setValMethod.Call(args)
return nil
}
+204
View File
@@ -0,0 +1,204 @@
package redis
import (
"context"
"net"
"time"
"github.com/redis/go-redis/v9/internal/otel"
"github.com/redis/go-redis/v9/internal/pool"
)
// ConnInfo provides information about a Redis connection for metrics.
type ConnInfo interface {
RemoteAddr() net.Addr
PoolName() string
}
type Pooler interface {
PoolStats() *pool.Stats
}
type PubSubPooler interface {
Stats() *pool.PubSubStats
}
// OTelRecorder is the interface for recording OpenTelemetry metrics.
type OTelRecorder interface {
// RecordOperationDuration records the total operation duration (including all retries)
RecordOperationDuration(ctx context.Context, duration time.Duration, cmd Cmder, attempts int, err error, cn ConnInfo, dbIndex int)
// RecordPipelineOperationDuration records the total pipeline/transaction duration.
// operationName should be "PIPELINE" for regular pipelines or "MULTI" for transactions.
RecordPipelineOperationDuration(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn ConnInfo, dbIndex int)
// RecordConnectionCreateTime records the time it took to create a new connection
RecordConnectionCreateTime(ctx context.Context, duration time.Duration, cn ConnInfo)
// RecordConnectionRelaxedTimeout records when connection timeout is relaxed/unrelaxed
// delta: +1 for relaxed, -1 for unrelaxed
// poolName: name of the connection pool (e.g., "main", "pubsub")
// notificationType: the notification type that triggered the timeout relaxation (e.g., "MOVING", "HANDOFF")
RecordConnectionRelaxedTimeout(ctx context.Context, delta int, cn ConnInfo, poolName, notificationType string)
// RecordConnectionHandoff records when a connection is handed off to another node
// poolName: name of the connection pool (e.g., "main", "pubsub")
RecordConnectionHandoff(ctx context.Context, cn ConnInfo, poolName string)
// RecordError records client errors (ASK, MOVED, handshake failures, etc.)
// errorType: type of error (e.g., "ASK", "MOVED", "HANDSHAKE_FAILED")
// statusCode: Redis response status code if available (e.g., "MOVED", "ASK")
// isInternal: whether this is an internal error
// retryAttempts: number of retry attempts made
RecordError(ctx context.Context, errorType string, cn ConnInfo, statusCode string, isInternal bool, retryAttempts int)
// RecordMaintenanceNotification records when a maintenance notification is received
// notificationType: the type of notification (e.g., "MOVING", "MIGRATING", etc.)
RecordMaintenanceNotification(ctx context.Context, cn ConnInfo, notificationType string)
// RecordConnectionWaitTime records the time spent waiting for a connection from the pool
RecordConnectionWaitTime(ctx context.Context, duration time.Duration, cn ConnInfo)
// RecordConnectionClosed records when a connection is closed
// reason: reason for closing (e.g., "idle", "max_lifetime", "error", "pool_closed")
// err: the error that caused the close (nil for non-error closures)
RecordConnectionClosed(ctx context.Context, cn ConnInfo, reason string, err error)
// RecordPubSubMessage records a Pub/Sub message
// direction: "sent" or "received"
// channel: channel name (may be hidden for cardinality reduction)
// sharded: true for sharded pub/sub (SPUBLISH/SSUBSCRIBE)
RecordPubSubMessage(ctx context.Context, cn ConnInfo, direction, channel string, sharded bool)
// RecordStreamLag records the lag for stream consumer group processing
// lag: time difference between message creation and consumption
// streamName: name of the stream (may be hidden for cardinality reduction)
// consumerGroup: name of the consumer group
// consumerName: name of the consumer
RecordStreamLag(ctx context.Context, lag time.Duration, cn ConnInfo, streamName, consumerGroup, consumerName string)
}
// This is used for async gauge metrics that need to pull stats from pools periodically.
type OTelPoolRegistrar interface {
// RegisterPool is called when a new client is created with its main connection pool.
// poolName: unique identifier for the pool (e.g., "main_abc123")
RegisterPool(poolName string, pool Pooler)
// UnregisterPool is called when a client is closed to remove its pool from the registry.
UnregisterPool(pool Pooler)
// RegisterPubSubPool is called when a new client is created with a PubSub pool.
// poolName: unique identifier for the pool (e.g., "main_abc123_pubsub")
RegisterPubSubPool(poolName string, pool PubSubPooler)
// UnregisterPubSubPool is called when a PubSub client is closed to remove its pool.
UnregisterPubSubPool(pool PubSubPooler)
}
// SetOTelRecorder sets the global OpenTelemetry recorder.
func SetOTelRecorder(r OTelRecorder) {
if r == nil {
otel.SetGlobalRecorder(nil)
return
}
otel.SetGlobalRecorder(&otelRecorderAdapter{r})
}
type otelRecorderAdapter struct {
recorder OTelRecorder
}
// toConnInfo converts *pool.Conn to ConnInfo interface properly.
// This ensures that a nil *pool.Conn becomes a true nil interface,
// not a non-nil interface containing a nil pointer.
func toConnInfo(cn *pool.Conn) ConnInfo {
if cn == nil {
return nil
}
return cn
}
func (a *otelRecorderAdapter) RecordOperationDuration(ctx context.Context, duration time.Duration, cmd otel.Cmder, attempts int, err error, cn *pool.Conn, dbIndex int) {
// Convert internal Cmder to public Cmder
if publicCmd, ok := cmd.(Cmder); ok {
a.recorder.RecordOperationDuration(ctx, duration, publicCmd, attempts, err, toConnInfo(cn), dbIndex)
}
}
func (a *otelRecorderAdapter) RecordPipelineOperationDuration(ctx context.Context, duration time.Duration, operationName string, cmdCount int, attempts int, err error, cn *pool.Conn, dbIndex int) {
a.recorder.RecordPipelineOperationDuration(ctx, duration, operationName, cmdCount, attempts, err, toConnInfo(cn), dbIndex)
}
func (a *otelRecorderAdapter) RecordConnectionCreateTime(ctx context.Context, duration time.Duration, cn *pool.Conn) {
a.recorder.RecordConnectionCreateTime(ctx, duration, toConnInfo(cn))
}
func (a *otelRecorderAdapter) RecordConnectionRelaxedTimeout(ctx context.Context, delta int, cn *pool.Conn, poolName, notificationType string) {
a.recorder.RecordConnectionRelaxedTimeout(ctx, delta, toConnInfo(cn), poolName, notificationType)
}
func (a *otelRecorderAdapter) RecordConnectionHandoff(ctx context.Context, cn *pool.Conn, poolName string) {
a.recorder.RecordConnectionHandoff(ctx, toConnInfo(cn), poolName)
}
func (a *otelRecorderAdapter) RecordError(ctx context.Context, errorType string, cn *pool.Conn, statusCode string, isInternal bool, retryAttempts int) {
a.recorder.RecordError(ctx, errorType, toConnInfo(cn), statusCode, isInternal, retryAttempts)
}
func (a *otelRecorderAdapter) RecordMaintenanceNotification(ctx context.Context, cn *pool.Conn, notificationType string) {
a.recorder.RecordMaintenanceNotification(ctx, toConnInfo(cn), notificationType)
}
func (a *otelRecorderAdapter) RecordConnectionWaitTime(ctx context.Context, duration time.Duration, cn *pool.Conn) {
a.recorder.RecordConnectionWaitTime(ctx, duration, toConnInfo(cn))
}
func (a *otelRecorderAdapter) RecordConnectionClosed(ctx context.Context, cn *pool.Conn, reason string, err error) {
a.recorder.RecordConnectionClosed(ctx, toConnInfo(cn), reason, err)
}
func (a *otelRecorderAdapter) RecordPubSubMessage(ctx context.Context, cn *pool.Conn, direction, channel string, sharded bool) {
a.recorder.RecordPubSubMessage(ctx, toConnInfo(cn), direction, channel, sharded)
}
func (a *otelRecorderAdapter) RecordStreamLag(ctx context.Context, lag time.Duration, cn *pool.Conn, streamName, consumerGroup, consumerName string) {
a.recorder.RecordStreamLag(ctx, lag, toConnInfo(cn), streamName, consumerGroup, consumerName)
}
func (a *otelRecorderAdapter) RegisterPool(poolName string, p pool.Pooler) {
if registrar, ok := a.recorder.(OTelPoolRegistrar); ok {
registrar.RegisterPool(poolName, &poolerAdapter{p})
}
}
func (a *otelRecorderAdapter) UnregisterPool(p pool.Pooler) {
if registrar, ok := a.recorder.(OTelPoolRegistrar); ok {
registrar.UnregisterPool(&poolerAdapter{p})
}
}
func (a *otelRecorderAdapter) RegisterPubSubPool(poolName string, p otel.PubSubPooler) {
if registrar, ok := a.recorder.(OTelPoolRegistrar); ok {
registrar.RegisterPubSubPool(poolName, &pubSubPoolerAdapter{p})
}
}
func (a *otelRecorderAdapter) UnregisterPubSubPool(p otel.PubSubPooler) {
if registrar, ok := a.recorder.(OTelPoolRegistrar); ok {
registrar.UnregisterPubSubPool(&pubSubPoolerAdapter{p})
}
}
type poolerAdapter struct {
p pool.Pooler
}
func (a *poolerAdapter) PoolStats() *pool.Stats {
return a.p.Stats()
}
type pubSubPoolerAdapter struct {
p otel.PubSubPooler
}
func (a *pubSubPoolerAdapter) Stats() *pool.PubSubStats {
return a.p.Stats()
}
+60 -12
View File
@@ -225,8 +225,9 @@ type ScanDumpCmd struct {
func newScanDumpCmd(ctx context.Context, args ...interface{}) *ScanDumpCmd {
return &ScanDumpCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeScanDump,
},
}
}
@@ -270,6 +271,13 @@ func (cmd *ScanDumpCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *ScanDumpCmd) Clone() Cmder {
return &ScanDumpCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // ScanDump is a simple struct, can be copied directly
}
}
// Returns information about a Bloom filter.
// For more information - https://redis.io/commands/bf.info/
func (c cmdable) BFInfo(ctx context.Context, key string) *BFInfoCmd {
@@ -296,8 +304,9 @@ type BFInfoCmd struct {
func NewBFInfoCmd(ctx context.Context, args ...interface{}) *BFInfoCmd {
return &BFInfoCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeBFInfo,
},
}
}
@@ -388,6 +397,13 @@ func (cmd *BFInfoCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *BFInfoCmd) Clone() Cmder {
return &BFInfoCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // BFInfo is a simple struct, can be copied directly
}
}
// BFInfoCapacity returns information about the capacity of a Bloom filter.
// For more information - https://redis.io/commands/bf.info/
func (c cmdable) BFInfoCapacity(ctx context.Context, key string) *BFInfoCmd {
@@ -625,8 +641,9 @@ type CFInfoCmd struct {
func NewCFInfoCmd(ctx context.Context, args ...interface{}) *CFInfoCmd {
return &CFInfoCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeCFInfo,
},
}
}
@@ -692,6 +709,13 @@ func (cmd *CFInfoCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *CFInfoCmd) Clone() Cmder {
return &CFInfoCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // CFInfo is a simple struct, can be copied directly
}
}
// CFInfo returns information about a Cuckoo filter.
// For more information - https://redis.io/commands/cf.info/
func (c cmdable) CFInfo(ctx context.Context, key string) *CFInfoCmd {
@@ -787,8 +811,9 @@ type CMSInfoCmd struct {
func NewCMSInfoCmd(ctx context.Context, args ...interface{}) *CMSInfoCmd {
return &CMSInfoCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeCMSInfo,
},
}
}
@@ -843,6 +868,13 @@ func (cmd *CMSInfoCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *CMSInfoCmd) Clone() Cmder {
return &CMSInfoCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // CMSInfo is a simple struct, can be copied directly
}
}
// CMSInfo returns information about a Count-Min Sketch filter.
// For more information - https://redis.io/commands/cms.info/
func (c cmdable) CMSInfo(ctx context.Context, key string) *CMSInfoCmd {
@@ -980,8 +1012,9 @@ type TopKInfoCmd struct {
func NewTopKInfoCmd(ctx context.Context, args ...interface{}) *TopKInfoCmd {
return &TopKInfoCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeTopKInfo,
},
}
}
@@ -1038,6 +1071,13 @@ func (cmd *TopKInfoCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *TopKInfoCmd) Clone() Cmder {
return &TopKInfoCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // TopKInfo is a simple struct, can be copied directly
}
}
// TopKInfo returns information about a Top-K filter.
// For more information - https://redis.io/commands/topk.info/
func (c cmdable) TopKInfo(ctx context.Context, key string) *TopKInfoCmd {
@@ -1227,8 +1267,9 @@ type TDigestInfoCmd struct {
func NewTDigestInfoCmd(ctx context.Context, args ...interface{}) *TDigestInfoCmd {
return &TDigestInfoCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeTDigestInfo,
},
}
}
@@ -1295,6 +1336,13 @@ func (cmd *TDigestInfoCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *TDigestInfoCmd) Clone() Cmder {
return &TDigestInfoCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // TDigestInfo is a simple struct, can be copied directly
}
}
// TDigestInfo returns information about a t-Digest data structure.
// For more information - https://redis.io/commands/tdigest.info/
func (c cmdable) TDigestInfo(ctx context.Context, key string) *TDigestInfoCmd {
+26 -13
View File
@@ -8,6 +8,7 @@ import (
"time"
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/otel"
"github.com/redis/go-redis/v9/internal/pool"
"github.com/redis/go-redis/v9/internal/proto"
"github.com/redis/go-redis/v9/push"
@@ -403,7 +404,7 @@ func (p *Pong) String() string {
return "Pong"
}
func (c *PubSub) newMessage(reply interface{}) (interface{}, error) {
func (c *PubSub) newMessage(ctx context.Context, cn *pool.Conn, reply interface{}) (interface{}, error) {
switch reply := reply.(type) {
case string:
return &Pong{
@@ -420,30 +421,42 @@ func (c *PubSub) newMessage(reply interface{}) (interface{}, error) {
Count: int(reply[2].(int64)),
}, nil
case "message", "smessage":
channel := reply[1].(string)
sharded := kind == "smessage"
switch payload := reply[2].(type) {
case string:
return &Message{
Channel: reply[1].(string),
msg := &Message{
Channel: channel,
Payload: payload,
}, nil
}
// Record PubSub message received
otel.RecordPubSubMessage(ctx, cn, "received", channel, sharded)
return msg, nil
case []interface{}:
ss := make([]string, len(payload))
for i, s := range payload {
ss[i] = s.(string)
}
return &Message{
Channel: reply[1].(string),
msg := &Message{
Channel: channel,
PayloadSlice: ss,
}, nil
}
// Record PubSub message received
otel.RecordPubSubMessage(ctx, cn, "received", channel, sharded)
return msg, nil
default:
return nil, fmt.Errorf("redis: unsupported pubsub message payload: %T", payload)
}
case "pmessage":
return &Message{
channel := reply[2].(string)
msg := &Message{
Pattern: reply[1].(string),
Channel: reply[2].(string),
Channel: channel,
Payload: reply[3].(string),
}, nil
}
// Record PubSub message received (pattern message, not sharded)
otel.RecordPubSubMessage(ctx, cn, "received", channel, false)
return msg, nil
case "pong":
return &Pong{
Payload: reply[1].(string),
@@ -485,7 +498,7 @@ func (c *PubSub) ReceiveTimeout(ctx context.Context, timeout time.Duration) (int
return nil, err
}
return c.newMessage(c.cmd.Val())
return c.newMessage(ctx, cn, c.cmd.Val())
}
// Receive returns a message as a Subscription, Message, Pong or error.
@@ -734,7 +747,7 @@ func (c *channel) initMsgChan() {
}
case <-timer.C:
internal.Logger.Printf(
ctx, "redis: %s channel is full for %s (message is dropped)",
ctx, "redis: %v channel is full for %s (message is dropped)",
c, c.chanSendTimeout)
}
default:
@@ -788,7 +801,7 @@ func (c *channel) initAllChan() {
}
case <-timer.C:
internal.Logger.Printf(
ctx, "redis: %s channel is full for %s (message is dropped)",
ctx, "redis: %v channel is full for %s (message is dropped)",
c, c.chanSendTimeout)
}
default:
+13 -1
View File
@@ -1,6 +1,10 @@
package redis
import "context"
import (
"context"
"github.com/redis/go-redis/v9/internal/otel"
)
type PubSubCmdable interface {
Publish(ctx context.Context, channel string, message interface{}) *IntCmd
@@ -16,12 +20,20 @@ type PubSubCmdable interface {
func (c cmdable) Publish(ctx context.Context, channel string, message interface{}) *IntCmd {
cmd := NewIntCmd(ctx, "publish", channel, message)
_ = c(ctx, cmd)
// Record PubSub message sent (if command succeeded)
if cmd.Err() == nil {
otel.RecordPubSubMessage(ctx, nil, "sent", channel, false)
}
return cmd
}
func (c cmdable) SPublish(ctx context.Context, channel string, message interface{}) *IntCmd {
cmd := NewIntCmd(ctx, "spublish", channel, message)
_ = c(ctx, cmd)
// Record PubSub message sent (if command succeeded)
if cmd.Err() == nil {
otel.RecordPubSubMessage(ctx, nil, "sent", channel, true)
}
return cmd
}
+194 -11
View File
@@ -13,6 +13,7 @@ import (
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/auth/streaming"
"github.com/redis/go-redis/v9/internal/hscan"
"github.com/redis/go-redis/v9/internal/otel"
"github.com/redis/go-redis/v9/internal/pool"
"github.com/redis/go-redis/v9/internal/proto"
"github.com/redis/go-redis/v9/maintnotifications"
@@ -27,7 +28,11 @@ const Nil = proto.Nil
// SetLogger set custom log
// Use with VoidLogger to disable logging.
// If logger is nil, the call is ignored and the existing logger is kept.
func SetLogger(logger internal.Logging) {
if logger == nil {
return
}
internal.Logger = logger
}
@@ -238,6 +243,7 @@ func (c *baseClient) clone() *baseClient {
clone := &baseClient{
opt: c.opt,
connPool: c.connPool,
pubSubPool: c.pubSubPool,
onClose: c.onClose,
pushProcessor: c.pushProcessor,
maintNotificationsManager: maintNotificationsManager,
@@ -298,6 +304,13 @@ func (c *baseClient) _getConn(ctx context.Context) (*pool.Conn, error) {
return nil, err
}
if dialStartNs := cn.GetDialStartNs(); dialStartNs > 0 {
if cb := pool.GetMetricConnectionCreateTimeCallback(); cb != nil {
duration := time.Duration(time.Now().UnixNano() - dialStartNs)
cb(ctx, duration, cn)
}
}
// initConn will transition to IDLE state, so we need to acquire it
// before returning it to the user.
if !cn.TryAcquire() {
@@ -537,7 +550,10 @@ func (c *baseClient) initConn(ctx context.Context, cn *pool.Conn) error {
c.optLock.RLock()
maintNotifEnabled := c.opt.MaintNotificationsConfig != nil && c.opt.MaintNotificationsConfig.Mode != maintnotifications.ModeDisabled
protocol := c.opt.Protocol
endpointType := c.opt.MaintNotificationsConfig.EndpointType
var endpointType maintnotifications.EndpointType
if maintNotifEnabled {
endpointType = c.opt.MaintNotificationsConfig.EndpointType
}
c.optLock.RUnlock()
var maintNotifHandshakeErr error
if maintNotifEnabled && protocol == 3 {
@@ -559,6 +575,12 @@ func (c *baseClient) initConn(ctx context.Context, cn *pool.Conn) error {
// enabled mode, fail the connection
c.optLock.Unlock()
cn.GetStateMachine().Transition(pool.StateClosed)
// Record handshake failure metric
if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
errorCallback(ctx, "HANDSHAKE_FAILED", cn, "HANDSHAKE_FAILED", true, 0)
}
return fmt.Errorf("failed to enable maintnotifications: %w", maintNotifHandshakeErr)
default: // will handle auto and any other
// Disabling logging here as it's too noisy.
@@ -662,20 +684,119 @@ func (c *baseClient) dial(ctx context.Context, network, addr string) (net.Conn,
}
func (c *baseClient) process(ctx context.Context, cmd Cmder) error {
// Start measuring total operation duration (includes all retries)
// Only call time.Now() if operation duration callback is set to avoid overhead
var operationStart time.Time
opDurationCallback := otel.GetOperationDurationCallback()
if opDurationCallback != nil {
operationStart = time.Now()
}
var lastConn *pool.Conn
var lastErr error
totalAttempts := 0
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
totalAttempts++
attempt := attempt
retry, err := c._process(ctx, cmd, attempt)
retry, cn, err := c._process(ctx, cmd, attempt)
if cn != nil {
lastConn = cn
}
if err == nil || !retry {
// Record total operation duration
if opDurationCallback != nil {
operationDuration := time.Since(operationStart)
opDurationCallback(ctx, operationDuration, cmd, totalAttempts, err, lastConn, c.opt.DB)
}
if err != nil {
if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
errorType, statusCode, isInternal := classifyCommandError(err)
errorCallback(ctx, errorType, lastConn, statusCode, isInternal, totalAttempts-1)
}
}
return err
}
lastErr = err
}
// Record failed operation after all retries
if opDurationCallback != nil {
operationDuration := time.Since(operationStart)
opDurationCallback(ctx, operationDuration, cmd, totalAttempts, lastErr, lastConn, c.opt.DB)
}
// Record error metric for exhausted retries
if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
errorType, statusCode, isInternal := classifyCommandError(lastErr)
errorCallback(ctx, errorType, lastConn, statusCode, isInternal, totalAttempts-1)
}
return lastErr
}
// classifyCommandError classifies an error for metrics reporting.
// Returns: errorType, statusCode, isInternal
// - errorType: A string describing the error type (e.g., "TIMEOUT", "NETWORK", "ERR")
// - statusCode: The Redis error prefix or error category
// - isInternal: true for network/timeout errors, false for Redis server errors
func classifyCommandError(err error) (errorType, statusCode string, isInternal bool) {
if err == nil {
return "", "", false
}
errStr := err.Error()
// Check for timeout errors
if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
return "TIMEOUT", "TIMEOUT", true
}
// Check for network errors
if _, ok := err.(net.Error); ok {
return "NETWORK", "NETWORK", true
}
// Check for context errors
if errors.Is(err, context.Canceled) {
return "CONTEXT_CANCELED", "CONTEXT_CANCELED", true
}
if errors.Is(err, context.DeadlineExceeded) {
return "CONTEXT_TIMEOUT", "CONTEXT_TIMEOUT", true
}
// Check for Redis errors
// Examples: "ERR ...", "WRONGTYPE ...", "CLUSTERDOWN ..."
if len(errStr) > 0 {
// Find the first space to extract the prefix
spaceIdx := 0
for i, c := range errStr {
if c == ' ' {
spaceIdx = i
break
}
}
if spaceIdx == 0 {
spaceIdx = len(errStr)
}
prefix := errStr[:spaceIdx]
isUppercase := true
for _, c := range prefix {
if c < 'A' || c > 'Z' {
isUppercase = false
break
}
}
if isUppercase && len(prefix) > 0 {
return prefix, prefix, false
}
}
return "UNKNOWN", "UNKNOWN", true
}
func (c *baseClient) assertUnstableCommand(cmd Cmder) (bool, error) {
switch cmd.(type) {
case *AggregateCmd, *FTInfoCmd, *FTSpellCheckCmd, *FTSearchCmd, *FTSynDumpCmd:
@@ -689,15 +810,17 @@ func (c *baseClient) assertUnstableCommand(cmd Cmder) (bool, error) {
}
}
func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int) (bool, error) {
func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int) (bool, *pool.Conn, error) {
if attempt > 0 {
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
return false, err
return false, nil, err
}
}
var usedConn *pool.Conn
retryTimeout := uint32(0)
if err := c.withConn(ctx, func(ctx context.Context, cn *pool.Conn) error {
usedConn = cn
// Process any pending push notifications before executing the command
if err := c.processPushNotifications(ctx, cn); err != nil {
internal.Logger.Printf(ctx, "push: error processing pending notifications before command: %v", err)
@@ -738,10 +861,10 @@ func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int) (bool
return nil
}); err != nil {
retry := shouldRetry(err, atomic.LoadUint32(&retryTimeout) == 1)
return retry, err
return retry, usedConn, err
}
return false, nil
return false, usedConn, nil
}
func (c *baseClient) retryBackoff(attempt int) time.Duration {
@@ -830,6 +953,10 @@ func (c *baseClient) Close() error {
firstErr = err
}
}
// Unregister pools from OTel before closing them
otel.UnregisterPools(c.connPool, c.pubSubPool)
if c.connPool != nil {
if err := c.connPool.Close(); err != nil && firstErr == nil {
firstErr = err
@@ -848,14 +975,14 @@ func (c *baseClient) getAddr() string {
}
func (c *baseClient) processPipeline(ctx context.Context, cmds []Cmder) error {
if err := c.generalProcessPipeline(ctx, cmds, c.pipelineProcessCmds); err != nil {
if err := c.generalProcessPipeline(ctx, cmds, c.pipelineProcessCmds, "PIPELINE"); err != nil {
return err
}
return cmdsFirstErr(cmds)
}
func (c *baseClient) processTxPipeline(ctx context.Context, cmds []Cmder) error {
if err := c.generalProcessPipeline(ctx, cmds, c.txPipelineProcessCmds); err != nil {
if err := c.generalProcessPipeline(ctx, cmds, c.txPipelineProcessCmds, "MULTI"); err != nil {
return err
}
return cmdsFirstErr(cmds)
@@ -864,13 +991,27 @@ func (c *baseClient) processTxPipeline(ctx context.Context, cmds []Cmder) error
type pipelineProcessor func(context.Context, *pool.Conn, []Cmder) (bool, error)
func (c *baseClient) generalProcessPipeline(
ctx context.Context, cmds []Cmder, p pipelineProcessor,
ctx context.Context, cmds []Cmder, p pipelineProcessor, operationName string,
) error {
// Only call time.Now() if pipeline operation duration callback is set to avoid overhead
var operationStart time.Time
pipelineOpDurationCallback := otel.GetPipelineOperationDurationCallback()
if pipelineOpDurationCallback != nil {
operationStart = time.Now()
}
var lastConn *pool.Conn
totalAttempts := 0
var lastErr error
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
totalAttempts++
if attempt > 0 {
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
setCmdsErr(cmds, err)
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, operationName, len(cmds), totalAttempts, err, lastConn, c.opt.DB)
}
return err
}
}
@@ -878,6 +1019,7 @@ func (c *baseClient) generalProcessPipeline(
// Enable retries by default to retry dial errors returned by withConn.
canRetry := true
lastErr = c.withConn(ctx, func(ctx context.Context, cn *pool.Conn) error {
lastConn = cn
// Process any pending push notifications before executing the pipeline
if err := c.processPushNotifications(ctx, cn); err != nil {
internal.Logger.Printf(ctx, "push: error processing pending notifications before processing pipeline: %v", err)
@@ -891,9 +1033,31 @@ func (c *baseClient) generalProcessPipeline(
if !isRedisError(lastErr) {
setCmdsErr(cmds, lastErr)
}
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, operationName, len(cmds), totalAttempts, lastErr, lastConn, c.opt.DB)
}
if lastErr != nil {
if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
errorType, statusCode, isInternal := classifyCommandError(lastErr)
errorCallback(ctx, errorType, lastConn, statusCode, isInternal, totalAttempts-1)
}
}
return lastErr
}
}
if pipelineOpDurationCallback != nil {
operationDuration := time.Since(operationStart)
pipelineOpDurationCallback(ctx, operationDuration, operationName, len(cmds), totalAttempts, lastErr, lastConn, c.opt.DB)
}
if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
errorType, statusCode, isInternal := classifyCommandError(lastErr)
errorCallback(ctx, errorType, lastConn, statusCode, isInternal, totalAttempts-1)
}
return lastErr
}
@@ -1055,13 +1219,18 @@ func NewClient(opt *Options) *Client {
// set opt push processor for child clients
c.opt.PushNotificationProcessor = c.pushProcessor
// Generate unique pool names for metrics
uniqueID := generateUniqueID()
mainPoolName := opt.Addr + "_" + uniqueID
pubsubPoolName := opt.Addr + "_" + uniqueID + "_pubsub"
// Create connection pools
var err error
c.connPool, err = newConnPool(opt, c.dialHook)
c.connPool, err = newConnPool(opt, c.dialHook, mainPoolName)
if err != nil {
panic(fmt.Errorf("redis: failed to create connection pool: %w", err))
}
c.pubSubPool, err = newPubSubPool(opt, c.dialHook)
c.pubSubPool, err = newPubSubPool(opt, c.dialHook, pubsubPoolName)
if err != nil {
panic(fmt.Errorf("redis: failed to create pubsub pool: %w", err))
}
@@ -1092,6 +1261,10 @@ func NewClient(opt *Options) *Client {
}
}
// Register pools with OTel recorder if it supports pool registration
// This allows async gauge metrics to pull stats from pools periodically
otel.RegisterPools(c.connPool, c.pubSubPool, opt.Addr)
return &c
}
@@ -1127,6 +1300,16 @@ func (c *Client) Options() *Options {
return c.opt
}
// NodeAddress returns the address of the Redis node as reported by the server.
// For cluster clients, this is the endpoint from CLUSTER SLOTS before any transformation
// (e.g., loopback replacement). For standalone clients, this defaults to Addr.
//
// This is useful for matching the source field in maintenance notifications
// (e.g. SMIGRATED).
func (c *Client) NodeAddress() string {
return c.opt.NodeAddress
}
// GetMaintNotificationsManager returns the maintnotifications manager instance for monitoring and control.
// Returns nil if maintnotifications are not enabled.
func (c *Client) GetMaintNotificationsManager() *maintnotifications.Manager {
+33 -18
View File
@@ -108,7 +108,18 @@ type RingOptions struct {
MinRetryBackoff time.Duration
MaxRetryBackoff time.Duration
DialTimeout time.Duration
DialTimeout time.Duration
// DialerRetries is the maximum number of retry attempts when dialing fails.
//
// default: 5
DialerRetries int
// DialerRetryTimeout is the backoff duration between retry attempts.
//
// default: 100 milliseconds
DialerRetryTimeout time.Duration
ReadTimeout time.Duration
WriteTimeout time.Duration
ContextTimeoutEnabled bool
@@ -116,13 +127,14 @@ type RingOptions struct {
// PoolFIFO uses FIFO mode for each node connection pool GET/PUT (default LIFO).
PoolFIFO bool
PoolSize int
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
PoolSize int
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
ConnMaxLifetimeJitter time.Duration
// ReadBufferSize is the size of the bufio.Reader buffer for each connection.
// Larger buffers can improve performance for commands that return large responses.
@@ -219,20 +231,23 @@ func (opt *RingOptions) clientOptions() *Options {
MaxRetries: -1,
DialTimeout: opt.DialTimeout,
DialerRetries: opt.DialerRetries,
DialerRetryTimeout: opt.DialerRetryTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
ContextTimeoutEnabled: opt.ContextTimeoutEnabled,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ReadBufferSize: opt.ReadBufferSize,
WriteBufferSize: opt.WriteBufferSize,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ConnMaxLifetimeJitter: opt.ConnMaxLifetimeJitter,
ReadBufferSize: opt.ReadBufferSize,
WriteBufferSize: opt.WriteBufferSize,
TLSConfig: opt.TLSConfig,
Limiter: opt.Limiter,
+359 -18
View File
@@ -372,12 +372,18 @@ const (
// FTHybridVectorExpression represents a vector expression in hybrid search
type FTHybridVectorExpression struct {
VectorField string
VectorData Vector
Method FTHybridVectorMethod
MethodParams []interface{}
Filter string
YieldScoreAs string
VectorField string
VectorData Vector
// VectorParamName specifies the parameter name for passing vector data via PARAMS mechanism.
// REQUIRED for Redis 8.6+ (inline vector blobs are not supported in 8.6+).
// Optional for Redis 8.4-8.5 (both inline and PARAMS are supported).
// When set, the vector blob will be passed as: VSIM @field $VectorParamName PARAMS ... VectorParamName <blob>
// When empty, the vector blob will be inlined: VSIM @field <blob> (fails on Redis 8.6+)
VectorParamName string
Method FTHybridVectorMethod
MethodParams []interface{}
Filter string
YieldScoreAs string
}
// FTHybridCombineOptions represents options for result fusion
@@ -768,8 +774,9 @@ func ProcessAggregateResult(data []interface{}) (*FTAggregateResult, error) {
func NewAggregateCmd(ctx context.Context, args ...interface{}) *AggregateCmd {
return &AggregateCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeAggregate,
},
}
}
@@ -810,6 +817,31 @@ func (cmd *AggregateCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *AggregateCmd) Clone() Cmder {
var val *FTAggregateResult
if cmd.val != nil {
val = &FTAggregateResult{
Total: cmd.val.Total,
}
if cmd.val.Rows != nil {
val.Rows = make([]AggregateRow, len(cmd.val.Rows))
for i, row := range cmd.val.Rows {
val.Rows[i] = AggregateRow{}
if row.Fields != nil {
val.Rows[i].Fields = make(map[string]interface{}, len(row.Fields))
for k, v := range row.Fields {
val.Rows[i].Fields[k] = v
}
}
}
}
}
return &AggregateCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
// FTAggregateWithArgs - Performs a search query on an index and applies a series of aggregate transformations to the result.
// The 'index' parameter specifies the index to search, and the 'query' parameter specifies the search query.
// This function also allows for specifying additional options such as: Verbatim, LoadAll, Load, Timeout, GroupBy, SortBy, SortByMax, Apply, LimitOffset, Limit, Filter, WithCursor, Params, and DialectVersion.
@@ -1597,8 +1629,9 @@ type FTInfoCmd struct {
func newFTInfoCmd(ctx context.Context, args ...interface{}) *FTInfoCmd {
return &FTInfoCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeFTInfo,
},
}
}
@@ -1660,6 +1693,68 @@ func (cmd *FTInfoCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *FTInfoCmd) Clone() Cmder {
val := FTInfoResult{
IndexErrors: cmd.val.IndexErrors,
BytesPerRecordAvg: cmd.val.BytesPerRecordAvg,
Cleaning: cmd.val.Cleaning,
CursorStats: cmd.val.CursorStats,
DocTableSizeMB: cmd.val.DocTableSizeMB,
GCStats: cmd.val.GCStats,
GeoshapesSzMB: cmd.val.GeoshapesSzMB,
HashIndexingFailures: cmd.val.HashIndexingFailures,
IndexDefinition: cmd.val.IndexDefinition,
IndexName: cmd.val.IndexName,
Indexing: cmd.val.Indexing,
InvertedSzMB: cmd.val.InvertedSzMB,
KeyTableSizeMB: cmd.val.KeyTableSizeMB,
MaxDocID: cmd.val.MaxDocID,
NumDocs: cmd.val.NumDocs,
NumRecords: cmd.val.NumRecords,
NumTerms: cmd.val.NumTerms,
NumberOfUses: cmd.val.NumberOfUses,
OffsetBitsPerRecordAvg: cmd.val.OffsetBitsPerRecordAvg,
OffsetVectorsSzMB: cmd.val.OffsetVectorsSzMB,
OffsetsPerTermAvg: cmd.val.OffsetsPerTermAvg,
PercentIndexed: cmd.val.PercentIndexed,
RecordsPerDocAvg: cmd.val.RecordsPerDocAvg,
SortableValuesSizeMB: cmd.val.SortableValuesSizeMB,
TagOverheadSzMB: cmd.val.TagOverheadSzMB,
TextOverheadSzMB: cmd.val.TextOverheadSzMB,
TotalIndexMemorySzMB: cmd.val.TotalIndexMemorySzMB,
TotalIndexingTime: cmd.val.TotalIndexingTime,
TotalInvertedIndexBlocks: cmd.val.TotalInvertedIndexBlocks,
VectorIndexSzMB: cmd.val.VectorIndexSzMB,
}
// Clone slices and maps
if cmd.val.Attributes != nil {
val.Attributes = make([]FTAttribute, len(cmd.val.Attributes))
copy(val.Attributes, cmd.val.Attributes)
}
if cmd.val.DialectStats != nil {
val.DialectStats = make(map[string]int, len(cmd.val.DialectStats))
for k, v := range cmd.val.DialectStats {
val.DialectStats[k] = v
}
}
if cmd.val.FieldStatistics != nil {
val.FieldStatistics = make([]FieldStatistic, len(cmd.val.FieldStatistics))
copy(val.FieldStatistics, cmd.val.FieldStatistics)
}
if cmd.val.IndexOptions != nil {
val.IndexOptions = make([]string, len(cmd.val.IndexOptions))
copy(val.IndexOptions, cmd.val.IndexOptions)
}
if cmd.val.IndexDefinition.Prefixes != nil {
val.IndexDefinition.Prefixes = make([]string, len(cmd.val.IndexDefinition.Prefixes))
copy(val.IndexDefinition.Prefixes, cmd.val.IndexDefinition.Prefixes)
}
return &FTInfoCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
// FTInfo - Retrieves information about an index.
// The 'index' parameter specifies the index to retrieve information about.
// For more information, please refer to the Redis documentation:
@@ -1716,8 +1811,9 @@ type FTSpellCheckCmd struct {
func newFTSpellCheckCmd(ctx context.Context, args ...interface{}) *FTSpellCheckCmd {
return &FTSpellCheckCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeFTSpellCheck,
},
}
}
@@ -1813,6 +1909,26 @@ func parseFTSpellCheck(data []interface{}) ([]SpellCheckResult, error) {
return results, nil
}
func (cmd *FTSpellCheckCmd) Clone() Cmder {
var val []SpellCheckResult
if cmd.val != nil {
val = make([]SpellCheckResult, len(cmd.val))
for i, result := range cmd.val {
val[i] = SpellCheckResult{
Term: result.Term,
}
if result.Suggestions != nil {
val[i].Suggestions = make([]SpellCheckSuggestion, len(result.Suggestions))
copy(val[i].Suggestions, result.Suggestions)
}
}
}
return &FTSpellCheckCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
func parseFTSearch(data []interface{}, noContent, withScores, withPayloads, withSortKeys bool) (FTSearchResult, error) {
if len(data) < 1 {
return FTSearchResult{}, fmt.Errorf("unexpected search result format")
@@ -1909,8 +2025,9 @@ type FTSearchCmd struct {
func newFTSearchCmd(ctx context.Context, options *FTSearchOptions, args ...interface{}) *FTSearchCmd {
return &FTSearchCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeFTSearch,
},
options: options,
}
@@ -1952,6 +2069,89 @@ func (cmd *FTSearchCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *FTSearchCmd) Clone() Cmder {
val := FTSearchResult{
Total: cmd.val.Total,
}
if cmd.val.Docs != nil {
val.Docs = make([]Document, len(cmd.val.Docs))
for i, doc := range cmd.val.Docs {
val.Docs[i] = Document{
ID: doc.ID,
Score: doc.Score,
Payload: doc.Payload,
SortKey: doc.SortKey,
}
if doc.Fields != nil {
val.Docs[i].Fields = make(map[string]string, len(doc.Fields))
for k, v := range doc.Fields {
val.Docs[i].Fields[k] = v
}
}
}
}
var options *FTSearchOptions
if cmd.options != nil {
options = &FTSearchOptions{
NoContent: cmd.options.NoContent,
Verbatim: cmd.options.Verbatim,
NoStopWords: cmd.options.NoStopWords,
WithScores: cmd.options.WithScores,
WithPayloads: cmd.options.WithPayloads,
WithSortKeys: cmd.options.WithSortKeys,
Slop: cmd.options.Slop,
Timeout: cmd.options.Timeout,
InOrder: cmd.options.InOrder,
Language: cmd.options.Language,
Expander: cmd.options.Expander,
Scorer: cmd.options.Scorer,
ExplainScore: cmd.options.ExplainScore,
Payload: cmd.options.Payload,
SortByWithCount: cmd.options.SortByWithCount,
LimitOffset: cmd.options.LimitOffset,
Limit: cmd.options.Limit,
CountOnly: cmd.options.CountOnly,
DialectVersion: cmd.options.DialectVersion,
}
// Clone slices and maps
if cmd.options.Filters != nil {
options.Filters = make([]FTSearchFilter, len(cmd.options.Filters))
copy(options.Filters, cmd.options.Filters)
}
if cmd.options.GeoFilter != nil {
options.GeoFilter = make([]FTSearchGeoFilter, len(cmd.options.GeoFilter))
copy(options.GeoFilter, cmd.options.GeoFilter)
}
if cmd.options.InKeys != nil {
options.InKeys = make([]interface{}, len(cmd.options.InKeys))
copy(options.InKeys, cmd.options.InKeys)
}
if cmd.options.InFields != nil {
options.InFields = make([]interface{}, len(cmd.options.InFields))
copy(options.InFields, cmd.options.InFields)
}
if cmd.options.Return != nil {
options.Return = make([]FTSearchReturn, len(cmd.options.Return))
copy(options.Return, cmd.options.Return)
}
if cmd.options.SortBy != nil {
options.SortBy = make([]FTSearchSortBy, len(cmd.options.SortBy))
copy(options.SortBy, cmd.options.SortBy)
}
if cmd.options.Params != nil {
options.Params = make(map[string]interface{}, len(cmd.options.Params))
for k, v := range cmd.options.Params {
options.Params[k] = v
}
}
}
return &FTSearchCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
options: options,
}
}
// FTHybridResult represents the result of a hybrid search operation
type FTHybridResult struct {
TotalResults int
@@ -2153,6 +2353,111 @@ func (cmd *FTHybridCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *FTHybridCmd) Clone() Cmder {
val := FTHybridResult{
TotalResults: cmd.val.TotalResults,
ExecutionTime: cmd.val.ExecutionTime,
}
if cmd.val.Results != nil {
val.Results = make([]map[string]interface{}, len(cmd.val.Results))
for i, result := range cmd.val.Results {
val.Results[i] = make(map[string]interface{}, len(result))
for k, v := range result {
val.Results[i][k] = v
}
}
}
if cmd.val.Warnings != nil {
val.Warnings = make([]string, len(cmd.val.Warnings))
copy(val.Warnings, cmd.val.Warnings)
}
var cursorVal *FTHybridCursorResult
if cmd.cursorVal != nil {
cursorVal = &FTHybridCursorResult{
SearchCursorID: cmd.cursorVal.SearchCursorID,
VsimCursorID: cmd.cursorVal.VsimCursorID,
}
}
var options *FTHybridOptions
if cmd.options != nil {
options = &FTHybridOptions{
CountExpressions: cmd.options.CountExpressions,
Load: cmd.options.Load,
Filter: cmd.options.Filter,
LimitOffset: cmd.options.LimitOffset,
Limit: cmd.options.Limit,
ExplainScore: cmd.options.ExplainScore,
Timeout: cmd.options.Timeout,
WithCursor: cmd.options.WithCursor,
}
// Clone slices and maps
if cmd.options.SearchExpressions != nil {
options.SearchExpressions = make([]FTHybridSearchExpression, len(cmd.options.SearchExpressions))
copy(options.SearchExpressions, cmd.options.SearchExpressions)
}
if cmd.options.VectorExpressions != nil {
options.VectorExpressions = make([]FTHybridVectorExpression, len(cmd.options.VectorExpressions))
copy(options.VectorExpressions, cmd.options.VectorExpressions)
}
if cmd.options.Combine != nil {
options.Combine = &FTHybridCombineOptions{
Method: cmd.options.Combine.Method,
Count: cmd.options.Combine.Count,
Window: cmd.options.Combine.Window,
Constant: cmd.options.Combine.Constant,
Alpha: cmd.options.Combine.Alpha,
Beta: cmd.options.Combine.Beta,
YieldScoreAs: cmd.options.Combine.YieldScoreAs,
}
}
if cmd.options.GroupBy != nil {
options.GroupBy = &FTHybridGroupBy{
Count: cmd.options.GroupBy.Count,
ReduceFunc: cmd.options.GroupBy.ReduceFunc,
ReduceCount: cmd.options.GroupBy.ReduceCount,
}
if cmd.options.GroupBy.Fields != nil {
options.GroupBy.Fields = make([]string, len(cmd.options.GroupBy.Fields))
copy(options.GroupBy.Fields, cmd.options.GroupBy.Fields)
}
if cmd.options.GroupBy.ReduceParams != nil {
options.GroupBy.ReduceParams = make([]interface{}, len(cmd.options.GroupBy.ReduceParams))
copy(options.GroupBy.ReduceParams, cmd.options.GroupBy.ReduceParams)
}
}
if cmd.options.Apply != nil {
options.Apply = make([]FTHybridApply, len(cmd.options.Apply))
copy(options.Apply, cmd.options.Apply)
}
if cmd.options.SortBy != nil {
options.SortBy = make([]FTSearchSortBy, len(cmd.options.SortBy))
copy(options.SortBy, cmd.options.SortBy)
}
if cmd.options.Params != nil {
options.Params = make(map[string]interface{}, len(cmd.options.Params))
for k, v := range cmd.options.Params {
options.Params[k] = v
}
}
if cmd.options.WithCursorOptions != nil {
options.WithCursorOptions = &FTHybridWithCursor{
MaxIdle: cmd.options.WithCursorOptions.MaxIdle,
Count: cmd.options.WithCursorOptions.Count,
}
}
}
return &FTHybridCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
cursorVal: cursorVal,
options: options,
withCursor: cmd.withCursor,
}
}
// FTSearch - Executes a search query on an index.
// The 'index' parameter specifies the index to search, and the 'query' parameter specifies the search query.
// For more information, please refer to the Redis documentation about [FT.SEARCH].
@@ -2412,8 +2717,9 @@ func (c cmdable) FTSearchWithArgs(ctx context.Context, index string, query strin
func NewFTSynDumpCmd(ctx context.Context, args ...interface{}) *FTSynDumpCmd {
return &FTSynDumpCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeFTSynDump,
},
}
}
@@ -2479,6 +2785,26 @@ func (cmd *FTSynDumpCmd) readReply(rd *proto.Reader) error {
return nil
}
func (cmd *FTSynDumpCmd) Clone() Cmder {
var val []FTSynDumpResult
if cmd.val != nil {
val = make([]FTSynDumpResult, len(cmd.val))
for i, result := range cmd.val {
val[i] = FTSynDumpResult{
Term: result.Term,
}
if result.Synonyms != nil {
val[i].Synonyms = make([]string, len(result.Synonyms))
copy(val[i].Synonyms, result.Synonyms)
}
}
}
return &FTSynDumpCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
// FTSynDump - Dumps the contents of a synonym group.
// The 'index' parameter specifies the index to dump.
// For more information, please refer to the Redis documentation:
@@ -2572,12 +2898,27 @@ func (c cmdable) FTHybridWithArgs(ctx context.Context, index string, options *FT
// For FT.HYBRID, we need to send just the raw vector bytes, not the Value() format
// Value() returns [format, data] but FT.HYBRID expects just the blob
vectorValue := vectorExpr.VectorData.Value()
var vectorBlob interface{}
if len(vectorValue) >= 2 {
// vectorValue is [format, data, ...] - we only want the data part
args = append(args, vectorValue[1])
vectorBlob = vectorValue[1]
} else {
// Fallback for unexpected format
args = append(args, vectorValue...)
vectorBlob = vectorValue
}
// If VectorParamName is provided, use PARAMS mechanism (required for Redis 8.6+)
// If not provided, inline the vector blob (works on Redis 8.4/8.5, fails on 8.6+)
if vectorExpr.VectorParamName != "" {
// Use PARAMS mechanism
args = append(args, "$"+vectorExpr.VectorParamName)
if options.Params == nil {
options.Params = make(map[string]interface{})
}
options.Params[vectorExpr.VectorParamName] = vectorBlob
} else {
// Inline the vector blob (deprecated in Redis 8.6+)
args = append(args, vectorBlob)
}
if vectorExpr.Method != "" {
+114 -75
View File
@@ -16,7 +16,6 @@ import (
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/pool"
"github.com/redis/go-redis/v9/internal/rand"
"github.com/redis/go-redis/v9/internal/util"
"github.com/redis/go-redis/v9/maintnotifications"
"github.com/redis/go-redis/v9/push"
)
@@ -89,7 +88,18 @@ type FailoverOptions struct {
MinRetryBackoff time.Duration
MaxRetryBackoff time.Duration
DialTimeout time.Duration
DialTimeout time.Duration
// DialerRetries is the maximum number of retry attempts when dialing fails.
//
// default: 5
DialerRetries int
// DialerRetryTimeout is the backoff duration between retry attempts.
//
// default: 100 milliseconds
DialerRetryTimeout time.Duration
ReadTimeout time.Duration
WriteTimeout time.Duration
ContextTimeoutEnabled bool
@@ -110,13 +120,19 @@ type FailoverOptions struct {
PoolFIFO bool
PoolSize int
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
PoolSize int
// MaxConcurrentDials is the maximum number of concurrent connection creation goroutines.
// If <= 0, defaults to PoolSize. If > PoolSize, it will be capped at PoolSize.
MaxConcurrentDials int
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
ConnMaxLifetimeJitter time.Duration
TLSConfig *tls.Config
@@ -141,6 +157,10 @@ type FailoverOptions struct {
UnstableResp3 bool
// PushNotificationProcessor is the processor for handling push notifications.
// If nil, a default processor will be created for RESP3 connections.
PushNotificationProcessor push.NotificationProcessor
// MaintNotificationsConfig is not supported for FailoverClients at the moment
// MaintNotificationsConfig provides custom configuration for maintnotifications upgrades.
// When MaintNotificationsConfig.Mode is not "disabled", the client will handle
@@ -174,27 +194,33 @@ func (opt *FailoverOptions) clientOptions() *Options {
ReadBufferSize: opt.ReadBufferSize,
WriteBufferSize: opt.WriteBufferSize,
DialTimeout: opt.DialTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
DialTimeout: opt.DialTimeout,
DialerRetries: opt.DialerRetries,
DialerRetryTimeout: opt.DialerRetryTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
ContextTimeoutEnabled: opt.ContextTimeoutEnabled,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
MaxConcurrentDials: opt.MaxConcurrentDials,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ConnMaxLifetimeJitter: opt.ConnMaxLifetimeJitter,
TLSConfig: opt.TLSConfig,
DisableIdentity: opt.DisableIdentity,
DisableIndentity: opt.DisableIndentity,
IdentitySuffix: opt.IdentitySuffix,
UnstableResp3: opt.UnstableResp3,
IdentitySuffix: opt.IdentitySuffix,
UnstableResp3: opt.UnstableResp3,
PushNotificationProcessor: opt.PushNotificationProcessor,
MaintNotificationsConfig: &maintnotifications.Config{
Mode: maintnotifications.ModeDisabled,
@@ -222,27 +248,33 @@ func (opt *FailoverOptions) sentinelOptions(addr string) *Options {
ReadBufferSize: 4096,
WriteBufferSize: 4096,
DialTimeout: opt.DialTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
DialTimeout: opt.DialTimeout,
DialerRetries: opt.DialerRetries,
DialerRetryTimeout: opt.DialerRetryTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
ContextTimeoutEnabled: opt.ContextTimeoutEnabled,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
MaxConcurrentDials: opt.MaxConcurrentDials,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
ConnMaxLifetimeJitter: opt.ConnMaxLifetimeJitter,
TLSConfig: opt.TLSConfig,
DisableIdentity: opt.DisableIdentity,
DisableIndentity: opt.DisableIndentity,
IdentitySuffix: opt.IdentitySuffix,
UnstableResp3: opt.UnstableResp3,
IdentitySuffix: opt.IdentitySuffix,
UnstableResp3: opt.UnstableResp3,
PushNotificationProcessor: opt.PushNotificationProcessor,
MaintNotificationsConfig: &maintnotifications.Config{
Mode: maintnotifications.ModeDisabled,
@@ -276,26 +308,31 @@ func (opt *FailoverOptions) clusterOptions() *ClusterOptions {
ReadBufferSize: opt.ReadBufferSize,
WriteBufferSize: opt.WriteBufferSize,
DialTimeout: opt.DialTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
DialTimeout: opt.DialTimeout,
DialerRetries: opt.DialerRetries,
DialerRetryTimeout: opt.DialerRetryTimeout,
ReadTimeout: opt.ReadTimeout,
WriteTimeout: opt.WriteTimeout,
ContextTimeoutEnabled: opt.ContextTimeoutEnabled,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
PoolFIFO: opt.PoolFIFO,
PoolSize: opt.PoolSize,
MaxConcurrentDials: opt.MaxConcurrentDials,
PoolTimeout: opt.PoolTimeout,
MinIdleConns: opt.MinIdleConns,
MaxIdleConns: opt.MaxIdleConns,
MaxActiveConns: opt.MaxActiveConns,
ConnMaxIdleTime: opt.ConnMaxIdleTime,
ConnMaxLifetime: opt.ConnMaxLifetime,
TLSConfig: opt.TLSConfig,
DisableIdentity: opt.DisableIdentity,
DisableIndentity: opt.DisableIndentity,
IdentitySuffix: opt.IdentitySuffix,
FailingTimeoutSeconds: opt.FailingTimeoutSeconds,
DisableIdentity: opt.DisableIdentity,
DisableIndentity: opt.DisableIndentity,
IdentitySuffix: opt.IdentitySuffix,
FailingTimeoutSeconds: opt.FailingTimeoutSeconds,
PushNotificationProcessor: opt.PushNotificationProcessor,
MaintNotificationsConfig: &maintnotifications.Config{
Mode: maintnotifications.ModeDisabled,
@@ -399,15 +436,21 @@ func setupFailoverConnParams(u *url.URL, o *FailoverOptions) (*FailoverOptions,
o.MinRetryBackoff = q.duration("min_retry_backoff")
o.MaxRetryBackoff = q.duration("max_retry_backoff")
o.DialTimeout = q.duration("dial_timeout")
o.DialerRetries = q.int("dialer_retries")
o.DialerRetryTimeout = q.duration("dialer_retry_timeout")
o.ReadTimeout = q.duration("read_timeout")
o.WriteTimeout = q.duration("write_timeout")
o.ContextTimeoutEnabled = q.bool("context_timeout_enabled")
o.PoolFIFO = q.bool("pool_fifo")
o.PoolSize = q.int("pool_size")
o.MaxConcurrentDials = q.int("max_concurrent_dials")
o.MinIdleConns = q.int("min_idle_conns")
o.MaxIdleConns = q.int("max_idle_conns")
o.MaxActiveConns = q.int("max_active_conns")
o.ConnMaxLifetime = q.duration("conn_max_lifetime")
if q.has("conn_max_lifetime_jitter") {
o.ConnMaxLifetimeJitter = min(q.duration("conn_max_lifetime_jitter"), o.ConnMaxLifetime)
}
o.ConnMaxIdleTime = q.duration("conn_max_idle_time")
o.PoolTimeout = q.duration("pool_timeout")
o.DisableIdentity = q.bool("disableIdentity")
@@ -490,12 +533,17 @@ func NewFailoverClient(failoverOpt *FailoverOptions) *Client {
// Use void processor by default for RESP2 connections
rdb.pushProcessor = initializePushProcessor(opt)
// Generate unique pool names for metrics
uniqueID := generateUniqueID()
mainPoolName := opt.Addr + "_" + uniqueID
pubsubPoolName := opt.Addr + "_" + uniqueID + "_pubsub"
var err error
rdb.connPool, err = newConnPool(opt, rdb.dialHook)
rdb.connPool, err = newConnPool(opt, rdb.dialHook, mainPoolName)
if err != nil {
panic(fmt.Errorf("redis: failed to create connection pool: %w", err))
}
rdb.pubSubPool, err = newPubSubPool(opt, rdb.dialHook)
rdb.pubSubPool, err = newPubSubPool(opt, rdb.dialHook, pubsubPoolName)
if err != nil {
panic(fmt.Errorf("redis: failed to create pubsub pool: %w", err))
}
@@ -574,12 +622,18 @@ func NewSentinelClient(opt *Options) *SentinelClient {
dial: c.baseClient.dial,
process: c.baseClient.process,
})
// Generate unique pool names for metrics
uniqueID := generateUniqueID()
mainPoolName := opt.Addr + "_" + uniqueID
pubsubPoolName := opt.Addr + "_" + uniqueID + "_pubsub"
var err error
c.connPool, err = newConnPool(opt, c.dialHook)
c.connPool, err = newConnPool(opt, c.dialHook, mainPoolName)
if err != nil {
panic(fmt.Errorf("redis: failed to create connection pool: %w", err))
}
c.pubSubPool, err = newPubSubPool(opt, c.dialHook)
c.pubSubPool, err = newPubSubPool(opt, c.dialHook, pubsubPoolName)
if err != nil {
panic(fmt.Errorf("redis: failed to create pubsub pool: %w", err))
}
@@ -827,7 +881,7 @@ func (c *sentinelFailover) MasterAddr(ctx context.Context) (string, error) {
if sentinel != nil {
addr, err := c.getMasterAddr(ctx, sentinel)
if err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
if isContextError(ctx.Err()) {
return "", err
}
// Continue on other errors
@@ -845,7 +899,7 @@ func (c *sentinelFailover) MasterAddr(ctx context.Context) (string, error) {
addr, err := c.getMasterAddr(ctx, c.sentinel)
if err != nil {
_ = c.closeSentinel()
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
if isContextError(ctx.Err()) {
return "", err
}
// Continue on other errors
@@ -904,22 +958,7 @@ func (c *sentinelFailover) MasterAddr(ctx context.Context) (string, error) {
for err := range errCh {
errs = append(errs, err)
}
return "", fmt.Errorf("redis: all sentinels specified in configuration are unreachable: %s", joinErrors(errs))
}
func joinErrors(errs []error) string {
if len(errs) == 0 {
return ""
}
if len(errs) == 1 {
return errs[0].Error()
}
b := []byte(errs[0].Error())
for _, err := range errs[1:] {
b = append(b, '\n')
b = append(b, err.Error()...)
}
return util.BytesToString(b)
return "", fmt.Errorf("redis: all sentinels specified in configuration are unreachable: %w", errors.Join(errs...))
}
func (c *sentinelFailover) replicaAddrs(ctx context.Context, useDisconnected bool) ([]string, error) {
@@ -930,7 +969,7 @@ func (c *sentinelFailover) replicaAddrs(ctx context.Context, useDisconnected boo
if sentinel != nil {
addrs, err := c.getReplicaAddrs(ctx, sentinel)
if err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
if isContextError(ctx.Err()) {
return nil, err
}
// Continue on other errors
@@ -948,7 +987,7 @@ func (c *sentinelFailover) replicaAddrs(ctx context.Context, useDisconnected boo
addrs, err := c.getReplicaAddrs(ctx, c.sentinel)
if err != nil {
_ = c.closeSentinel()
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
if isContextError(ctx.Err()) {
return nil, err
}
// Continue on other errors
@@ -970,7 +1009,7 @@ func (c *sentinelFailover) replicaAddrs(ctx context.Context, useDisconnected boo
replicas, err := sentinel.Replicas(ctx, c.opt.MasterName).Result()
if err != nil {
_ = sentinel.Close()
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
if isContextError(ctx.Err()) {
return nil, err
}
internal.Logger.Printf(ctx, "sentinel: Replicas master=%q failed: %s",
+142 -9
View File
@@ -6,6 +6,8 @@ import (
"github.com/redis/go-redis/v9/internal/hashtag"
)
// SetCmdable is an interface for Redis set commands.
// Sets are unordered collections of unique strings.
type SetCmdable interface {
SAdd(ctx context.Context, key string, members ...interface{}) *IntCmd
SCard(ctx context.Context, key string) *IntCmd
@@ -29,8 +31,12 @@ type SetCmdable interface {
SUnionStore(ctx context.Context, destination string, keys ...string) *IntCmd
}
//------------------------------------------------------------------------------
// Returns the number of elements that were added to the set, not including all
// the elements already present in the set.
//
// For more information about the command please refer to [SADD].
//
// [SADD]: (https://redis.io/docs/latest/commands/sadd/)
func (c cmdable) SAdd(ctx context.Context, key string, members ...interface{}) *IntCmd {
args := make([]interface{}, 2, 2+len(members))
args[0] = "sadd"
@@ -41,12 +47,25 @@ func (c cmdable) SAdd(ctx context.Context, key string, members ...interface{}) *
return cmd
}
// Returns the set cardinality (number of elements) of the set stored at key.
// Returns 0 if key does not exist.
//
// For more information about the command please refer to [SCARD].
//
// [SCARD]: (https://redis.io/docs/latest/commands/scard/)
func (c cmdable) SCard(ctx context.Context, key string) *IntCmd {
cmd := NewIntCmd(ctx, "scard", key)
_ = c(ctx, cmd)
return cmd
}
// Returns the members of the set resulting from the difference between the first set
// and all the successive sets.
// Keys that do not exist are considered to be empty sets.
//
// For more information about the command please refer to [SDIFF].
//
// [SDIFF]: (https://redis.io/docs/latest/commands/sdiff/)
func (c cmdable) SDiff(ctx context.Context, keys ...string) *StringSliceCmd {
args := make([]interface{}, 1+len(keys))
args[0] = "sdiff"
@@ -58,6 +77,13 @@ func (c cmdable) SDiff(ctx context.Context, keys ...string) *StringSliceCmd {
return cmd
}
// Stores the members of the set resulting from the difference between the first set
// and all the successive sets into destination.
// If destination already exists, it is overwritten.
//
// For more information about the command please refer to [SDIFFSTORE].
//
// [SDIFFSTORE]: (https://redis.io/docs/latest/commands/sdiffstore/)
func (c cmdable) SDiffStore(ctx context.Context, destination string, keys ...string) *IntCmd {
args := make([]interface{}, 2+len(keys))
args[0] = "sdiffstore"
@@ -70,6 +96,13 @@ func (c cmdable) SDiffStore(ctx context.Context, destination string, keys ...str
return cmd
}
// Returns the members of the set resulting from the intersection of all the given sets.
// Keys that do not exist are considered to be empty sets.
// With one of the keys being an empty set, the resulting set is also empty.
//
// For more information about the command please refer to [SINTER].
//
// [SINTER]: (https://redis.io/docs/latest/commands/sinter/)
func (c cmdable) SInter(ctx context.Context, keys ...string) *StringSliceCmd {
args := make([]interface{}, 1+len(keys))
args[0] = "sinter"
@@ -81,6 +114,16 @@ func (c cmdable) SInter(ctx context.Context, keys ...string) *StringSliceCmd {
return cmd
}
// Returns the cardinality of the set resulting from the intersection of all the given sets.
// Keys that do not exist are considered to be empty sets.
// With one of the keys being an empty set, the resulting set is also empty.
//
// The limit parameter sets an upper bound on the number of results returned.
// If limit is 0, no limit is applied.
//
// For more information about the command please refer to [SINTERCARD].
//
// [SINTERCARD]: (https://redis.io/docs/latest/commands/sintercard/)
func (c cmdable) SInterCard(ctx context.Context, limit int64, keys ...string) *IntCmd {
numKeys := len(keys)
args := make([]interface{}, 4+numKeys)
@@ -96,6 +139,13 @@ func (c cmdable) SInterCard(ctx context.Context, limit int64, keys ...string) *I
return cmd
}
// Stores the members of the set resulting from the intersection of all the given sets
// into destination.
// If destination already exists, it is overwritten.
//
// For more information about the command please refer to [SINTERSTORE].
//
// [SINTERSTORE]: (https://redis.io/docs/latest/commands/sinterstore/)
func (c cmdable) SInterStore(ctx context.Context, destination string, keys ...string) *IntCmd {
args := make([]interface{}, 2+len(keys))
args[0] = "sinterstore"
@@ -108,13 +158,26 @@ func (c cmdable) SInterStore(ctx context.Context, destination string, keys ...st
return cmd
}
// Returns if member is a member of the set stored at key.
// Returns true if the element is a member of the set, false if it is not a member
// or if key does not exist.
//
// For more information about the command please refer to [SISMEMBER].
//
// [SISMEMBER]: (https://redis.io/docs/latest/commands/sismember/)
func (c cmdable) SIsMember(ctx context.Context, key string, member interface{}) *BoolCmd {
cmd := NewBoolCmd(ctx, "sismember", key, member)
_ = c(ctx, cmd)
return cmd
}
// SMIsMember Redis `SMISMEMBER key member [member ...]` command.
// Returns whether each member is a member of the set stored at key.
// For each member, returns true if the element is a member of the set, false if it is not
// a member or if key does not exist.
//
// For more information about the command please refer to [SMISMEMBER].
//
// [SMISMEMBER]: (https://redis.io/docs/latest/commands/smismember/)
func (c cmdable) SMIsMember(ctx context.Context, key string, members ...interface{}) *BoolSliceCmd {
args := make([]interface{}, 2, 2+len(members))
args[0] = "smismember"
@@ -125,54 +188,100 @@ func (c cmdable) SMIsMember(ctx context.Context, key string, members ...interfac
return cmd
}
// SMembers Redis `SMEMBERS key` command output as a slice.
// Returns all the members of the set value stored at key.
// Returns an empty slice if key does not exist.
//
// For more information about the command please refer to [SMEMBERS].
//
// [SMEMBERS]: (https://redis.io/docs/latest/commands/smembers/)
func (c cmdable) SMembers(ctx context.Context, key string) *StringSliceCmd {
cmd := NewStringSliceCmd(ctx, "smembers", key)
_ = c(ctx, cmd)
return cmd
}
// SMembersMap Redis `SMEMBERS key` command output as a map.
// Returns all the members of the set value stored at key as a map.
// Returns an empty map if key does not exist.
//
// For more information about the command please refer to [SMEMBERS].
//
// [SMEMBERS]: (https://redis.io/docs/latest/commands/smembers/)
func (c cmdable) SMembersMap(ctx context.Context, key string) *StringStructMapCmd {
cmd := NewStringStructMapCmd(ctx, "smembers", key)
_ = c(ctx, cmd)
return cmd
}
// Moves member from the set at source to the set at destination.
// This operation is atomic. In every given moment the element will appear to be a member
// of source or destination for other clients.
//
// For more information about the command please refer to [SMOVE].
//
// [SMOVE]: (https://redis.io/docs/latest/commands/smove/)
func (c cmdable) SMove(ctx context.Context, source, destination string, member interface{}) *BoolCmd {
cmd := NewBoolCmd(ctx, "smove", source, destination, member)
_ = c(ctx, cmd)
return cmd
}
// SPop Redis `SPOP key` command.
// Removes and returns one or more random members from the set value stored at key.
// This version returns a single random member.
//
// For more information about the command please refer to [SPOP].
//
// [SPOP]: (https://redis.io/docs/latest/commands/spop/)
func (c cmdable) SPop(ctx context.Context, key string) *StringCmd {
cmd := NewStringCmd(ctx, "spop", key)
_ = c(ctx, cmd)
return cmd
}
// SPopN Redis `SPOP key count` command.
// Removes and returns one or more random members from the set value stored at key.
// This version returns up to count random members.
//
// For more information about the command please refer to [SPOP].
//
// [SPOP]: (https://redis.io/docs/latest/commands/spop/)
func (c cmdable) SPopN(ctx context.Context, key string, count int64) *StringSliceCmd {
cmd := NewStringSliceCmd(ctx, "spop", key, count)
_ = c(ctx, cmd)
return cmd
}
// SRandMember Redis `SRANDMEMBER key` command.
// Returns a random member from the set value stored at key.
// This version returns a single random member without removing it.
//
// For more information about the command please refer to [SRANDMEMBER].
//
// [SRANDMEMBER]: (https://redis.io/docs/latest/commands/srandmember/)
func (c cmdable) SRandMember(ctx context.Context, key string) *StringCmd {
cmd := NewStringCmd(ctx, "srandmember", key)
_ = c(ctx, cmd)
return cmd
}
// SRandMemberN Redis `SRANDMEMBER key count` command.
// Returns an array of random members from the set value stored at key.
// This version returns up to count random members without removing them.
// When called with a positive count, returns distinct elements.
// When called with a negative count, allows for repeated elements.
//
// For more information about the command please refer to [SRANDMEMBER].
//
// [SRANDMEMBER]: (https://redis.io/docs/latest/commands/srandmember/)
func (c cmdable) SRandMemberN(ctx context.Context, key string, count int64) *StringSliceCmd {
cmd := NewStringSliceCmd(ctx, "srandmember", key, count)
_ = c(ctx, cmd)
return cmd
}
// Removes the specified members from the set stored at key.
// Specified members that are not a member of this set are ignored.
// If key does not exist, it is treated as an empty set and this command returns 0.
//
// For more information about the command please refer to [SREM].
//
// [SREM]: (https://redis.io/docs/latest/commands/srem/)
func (c cmdable) SRem(ctx context.Context, key string, members ...interface{}) *IntCmd {
args := make([]interface{}, 2, 2+len(members))
args[0] = "srem"
@@ -183,6 +292,12 @@ func (c cmdable) SRem(ctx context.Context, key string, members ...interface{}) *
return cmd
}
// Returns the members of the set resulting from the union of all the given sets.
// Keys that do not exist are considered to be empty sets.
//
// For more information about the command please refer to [SUNION].
//
// [SUNION]: (https://redis.io/docs/latest/commands/sunion/)
func (c cmdable) SUnion(ctx context.Context, keys ...string) *StringSliceCmd {
args := make([]interface{}, 1+len(keys))
args[0] = "sunion"
@@ -194,6 +309,13 @@ func (c cmdable) SUnion(ctx context.Context, keys ...string) *StringSliceCmd {
return cmd
}
// Stores the members of the set resulting from the union of all the given sets
// into destination.
// If destination already exists, it is overwritten.
//
// For more information about the command please refer to [SUNIONSTORE].
//
// [SUNIONSTORE]: (https://redis.io/docs/latest/commands/sunionstore/)
func (c cmdable) SUnionStore(ctx context.Context, destination string, keys ...string) *IntCmd {
args := make([]interface{}, 2+len(keys))
args[0] = "sunionstore"
@@ -206,6 +328,17 @@ func (c cmdable) SUnionStore(ctx context.Context, destination string, keys ...st
return cmd
}
// Incrementally iterates the set elements stored at key.
// This is a cursor-based iterator that allows scanning large sets efficiently.
//
// Parameters:
// - cursor: The cursor value for the iteration (use 0 to start a new scan)
// - match: Optional pattern to match elements (empty string means no pattern)
// - count: Optional hint about how many elements to return per iteration
//
// For more information about the command please refer to [SSCAN].
//
// [SSCAN]: (https://redis.io/docs/latest/commands/sscan/)
func (c cmdable) SScan(ctx context.Context, key string, cursor uint64, match string, count int64) *ScanCmd {
args := []interface{}{"sscan", key, cursor}
if match != "" {
+15
View File
@@ -479,10 +479,16 @@ func (c cmdable) zRangeBy(ctx context.Context, zcmd, key string, opt *ZRangeBy,
return cmd
}
// ZRangeByScore returns members in a sorted set within a range of scores.
//
// Deprecated: Use ZRangeArgs with ByScore option instead as of Redis 6.2.0.
func (c cmdable) ZRangeByScore(ctx context.Context, key string, opt *ZRangeBy) *StringSliceCmd {
return c.zRangeBy(ctx, "zrangebyscore", key, opt, false)
}
// ZRangeByLex returns members in a sorted set within a lexicographical range.
//
// Deprecated: Use ZRangeArgs with ByLex option instead as of Redis 6.2.0.
func (c cmdable) ZRangeByLex(ctx context.Context, key string, opt *ZRangeBy) *StringSliceCmd {
return c.zRangeBy(ctx, "zrangebylex", key, opt, false)
}
@@ -559,6 +565,9 @@ func (c cmdable) ZRemRangeByLex(ctx context.Context, key, min, max string) *IntC
return cmd
}
// ZRevRange returns members in a sorted set within a range of indexes in reverse order.
//
// Deprecated: Use ZRangeArgs with Rev option instead as of Redis 6.2.0.
func (c cmdable) ZRevRange(ctx context.Context, key string, start, stop int64) *StringSliceCmd {
cmd := NewStringSliceCmd(ctx, "zrevrange", key, start, stop)
_ = c(ctx, cmd)
@@ -588,10 +597,16 @@ func (c cmdable) zRevRangeBy(ctx context.Context, zcmd, key string, opt *ZRangeB
return cmd
}
// ZRevRangeByScore returns members in a sorted set within a range of scores in reverse order.
//
// Deprecated: Use ZRangeArgs with Rev and ByScore options instead as of Redis 6.2.0.
func (c cmdable) ZRevRangeByScore(ctx context.Context, key string, opt *ZRangeBy) *StringSliceCmd {
return c.zRevRangeBy(ctx, "zrevrangebyscore", key, opt)
}
// ZRevRangeByLex returns members in a sorted set within a lexicographical range in reverse order.
//
// Deprecated: Use ZRangeArgs with Rev and ByLex options instead as of Redis 6.2.0.
func (c cmdable) ZRevRangeByLex(ctx context.Context, key string, opt *ZRangeBy) *StringSliceCmd {
return c.zRevRangeBy(ctx, "zrevrangebylex", key, opt)
}
+88 -12
View File
@@ -2,7 +2,11 @@ package redis
import (
"context"
"strconv"
"strings"
"time"
"github.com/redis/go-redis/v9/internal/otel"
)
type StreamCmdable interface {
@@ -43,6 +47,7 @@ type StreamCmdable interface {
XInfoStream(ctx context.Context, key string) *XInfoStreamCmd
XInfoStreamFull(ctx context.Context, key string, count int) *XInfoStreamFullCmd
XInfoConsumers(ctx context.Context, key string, group string) *XInfoConsumersCmd
XCfgSet(ctx context.Context, a *XCfgSetArgs) *StatusCmd
}
// XAddArgs accepts values in the following formats:
@@ -52,47 +57,69 @@ type StreamCmdable interface {
//
// Note that map will not preserve the order of key-value pairs.
// MaxLen/MaxLenApprox and MinID are in conflict, only one of them can be used.
//
// For idempotent production (at-most-once production):
// - ProducerID: A unique identifier for the producer (required for both IDMP and IDMPAUTO)
// - IdempotentID: A unique identifier for the message (used with IDMP)
// - IdempotentAuto: If true, Redis will auto-generate an idempotent ID based on message content (IDMPAUTO)
//
// ProducerID and IdempotentID are mutually exclusive with IdempotentAuto.
// When using idempotent production, ID must be "*" or empty.
type XAddArgs struct {
Stream string
NoMkStream bool
MaxLen int64 // MAXLEN N
MinID string
// Approx causes MaxLen and MinID to use "~" matcher (instead of "=").
Approx bool
Limit int64
Mode string
ID string
Values interface{}
Approx bool
Limit int64
Mode string
ID string
Values interface{}
ProducerID string // Producer ID for idempotent production (IDMP or IDMPAUTO)
IdempotentID string // Idempotent ID for IDMP
IdempotentAuto bool // Use IDMPAUTO to auto-generate idempotent ID based on content
}
func (c cmdable) XAdd(ctx context.Context, a *XAddArgs) *StringCmd {
args := make([]interface{}, 0, 11)
args := make([]interface{}, 0, 15)
args = append(args, "xadd", a.Stream)
if a.NoMkStream {
args = append(args, "nomkstream")
}
if a.Mode != "" {
args = append(args, a.Mode)
}
if a.ProducerID != "" {
if a.IdempotentAuto {
// IDMPAUTO pid
args = append(args, "idmpauto", a.ProducerID)
} else if a.IdempotentID != "" {
// IDMP pid iid
args = append(args, "idmp", a.ProducerID, a.IdempotentID)
}
}
switch {
case a.MaxLen > 0:
if a.Approx {
args = append(args, "maxlen", "~", a.MaxLen)
} else {
args = append(args, "maxlen", a.MaxLen)
args = append(args, "maxlen", "=", a.MaxLen)
}
case a.MinID != "":
if a.Approx {
args = append(args, "minid", "~", a.MinID)
} else {
args = append(args, "minid", a.MinID)
args = append(args, "minid", "=", a.MinID)
}
}
if a.Limit > 0 {
args = append(args, "limit", a.Limit)
}
if a.Mode != "" {
args = append(args, a.Mode)
}
if a.ID != "" {
args = append(args, a.ID)
} else {
@@ -299,6 +326,26 @@ func (c cmdable) XReadGroup(ctx context.Context, a *XReadGroupArgs) *XStreamSlic
}
cmd.SetFirstKeyPos(keyPos)
_ = c(ctx, cmd)
// Record stream lag for each message (if command succeeded)
if cmd.Err() == nil {
streams := cmd.Val()
for _, stream := range streams {
for _, msg := range stream.Messages {
// Parse message ID to extract timestamp (format: "millisecondsTime-sequenceNumber")
if parts := strings.SplitN(msg.ID, "-", 2); len(parts) == 2 {
if timestampMs, err := strconv.ParseInt(parts[0], 10, 64); err == nil {
// Calculate lag (time since message was created)
messageTime := time.Unix(0, timestampMs*int64(time.Millisecond))
lag := time.Since(messageTime)
// Record lag metric
otel.RecordStreamLag(ctx, lag, nil, stream.Stream, a.Group, a.Consumer)
}
}
}
}
}
return cmd
}
@@ -429,6 +476,8 @@ func (c cmdable) xTrim(
args = append(args, "xtrim", key, strategy)
if approx {
args = append(args, "~")
} else {
args = append(args, "=")
}
args = append(args, threshold)
if limit > 0 {
@@ -466,6 +515,8 @@ func (c cmdable) xTrimMode(
args = append(args, "xtrim", key, strategy)
if approx {
args = append(args, "~")
} else {
args = append(args, "=")
}
args = append(args, threshold)
if limit > 0 {
@@ -523,3 +574,28 @@ func (c cmdable) XInfoStreamFull(ctx context.Context, key string, count int) *XI
_ = c(ctx, cmd)
return cmd
}
// XCfgSetArgs represents the arguments for the XCFGSET command.
// Duration is the duration, in seconds, that Redis keeps each idempotent ID.
// MaxSize is the maximum number of most recent idempotent IDs that Redis keeps for each producer ID.
type XCfgSetArgs struct {
Stream string
Duration int64
MaxSize int64
}
// XCfgSet sets the idempotent production configuration for a stream.
// XCFGSET key [IDMP-DURATION duration] [IDMP-MAXSIZE maxsize]
func (c cmdable) XCfgSet(ctx context.Context, a *XCfgSetArgs) *StatusCmd {
args := make([]interface{}, 0, 6)
args = append(args, "xcfgset", a.Stream)
if a.Duration > 0 {
args = append(args, "idmp-duration", a.Duration)
}
if a.MaxSize > 0 {
args = append(args, "idmp-maxsize", a.MaxSize)
}
cmd := NewStatusCmd(ctx, args...)
_ = c(ctx, cmd)
return cmd
}
+9 -2
View File
@@ -143,6 +143,9 @@ func (c cmdable) GetRange(ctx context.Context, key string, start, end int64) *St
return cmd
}
// GetSet returns the old value stored at key and sets it to the new value.
//
// Deprecated: Use SetArgs with Get option instead as of Redis 6.2.0.
func (c cmdable) GetSet(ctx context.Context, key string, value interface{}) *StringCmd {
cmd := NewStringCmd(ctx, "getset", key, value)
_ = c(ctx, cmd)
@@ -415,14 +418,18 @@ func (c cmdable) SetArgs(ctx context.Context, key string, value interface{}, a S
return cmd
}
// SetEx Redis `SETEx key expiration value` command.
// SetEx sets the value and expiration of a key.
//
// Deprecated: Use Set with expiration instead as of Redis 2.6.12.
func (c cmdable) SetEx(ctx context.Context, key string, value interface{}, expiration time.Duration) *StatusCmd {
cmd := NewStatusCmd(ctx, "setex", key, formatSec(ctx, expiration), value)
_ = c(ctx, cmd)
return cmd
}
// SetNX Redis `SET key value [expiration] NX` command.
// SetNX sets the value of a key only if the key does not exist.
//
// Deprecated: Use Set with NX option instead as of Redis 2.6.12.
//
// Zero expiration means the key has no expiration time.
// KeepTTL is a Redis KEEPTTL option to keep existing TTL, it requires your redis-server version >= 6.0,
+34 -7
View File
@@ -2,9 +2,9 @@ package redis
import (
"context"
"strconv"
"github.com/redis/go-redis/v9/internal/proto"
"github.com/redis/go-redis/v9/internal/util"
)
type TimeseriesCmdable interface {
@@ -96,6 +96,8 @@ const (
VarP
VarS
Twa
CountNaN
CountAll
)
func (a Aggregator) String() string {
@@ -128,6 +130,10 @@ func (a Aggregator) String() string {
return "VAR.S"
case Twa:
return "TWA"
case CountNaN:
return "COUNTNAN"
case CountAll:
return "COUNTALL"
default:
return ""
}
@@ -486,8 +492,9 @@ type TSTimestampValueCmd struct {
func newTSTimestampValueCmd(ctx context.Context, args ...interface{}) *TSTimestampValueCmd {
return &TSTimestampValueCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeTSTimestampValue,
},
}
}
@@ -524,7 +531,7 @@ func (cmd *TSTimestampValueCmd) readReply(rd *proto.Reader) (err error) {
return err
}
cmd.val.Timestamp = timestamp
cmd.val.Value, err = strconv.ParseFloat(value, 64)
cmd.val.Value, err = util.ParseStringToFloat(value)
if err != nil {
return err
}
@@ -533,6 +540,13 @@ func (cmd *TSTimestampValueCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *TSTimestampValueCmd) Clone() Cmder {
return &TSTimestampValueCmd{
baseCmd: cmd.cloneBaseCmd(),
val: cmd.val, // TSTimestampValue is a simple struct, can be copied directly
}
}
// TSInfo - Returns information about a time-series key.
// For more information - https://redis.io/commands/ts.info/
func (c cmdable) TSInfo(ctx context.Context, key string) *MapStringInterfaceCmd {
@@ -704,8 +718,9 @@ type TSTimestampValueSliceCmd struct {
func newTSTimestampValueSliceCmd(ctx context.Context, args ...interface{}) *TSTimestampValueSliceCmd {
return &TSTimestampValueSliceCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
ctx: ctx,
args: args,
cmdType: CmdTypeTSTimestampValueSlice,
},
}
}
@@ -743,7 +758,7 @@ func (cmd *TSTimestampValueSliceCmd) readReply(rd *proto.Reader) (err error) {
return err
}
cmd.val[i].Timestamp = timestamp
cmd.val[i].Value, err = strconv.ParseFloat(value, 64)
cmd.val[i].Value, err = util.ParseStringToFloat(value)
if err != nil {
return err
}
@@ -752,6 +767,18 @@ func (cmd *TSTimestampValueSliceCmd) readReply(rd *proto.Reader) (err error) {
return nil
}
func (cmd *TSTimestampValueSliceCmd) Clone() Cmder {
var val []TSTimestampValue
if cmd.val != nil {
val = make([]TSTimestampValue, len(cmd.val))
copy(val, cmd.val)
}
return &TSTimestampValueSliceCmd{
baseCmd: cmd.cloneBaseCmd(),
val: val,
}
}
// TSMRange - Returns a range of samples from multiple time-series keys.
// For more information - https://redis.io/commands/ts.mrange/
func (c cmdable) TSMRange(ctx context.Context, fromTimestamp int, toTimestamp int, filterExpr []string) *MapStringSliceInterfaceCmd {
+96 -57
View File
@@ -8,6 +8,7 @@ import (
"github.com/redis/go-redis/v9/auth"
"github.com/redis/go-redis/v9/maintnotifications"
"github.com/redis/go-redis/v9/push"
)
// UniversalOptions information is required by UniversalClient to establish
@@ -57,7 +58,18 @@ type UniversalOptions struct {
MinRetryBackoff time.Duration
MaxRetryBackoff time.Duration
DialTimeout time.Duration
DialTimeout time.Duration
// DialerRetries is the maximum number of retry attempts when dialing fails.
//
// default: 5
DialerRetries int
// DialerRetryTimeout is the backoff duration between retry attempts.
//
// default: 100 milliseconds
DialerRetryTimeout time.Duration
ReadTimeout time.Duration
WriteTimeout time.Duration
ContextTimeoutEnabled bool
@@ -79,13 +91,19 @@ type UniversalOptions struct {
// PoolFIFO uses FIFO mode for each node connection pool GET/PUT (default LIFO).
PoolFIFO bool
PoolSize int
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
PoolSize int
// MaxConcurrentDials is the maximum number of concurrent connection creation goroutines.
// If <= 0, defaults to PoolSize. If > PoolSize, it will be capped at PoolSize.
MaxConcurrentDials int
PoolTimeout time.Duration
MinIdleConns int
MaxIdleConns int
MaxActiveConns int
ConnMaxIdleTime time.Duration
ConnMaxLifetime time.Duration
ConnMaxLifetimeJitter time.Duration
TLSConfig *tls.Config
@@ -121,6 +139,10 @@ type UniversalOptions struct {
UnstableResp3 bool
// PushNotificationProcessor is the processor for handling push notifications.
// If nil, a default processor will be created for RESP3 connections.
PushNotificationProcessor push.NotificationProcessor
// IsClusterMode can be used when only one Addrs is provided (e.g. Elasticache supports setting up cluster mode with configuration endpoint).
IsClusterMode bool
@@ -156,32 +178,37 @@ func (o *UniversalOptions) Cluster() *ClusterOptions {
MinRetryBackoff: o.MinRetryBackoff,
MaxRetryBackoff: o.MaxRetryBackoff,
DialTimeout: o.DialTimeout,
ReadTimeout: o.ReadTimeout,
WriteTimeout: o.WriteTimeout,
DialTimeout: o.DialTimeout,
DialerRetries: o.DialerRetries,
DialerRetryTimeout: o.DialerRetryTimeout,
ReadTimeout: o.ReadTimeout,
WriteTimeout: o.WriteTimeout,
ContextTimeoutEnabled: o.ContextTimeoutEnabled,
ReadBufferSize: o.ReadBufferSize,
WriteBufferSize: o.WriteBufferSize,
PoolFIFO: o.PoolFIFO,
PoolSize: o.PoolSize,
PoolTimeout: o.PoolTimeout,
MinIdleConns: o.MinIdleConns,
MaxIdleConns: o.MaxIdleConns,
MaxActiveConns: o.MaxActiveConns,
ConnMaxIdleTime: o.ConnMaxIdleTime,
ConnMaxLifetime: o.ConnMaxLifetime,
PoolFIFO: o.PoolFIFO,
PoolSize: o.PoolSize,
MaxConcurrentDials: o.MaxConcurrentDials,
PoolTimeout: o.PoolTimeout,
MinIdleConns: o.MinIdleConns,
MaxIdleConns: o.MaxIdleConns,
MaxActiveConns: o.MaxActiveConns,
ConnMaxIdleTime: o.ConnMaxIdleTime,
ConnMaxLifetime: o.ConnMaxLifetime,
ConnMaxLifetimeJitter: o.ConnMaxLifetimeJitter,
TLSConfig: o.TLSConfig,
DisableIdentity: o.DisableIdentity,
DisableIndentity: o.DisableIndentity,
IdentitySuffix: o.IdentitySuffix,
FailingTimeoutSeconds: o.FailingTimeoutSeconds,
UnstableResp3: o.UnstableResp3,
MaintNotificationsConfig: o.MaintNotificationsConfig,
DisableIdentity: o.DisableIdentity,
DisableIndentity: o.DisableIndentity,
IdentitySuffix: o.IdentitySuffix,
FailingTimeoutSeconds: o.FailingTimeoutSeconds,
UnstableResp3: o.UnstableResp3,
PushNotificationProcessor: o.PushNotificationProcessor,
MaintNotificationsConfig: o.MaintNotificationsConfig,
}
}
@@ -217,31 +244,37 @@ func (o *UniversalOptions) Failover() *FailoverOptions {
MinRetryBackoff: o.MinRetryBackoff,
MaxRetryBackoff: o.MaxRetryBackoff,
DialTimeout: o.DialTimeout,
ReadTimeout: o.ReadTimeout,
WriteTimeout: o.WriteTimeout,
DialTimeout: o.DialTimeout,
DialerRetries: o.DialerRetries,
DialerRetryTimeout: o.DialerRetryTimeout,
ReadTimeout: o.ReadTimeout,
WriteTimeout: o.WriteTimeout,
ContextTimeoutEnabled: o.ContextTimeoutEnabled,
ReadBufferSize: o.ReadBufferSize,
WriteBufferSize: o.WriteBufferSize,
PoolFIFO: o.PoolFIFO,
PoolSize: o.PoolSize,
PoolTimeout: o.PoolTimeout,
MinIdleConns: o.MinIdleConns,
MaxIdleConns: o.MaxIdleConns,
MaxActiveConns: o.MaxActiveConns,
ConnMaxIdleTime: o.ConnMaxIdleTime,
ConnMaxLifetime: o.ConnMaxLifetime,
PoolFIFO: o.PoolFIFO,
PoolSize: o.PoolSize,
MaxConcurrentDials: o.MaxConcurrentDials,
PoolTimeout: o.PoolTimeout,
MinIdleConns: o.MinIdleConns,
MaxIdleConns: o.MaxIdleConns,
MaxActiveConns: o.MaxActiveConns,
ConnMaxIdleTime: o.ConnMaxIdleTime,
ConnMaxLifetime: o.ConnMaxLifetime,
ConnMaxLifetimeJitter: o.ConnMaxLifetimeJitter,
TLSConfig: o.TLSConfig,
ReplicaOnly: o.ReadOnly,
DisableIdentity: o.DisableIdentity,
DisableIndentity: o.DisableIndentity,
IdentitySuffix: o.IdentitySuffix,
UnstableResp3: o.UnstableResp3,
DisableIdentity: o.DisableIdentity,
DisableIndentity: o.DisableIndentity,
IdentitySuffix: o.IdentitySuffix,
UnstableResp3: o.UnstableResp3,
PushNotificationProcessor: o.PushNotificationProcessor,
// Note: MaintNotificationsConfig not supported for FailoverOptions
}
}
@@ -271,30 +304,36 @@ func (o *UniversalOptions) Simple() *Options {
MinRetryBackoff: o.MinRetryBackoff,
MaxRetryBackoff: o.MaxRetryBackoff,
DialTimeout: o.DialTimeout,
ReadTimeout: o.ReadTimeout,
WriteTimeout: o.WriteTimeout,
DialTimeout: o.DialTimeout,
DialerRetries: o.DialerRetries,
DialerRetryTimeout: o.DialerRetryTimeout,
ReadTimeout: o.ReadTimeout,
WriteTimeout: o.WriteTimeout,
ContextTimeoutEnabled: o.ContextTimeoutEnabled,
ReadBufferSize: o.ReadBufferSize,
WriteBufferSize: o.WriteBufferSize,
PoolFIFO: o.PoolFIFO,
PoolSize: o.PoolSize,
PoolTimeout: o.PoolTimeout,
MinIdleConns: o.MinIdleConns,
MaxIdleConns: o.MaxIdleConns,
MaxActiveConns: o.MaxActiveConns,
ConnMaxIdleTime: o.ConnMaxIdleTime,
ConnMaxLifetime: o.ConnMaxLifetime,
PoolFIFO: o.PoolFIFO,
PoolSize: o.PoolSize,
MaxConcurrentDials: o.MaxConcurrentDials,
PoolTimeout: o.PoolTimeout,
MinIdleConns: o.MinIdleConns,
MaxIdleConns: o.MaxIdleConns,
MaxActiveConns: o.MaxActiveConns,
ConnMaxIdleTime: o.ConnMaxIdleTime,
ConnMaxLifetime: o.ConnMaxLifetime,
ConnMaxLifetimeJitter: o.ConnMaxLifetimeJitter,
TLSConfig: o.TLSConfig,
DisableIdentity: o.DisableIdentity,
DisableIndentity: o.DisableIndentity,
IdentitySuffix: o.IdentitySuffix,
UnstableResp3: o.UnstableResp3,
MaintNotificationsConfig: o.MaintNotificationsConfig,
DisableIdentity: o.DisableIdentity,
DisableIndentity: o.DisableIndentity,
IdentitySuffix: o.IdentitySuffix,
UnstableResp3: o.UnstableResp3,
PushNotificationProcessor: o.PushNotificationProcessor,
MaintNotificationsConfig: o.MaintNotificationsConfig,
}
}
+1 -1
View File
@@ -2,5 +2,5 @@ package redis
// Version is the current release version.
func Version() string {
return "9.17.2"
return "9.18.0"
}
+500
View File
@@ -0,0 +1,500 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.21.1] - 2026-04-08
### Added
- **`slackevents.ChannelType*` constants and `MessageEvent` helpers** — Added
`ChannelTypeChannel`, `ChannelTypeGroup`, `ChannelTypeIM`, `ChannelTypeMPIM` constants
and `IsChannel()`, `IsGroup()`, `IsIM()`, `IsMpIM()` methods on `MessageEvent` so
callers no longer need to compare raw strings.
### Fixed
- **Duplicate attachment/block serialization in `MsgOptionAttachments` / `MsgOptionBlocks`**
Attachments and blocks were serialized twice: once into typed struct fields (for the JSON
response-URL path) and again into `url.Values` (for the form POST path). Serialization for
the form path now happens inside `formSender.BuildRequestContext`, so each sender owns its
own marshalling. This fixes the long-standing FIXME and eliminates redundant `json.Marshal`
calls in the option functions. ([#1547])
> [!NOTE]
> `UnsafeApplyMsgOptions` returns `config.values` directly. After this change,
> `attachments` and `blocks` keys are no longer present in those values since
> marshalling is deferred to send time. This function is documented as unsupported.
## [0.21.0] - 2026-04-05
### Deprecated
- **`slackevents.ParseActionEvent`** — Cannot parse `block_actions` payloads (returns
unmarshalling error). Use `slack.InteractionCallback` with `json.Unmarshal` instead,
or `slack.InteractionCallbackParse` for HTTP requests. `InteractionCallback` handles
all interaction types. ([#596])
- **`slackevents.MessageAction`**, **`MessageActionEntity`**, **`MessageActionResponse`** —
Associated types that only support legacy `interactive_message` payloads.
### Removed
- **`IM` struct** — Removed the `IM` struct (and unused internal types `imChannel`,
`imResponseFull`). The `IsUserDeleted` field has been moved to `Conversation`, where it
is populated for IM-type conversations. Code using `IM` should switch to `Conversation`.
> [!NOTE]
> In practice no user should be affected — `IM` was never returned by any public API
> method in this library, so there was no way to obtain one outside of manual construction.
- **`Info.GetBotByID`, `GetUserByID`, `GetChannelByID`, `GetGroupByID`, `GetIMByID`** —
These methods were deprecated and returned `nil` unconditionally. They have been removed.
> [!WARNING]
> **Breaking change.** If you are calling any of these methods, remove those calls — they
> were already no-ops.
### Added
- **`admin.teams.settings.*` API support** — `AdminTeamsSettingsInfo`,
`AdminTeamsSettingsSetDefaultChannels`, `AdminTeamsSettingsSetDescription`,
`AdminTeamsSettingsSetDiscoverability`, `AdminTeamsSettingsSetIcon`, and
`AdminTeamsSettingsSetName`. Includes `TeamDiscoverability` enum with `Open`,
`InviteOnly`, `Closed`, and `Unlisted` variants. ([#960])
- **`OAuthOptionAPIURL` for package-level OAuth functions** — All package-level OAuth
functions (`GetOAuthV2Response`, `GetOpenIDConnectToken`, `RefreshOAuthV2Token`, etc.)
now accept variadic `OAuthOption` arguments. Use `OAuthOptionAPIURL(url)` to override
the default Slack API URL, enabling integration tests against a local HTTP server.
Existing callers are unaffected. ([#744])
- **`GetOpenIDConnectUserInfo` / `GetOpenIDConnectUserInfoContext`** — Returns identity
information about the user associated with the token via `openid.connect.userInfo`.
Complements the existing `GetOpenIDConnectToken` method. ([#967])
- **HTTP response headers** — Slack API response headers (e.g. `X-OAuth-Scopes`,
`X-Accepted-OAuth-Scopes`, `X-Ratelimit-*`) are now accessible. `AuthTestResponse`
exposes a `Header` field directly. For all other methods, use
`OptionOnResponseHeaders(func(method string, headers http.Header))` to register a
callback that fires after every API call. ([#1076])
- **`DNDOptionTeamID`** — `GetDNDInfo` and `GetDNDTeamInfo` now accept optional
`ParamOption` arguments. Use `DNDOptionTeamID("T...")` to pass `team_id`, which is
required after workspace migration (Slack returns `missing_argument` without it).
([#1157])
- **`UpdateUserGroupMembersList` / `UpdateUserGroupMembersListContext`** — Convenience
wrappers around `UpdateUserGroupMembers` that accept `[]string` instead of a
comma-separated string, enabling clean chaining with `GetUserGroupMembers`. ([#1172])
- **`SetUserProfile` / `SetUserProfileContext`** — Set multiple user profile fields in a
single API call by passing a `*UserProfile` struct to `users.profile.set`. Complements
the existing single-field methods (`SetUserRealName`, `SetUserCustomStatus`, etc.).
([#1158])
- **API warning callbacks** — Slack API responses may include a `warnings` field with
deprecation notices or usage hints. Use `OptionWarnings(func(warnings []string))` to
register a callback that receives these warnings. ([#1540])
- **RTM support for `user_status_changed`, `user_huddle_changed`, `user_profile_changed`
events** — these events are now mapped in `EventMapping` with dedicated structs
(`UserStatusChangedEvent`, `UserHuddleChangedEvent`, `UserProfileChangedEvent`).
Previously they triggered `UnmarshallingErrorEvent`. ([#1541])
- **RTM support for `sh_room_join`, `sh_room_leave`, `sh_room_update`, `channel_updated`
events** — Slack Call/Huddle room events and channel property updates are now mapped with
dedicated structs (`SHRoomJoinEvent`, `SHRoomLeaveEvent`, `SHRoomUpdateEvent`,
`ChannelUpdatedEvent`). ([#858])
- **`CacheTS` and `EventTS` fields on `UserChangeEvent`** — these fields were sent by Slack
but silently dropped during unmarshalling.
- **`workflows.featured` API support** — add, list, remove, and set featured workflows on
channels via `WorkflowsFeaturedAdd`, `WorkflowsFeaturedList`, `WorkflowsFeaturedRemove`,
and `WorkflowsFeaturedSet`
- **`IsConnectorBot` and `IsWorkflowBot` in `User`** — boolean flags for connector and
workflow bot users
- **`GuestInvitedBy` in `UserProfile`** — user ID of whoever invited a guest user
- **`Blocks` field on `MessageEvent`** — block data from webhook payloads is now directly
accessible via `event.Blocks` instead of only through `event.Message.Blocks`. ([#1257])
- **`Username` field on `User`** — Slack's interaction payloads (block_actions, shortcuts)
include a `username` field in the user object that was previously dropped during
unmarshalling. ([#1218])
- **`Blocks`, `Attachments`, `Files`, `Upload` fields on `AppMentionEvent`** — these fields
are sent by Slack in `app_mention` event payloads but were silently dropped. ([#961])
- **`HandleShortcut`, `HandleViewSubmission`, `HandleViewClosed` in socketmode handler** —
Level 3 handlers that dispatch `shortcut`/`message_action`, `view_submission`, and
`view_closed` interactions by `CallbackID`, matching the pattern of
`HandleInteractionBlockAction` and `HandleSlashCommand`. ([#1161])
- **`BlockFromJSON` / `MustBlockFromJSON`** — Create blocks from raw JSON strings, enabling
direct use of output from Slack's [Block Kit Builder](https://app.slack.com/block-kit-builder)
or quick adoption of new block types before the library adds typed support. The original
JSON is preserved through marshalling. ([#1497])
### Documentation
- **`ViewSubmissionResponse` constructors** — `NewClearViewSubmissionResponse`,
`NewUpdateViewSubmissionResponse`, `NewPushViewSubmissionResponse`, and
`NewErrorsViewSubmissionResponse` now have doc comments explaining the HTTP response
pattern (write JSON and return promptly) and the Socket Mode pattern (pass as Ack
payload). `NewErrorsViewSubmissionResponse` additionally documents that map keys must
be `BlockID`s of `InputBlock` elements. ([#726], [#1013])
- **Socket Mode examples**`examples/socketmode/` and `examples/socketmode_handler/` now
have doc comments explaining the two-token requirement: app-level token (`xapp-`) for the
WebSocket connection and bot token (`xoxb-`) for API calls. ([#941])
### Fixed
- **`UnknownBlock` round-trip data loss** — Unrecognized block types (e.g. new Slack block
types not yet supported by this library) now preserve their full JSON through
unmarshal/marshal cycles. Previously only `type` and `block_id` were retained, silently
discarding all other fields.
### Changed
- Adjusted some `admin` errors that started with uppercase to be lowercase per go
conventions.
> [!WARNING]
> **Breaking change.** If you are matching the error content in your code, this is a
> BREAKING CHANGE.
- **`WebhookMessage.UnfurlLinks` and `UnfurlMedia` are now `*bool`** — Previously these
were `bool` with `omitempty`, which meant `false` was silently stripped from the JSON
payload. Users could not explicitly disable link or media unfurling via webhooks. The
fields are now `*bool` so that `nil` (omit), `false`, and `true` all serialize correctly.
([#1231])
> [!WARNING]
> **Breaking change.** Code that sets these fields directly must be updated:
>
> ```go
> // Before
> msg := slack.WebhookMessage{UnfurlLinks: true}
>
> // After — use a helper or a variable
> t := true
> msg := slack.WebhookMessage{UnfurlLinks: &t}
> ```
>
> Leaving the fields unset (`nil`) preserves the previous default behavior — Slack's
> server-side defaults apply (`unfurl_links=false`, `unfurl_media=true`).
- **`User.Has2FA` is now `*bool`** — When using a bot token, Slack's `users.list` API omits
`has_2fa` entirely. With a plain `bool`, this was indistinguishable from explicitly `false`.
Now `nil` means absent/unknown, `false` means explicitly disabled, `true` means enabled.
([#1121])
> [!WARNING]
> **Breaking change.** Code that reads `Has2FA` must handle the pointer:
>
> ```go
> // Before
> if user.Has2FA { ... }
>
> // After
> if user.Has2FA != nil && *user.Has2FA { ... }
> ```
- **`ListReactions` now uses cursor-based pagination** — `ListReactionsParameters` replaces
`Count`/`Page` with `Cursor`/`Limit`, and `ListReactions`/`ListReactionsContext` now return
`([]ReactedItem, string, error)` where the string is the next cursor, instead of
`([]ReactedItem, *Paging, error)`. ([#825])
> [!WARNING]
> **Breaking change.** Both the parameters and return signature have changed:
>
> ```go
> // Before
> params := slack.NewListReactionsParameters()
> params.Count = 100
> params.Page = 2
> items, paging, err := api.ListReactions(params)
>
> // After
> params := slack.NewListReactionsParameters()
> params.Limit = 100
> items, nextCursor, err := api.ListReactions(params)
> // Use nextCursor for the next page: params.Cursor = nextCursor
> ```
- **`ListStars`/`GetStarred` now use cursor-based pagination** — `StarsParameters` replaces
`Count`/`Page` with `Cursor`/`Limit` (and adds `TeamID`), and `ListStars`/`ListStarsContext`/
`GetStarred`/`GetStarredContext` now return `string` (next cursor) instead of `*Paging`.
Slack's `stars.list` API no longer returns `paging` data — only `response_metadata.next_cursor`.
> [!WARNING]
> **Breaking change.** Both the parameters and return signature have changed:
>
> ```go
> // Before
> params := slack.NewStarsParameters()
> params.Count = 100
> params.Page = 2
> items, paging, err := api.ListStars(params)
>
> // After
> params := slack.NewStarsParameters()
> params.Limit = 100
> items, nextCursor, err := api.ListStars(params)
> // Use nextCursor for the next page: params.Cursor = nextCursor
> ```
- **`GetAccessLogs` now uses cursor-based pagination** — `AccessLogParameters` replaces
`Count`/`Page` with `Cursor`/`Limit` (and adds `Before`), and `GetAccessLogs`/
`GetAccessLogsContext` now return `string` (next cursor) instead of `*Paging`.
Slack's `team.accessLogs` API warns `use_cursor_pagination_instead` when using the old
parameters.
> [!WARNING]
> **Breaking change.** Both the parameters and return signature have changed:
>
> ```go
> // Before
> params := slack.NewAccessLogParameters()
> params.Count = 100
> params.Page = 2
> logins, paging, err := api.GetAccessLogs(params)
>
> // After
> params := slack.NewAccessLogParameters()
> params.Limit = 100
> logins, nextCursor, err := api.GetAccessLogs(params)
> // Use nextCursor for the next page: params.Cursor = nextCursor
> ```
### Fixed
- **Socket Mode: large Ack payloads no longer silently fail** — Two issues caused `Ack()`
payloads to be silently dropped by Slack. First, gorilla/websocket's default 4KB write
buffer fragmented messages into WebSocket continuation frames that Slack does not
reassemble. The library now uses a 32KB write buffer. Second, Slack silently drops
Socket Mode responses at or above 20KB — `Ack()`, `Send()`, and `SendCtx()` now return
an error when the serialized response reaches this limit. ([#1196])
> [!WARNING]
> **Breaking change.** `Ack()` and `Send()` now return `error`. Existing call sites that
> don't capture the return value continue to compile without changes.
- **`MsgOptionBlocks()` with no arguments now sends `blocks=[]`** — Previously, calling
`MsgOptionBlocks()` with no arguments or a nil spread was a silent no-op, making it
impossible to clear blocks from a message via `chat.update`. The Slack API requires an
explicit `blocks=[]` to reliably remove blocks. ([#1214])
> [!WARNING]
> **Breaking change.** `MsgOptionBlocks()` with no arguments now sends `blocks=[]` instead
> of being a no-op. If you were relying on this to be a no-op, remove the option entirely:
>
> ```go
> // Before — this was a no-op, now it sends blocks=[]
> api.PostMessage(ch, slack.MsgOptionBlocks(), slack.MsgOptionText("text", false))
>
> // After — omit MsgOptionBlocks entirely to not set blocks
> api.PostMessage(ch, slack.MsgOptionText("text", false))
> ```
- **`WorkflowButtonBlockElement` missing from `UnmarshalJSON`** — `workflow_button` blocks
now unmarshal correctly through `BlockElements`, `InputBlock`, and `Accessory` paths.
Also adds missing `multi_*_select` and `file_input` cases to `BlockElements.UnmarshalJSON`,
and fixes `toBlockElement` for `RichTextInputElement` and `WorkflowButtonElement`. ([#1539])
- **`NewBlockHeader` nil pointer dereference** — passing a nil text object no longer panics. ([#1236])
- **`ValidateUniqueBlockID` rejects empty block IDs** — multiple input blocks with no
explicit `block_id` set (empty string) were incorrectly flagged as duplicates, causing
`OpenView` to fail. ([#1184])
## [0.20.0] - 2026-03-21
> [!WARNING]
> `trigger_id` and `workflow_id` are NOT in any documentation or in any of the official
libraries, so exercise caution if you use these.
### Added
- **`workflow_id` and `trigger_id` in `Message`** — It seems that some types of messages,
e.g: `bot_message`, can carry `trigger_id` and `workflow_id`.
- **`RichTextQuote.Border` field** — optional border toggle (matches the docs now)
- **`RichTextPreformatted.Language` field** — enables syntax highlighting for preformatted
blocks
### Fixed
- **Remove embedding of `RichTextSection`**`RichTextQuote` and `RichTextPreformatted`
are now flattened as they should have always been. This is a breaking change for anyone
using these structs directly.
## [0.19.0] - 2026-03-04
### Added
- **Optional HTTP retry for Web API** — Retries are off by default. Enable with `OptionRetry(n)` for 429-only retries or `OptionRetryConfig(cfg)` for full control including 5xx and connection errors with exponential backoff. ([#1532])
- **`task_card` and `plan` agent blocks** — New block types for task cards and plan agent blocks. ([#1536])
### Changed
- CI: bumped `actions/stale` from 10.1.1 to 10.2.0. ([#1534])
- Use `golangci-lint` in Makefile. ([#1533])
## [0.18.0] - 2026-02-21
### Added
- **`focus_on_load` support for remaining block elements** — Static/external/users/conversations/channels select, multi-select variants, datepicker, timepicker, plain_text_input, checkboxes, radio_buttons, and number_input. ([#1519])
- **`PlainText` and `PreviewPlainText` fields on `File`** — Email file objects now include the plain text body fields instead of silently discarding them. ([#1522])
- **Missing fields on `User`, `UserProfile`, and `EnterpriseUser`**`who_can_share_contact_card`, `always_active`, `pronouns`, `image_1024`, `is_custom_image`, `status_text_canonical`, `huddle_state`, `huddle_state_expiration_ts`, `start_date`, and `is_primary_owner`. ([#1526])
- **Work Objects support** — Chat unfurl with Work Object metadata, entity details (flexpane), `entity_details_requested` event, and associated types (`WorkObjectMetadata`, `WorkObjectEntity`, `WorkObjectExternalRef`). ([#1529])
- **`admin.roles.*` API methods** — `admin.roles.listAssignments`, `admin.roles.addAssignments`, and `admin.roles.removeAssignments`. ([#1520])
### Fixed
- **`UserProfile.Skype` JSON tag** — Corrected typo from `"skyp"` to `"skype"`. ([#1524])
- **`assistant.threads.setSuggestedPrompts` title parameter** — Title is now sent when non-empty. ([#1528])
### Changed
- CI test matrix updated: dropped Go 1.24, added Go 1.26; bumped golangci-lint to v2.10.1. ([#1530])
## [0.18.0-rc2] - 2026-01-28
### Added
- **Audit Logs example** - New example demonstrating how to use the Audit Logs API. ([#1144])
- **Admin Conversations API support** - Comprehensive support for `admin.conversations.*`
methods including core operations (archive, unarchive, create, delete, rename, invite,
search, lookup, getTeams, convertToPrivate, convertToPublic, disconnectShared, setTeams),
bulk operations (bulkArchive, bulkDelete, bulkMove), preferences, retention management,
restrict access controls, and EKM channel info. ([#1329])
### Changed
- **BREAKING**: Removed deprecated `UploadFile`, `UploadFileContext`, and
`FileUploadParameters`. The `files.upload` API was discontinued by Slack on November
12, 2025. ([#1481])
- **BREAKING**: Renamed `UploadFileV2``UploadFile`, `UploadFileV2Context`
`UploadFileContext`, and `UploadFileV2Parameters``UploadFileParameters`. The "V2"
suffix is no longer needed now that the old API is removed. ([#1481])
### Fixed
- **File upload error wrapping** - `UploadFile` now wraps errors with the step name
(`GetUploadURLExternal`, `UploadToURL`, or `CompleteUploadExternal`) so callers can
identify which of the three upload steps failed. ([#1491])
- **Audit Logs API endpoint** - Fixed `GetAuditLogs` to use the correct endpoint
(`api.slack.com`) instead of the regular API endpoint (`slack.com/api`). The Audit
Logs API requires a different base URL. Added `OptionAuditAPIURL` for testing. ([#1144])
- **Socket mode websocket dial debugging** - Added debug logging when a custom dialer is
used including HTTP response status on dial failures. This helps diagnose proxy/TLS
issues like "bad handshake" errors. ([#1360])
- **`MsgOptionPostMessageParameters` now passes `MetaData`** - Previously, metadata was
silently dropped when using `PostMessageParameters`. ([#1343])
## [0.18.0-rc1] - 2026-01-26
### Added
- **Huddle support** - New `HuddleRoom`, `HuddleParticipantEvent`, and `HuddleRecording`
types for handling Slack huddle events (`huddle_thread` subtype messages).
- **Call block data parsing** - `CallBlock` now includes full call data when retrieved
from Slack messages, with new `CallBlockData`, `CallBlockDataV1`, and `CallBlockIconURLs`
types. ([#897])
- **Chat Streaming API support** - New streaming API for real-time chat interactions
with example usage. ([#1506])
- **Data Access API support** - Full support for Slack's Data Access API with
example implementation. ([#1439])
- **Cursor-based pagination for `GetUsers`** - More efficient user retrieval
with cursor pagination. ([#1465])
- **`GetAllConversations` with pagination** - Retrieve all conversations with
automatic pagination handling, including rate limit and server error handling. ([#1463])
- **Table blocks support** - Parse and create table blocks with proper
unmarshaling. ([#1490], [#1511])
- **Context actions block support** - New `context_actions` block type. ([#1495])
- **Workflow button block element** - Support for `workflow_button` in block
elements. ([#1499])
- **`loading_messages` parameter for `SetAssistantThreadsStatus`** - Optional
parameter to customize loading state messages. ([#1489])
- **Attachment image fields** - Added `ImageBytes`, `ImageHeight`, and `ImageWidth`
fields to attachments. ([#1516])
- **`RecordChannel` to conversation properties** - New property for conversation
metadata. ([#1513])
- **Title argument for `CreateChannelCanvas`** - Canvas creation now supports
custom titles. ([#1483])
- **`PostEphemeral` handler for slacktest** - Audit outgoing ephemeral messages
in test environments. ([#1517])
- **`PreviewImageName` for remote files** - Customize preview image filename
instead of using the default `preview.jpg`.
### Fixed
- **`PublishView` no longer sends empty hash** - Prevents unnecessary payload
when hash is empty. ([#1515])
- **`ImageBlockElement` validation** - Now properly validates that either
`imageURL` or `SlackFile` is provided. ([#1488])
- **Rich text section channel return** - Correctly returns channel for section
channel rich text elements. ([#1472])
- **`KickUserFromConversation` error handling** - Errors are now properly parsed
as a map structure. ([#1471])
### Changed
- **BREAKING**: `GetReactions` now returns `ReactedItem` instead of `[]ItemReaction`.
This aligns the response with the actual Slack API, which includes the item itself
(message, file, or file_comment) alongside reactions. To migrate, use `resp.Reactions`
to access the slice of reactions. ([#1480])
- **BREAKING**: `Settings` struct fields `Interactivity` and `EventSubscriptions`
are now pointers, allowing them to be omitted when empty. ([#1461])
- Minimum Go version bumped to 1.24. ([#1504])
## [0.17.3] - 2025-07-04
Previous release. See [GitHub releases](https://github.com/slack-go/slack/releases/tag/v0.17.3)
for details.
[#897]: https://github.com/slack-go/slack/issues/897
[#1236]: https://github.com/slack-go/slack/issues/1236
[#1257]: https://github.com/slack-go/slack/issues/1257
[#1144]: https://github.com/slack-go/slack/issues/1144
[#1329]: https://github.com/slack-go/slack/issues/1329
[#1343]: https://github.com/slack-go/slack/issues/1343
[#1360]: https://github.com/slack-go/slack/issues/1360
[#1439]: https://github.com/slack-go/slack/pull/1439
[#1461]: https://github.com/slack-go/slack/pull/1461
[#1463]: https://github.com/slack-go/slack/pull/1463
[#1465]: https://github.com/slack-go/slack/pull/1465
[#1471]: https://github.com/slack-go/slack/pull/1471
[#1472]: https://github.com/slack-go/slack/pull/1472
[#1480]: https://github.com/slack-go/slack/pull/1480
[#1483]: https://github.com/slack-go/slack/pull/1483
[#1488]: https://github.com/slack-go/slack/pull/1488
[#1489]: https://github.com/slack-go/slack/pull/1489
[#1490]: https://github.com/slack-go/slack/pull/1490
[#1491]: https://github.com/slack-go/slack/issues/1491
[#1495]: https://github.com/slack-go/slack/pull/1495
[#1497]: https://github.com/slack-go/slack/pull/1497
[#1499]: https://github.com/slack-go/slack/pull/1499
[#1504]: https://github.com/slack-go/slack/pull/1504
[#1506]: https://github.com/slack-go/slack/pull/1506
[#1511]: https://github.com/slack-go/slack/pull/1511
[#1513]: https://github.com/slack-go/slack/pull/1513
[#1515]: https://github.com/slack-go/slack/pull/1515
[#1516]: https://github.com/slack-go/slack/pull/1516
[#1517]: https://github.com/slack-go/slack/pull/1517
[#1519]: https://github.com/slack-go/slack/pull/1519
[#1520]: https://github.com/slack-go/slack/pull/1520
[#1522]: https://github.com/slack-go/slack/pull/1522
[#1524]: https://github.com/slack-go/slack/pull/1524
[#1526]: https://github.com/slack-go/slack/pull/1526
[#1528]: https://github.com/slack-go/slack/pull/1528
[#1529]: https://github.com/slack-go/slack/pull/1529
[#1530]: https://github.com/slack-go/slack/pull/1530
[#1532]: https://github.com/slack-go/slack/pull/1532
[#1533]: https://github.com/slack-go/slack/pull/1533
[#1534]: https://github.com/slack-go/slack/pull/1534
[#1536]: https://github.com/slack-go/slack/pull/1536
[#596]: https://github.com/slack-go/slack/issues/596
[#1541]: https://github.com/slack-go/slack/issues/1541
[#1172]: https://github.com/slack-go/slack/issues/1172
[#1076]: https://github.com/slack-go/slack/issues/1076
[#1157]: https://github.com/slack-go/slack/issues/1157
[#1196]: https://github.com/slack-go/slack/issues/1196
[#1547]: https://github.com/slack-go/slack/pull/1547
[Unreleased]: https://github.com/slack-go/slack/compare/v0.21.1...HEAD
[0.21.1]: https://github.com/slack-go/slack/compare/v0.21.0...v0.21.1
[0.21.0]: https://github.com/slack-go/slack/compare/v0.20.0...v0.21.0
[0.20.0]: https://github.com/slack-go/slack/compare/v0.19.0...v0.20.0
[0.19.0]: https://github.com/slack-go/slack/compare/v0.18.0...v0.19.0
[0.18.0]: https://github.com/slack-go/slack/compare/v0.18.0-rc2...v0.18.0
[0.18.0-rc2]: https://github.com/slack-go/slack/releases/tag/v0.18.0-rc2
[0.18.0-rc1]: https://github.com/slack-go/slack/releases/tag/v0.18.0-rc1
[0.17.3]: https://github.com/slack-go/slack/releases/tag/v0.17.3

Some files were not shown because too many files have changed in this diff Show More