mirrors/tendermint
1
0
Fork 0
mirror of https://github.com/tendermint/tendermint.git synced 2026-02-04 19:12:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into wb/add-consensus-param-internal

Browse Source
This commit is contained in:
William Banfield
2022-05-15 23:27:32 -04:00
committed by GitHub
parent 9d6e0235e9 c780619db5
commit e80d2bd494
46 changed files with 1386 additions and 791 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
.github/dependabot.yml vendored
View File

@@ -4,6 +4,30 @@ updates:
directory: "/"
schedule:
interval: weekly
day: monday
target-branch: "master"
open-pull-requests-limit: 10
labels:
- T:dependencies
- S:automerge
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
day: monday
target-branch: "v0.35.x"
open-pull-requests-limit: 10
labels:
- T:dependencies
- S:automerge
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
day: monday
target-branch: "v0.34.x"
open-pull-requests-limit: 10
labels:
- T:dependencies
@@ -13,6 +37,7 @@ updates:
directory: "/docs"
schedule:
interval: weekly
day: monday
open-pull-requests-limit: 10
###################################

76
.github/workflows/check-generated.yml vendored Normal file
View File

@@ -0,0 +1,76 @@
# Verify that generated code is up-to-date.
#
# Note that we run these checks regardless whether the input files have
# changed, because generated code can change in response to toolchain updates
# even if no files in the repository are modified.
name: Check generated code
on:
pull_request:
branches:
- master
permissions:
contents: read
jobs:
check-mocks:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v3
with:
go-version: '1.17'
- uses: actions/checkout@v3
- name: "Check generated mocks"
run: |
set -euo pipefail
make mockery 2>/dev/null
if ! git diff --stat --exit-code ; then
echo ">> ERROR:"
echo ">>"
echo ">> Generated mocks require update (either Mockery or source files may have changed)."
echo ">> Ensure your tools are up-to-date, re-run 'make mockery' and update this PR."
echo ">>"
exit 1
fi
check-proto:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v3
with:
go-version: '1.17'
- uses: actions/checkout@v3
with:
fetch-depth: 1 # we need a .git directory to run git diff
- name: "Check protobuf generated code"
run: |
set -euo pipefail
# Install buf and gogo tools, so that differences that arise from
# toolchain differences are also caught.
readonly tools="$(mktemp -d)"
export PATH="${PATH}:${tools}/bin"
export GOBIN="${tools}/bin"
readonly base='https://github.com/bufbuild/buf/releases/latest/download'
readonly OS="$(uname -s)" ARCH="$(uname -m)"
curl -sSL "${base}/buf-${OS}-${ARCH}.tar.gz" \
| tar -xzf - -C "$tools" --strip-components=1
go install github.com/gogo/protobuf/protoc-gen-gogofaster@latest
make proto-gen
if ! git diff --stat --exit-code ; then
echo ">> ERROR:"
echo ">>"
echo ">> Protobuf generated code requires update (either tools or .proto files may have changed)."
echo ">> Ensure your tools are up-to-date, re-run 'make proto-gen' and update this PR."
echo ">>"
exit 1
fi

2
.github/workflows/lint.yml vendored
View File

@@ -27,7 +27,7 @@ jobs:
**/**.go
go.mod
go.sum
- uses: golangci/golangci-lint-action@v3.1.0
- uses: golangci/golangci-lint-action@v3
with:
# Required: the version of golangci-lint is required and
# must be specified without patch version: we always use the

2
docs/nodes/README.md
View File

@@ -45,4 +45,4 @@ We will cover the various types of node types within Tendermint.
Validators are nodes that participate in the security of a network. Validators have an associated power in Tendermint, this power can represent stake in a [proof of stake](https://en.wikipedia.org/wiki/Proof_of_stake) system, reputation in [proof of authority](https://en.wikipedia.org/wiki/Proof_of_authority) or any sort of measurable unit. Running a secure and consistently online validator is crucial to a networks health. A validator must be secure and fault tolerant, it is recommended to run your validator with 2 or more sentry nodes.
As a validator there is the potential to have your weight reduced, this is defined by the application. Tendermint is notified by the application if a validator should have there weight increased or reduced. Application have different types of malicious behavior which lead to slashing of the validators power. Please check the documentation of the application you will be running in order to find more information.
As a validator there is the potential to have your weight reduced, this is defined by the application. Tendermint is notified by the application if a validator should have their weight increased or reduced. Application have different types of malicious behavior which lead to slashing of the validators power. Please check the documentation of the application you will be running in order to find more information.

256
docs/tendermint-core/subscription.md
View File

@@ -2,74 +2,228 @@
order: 7
---
# Subscribing to events via Websocket
# Subscribing to Events
Tendermint emits different events, which you can subscribe to via
[Websocket](https://en.wikipedia.org/wiki/WebSocket). This can be useful
for third-party applications (for analysis) or for inspecting state.
A Tendermint node emits events about important state transitions during
consensus. These events can be queried by clients via the [RPC interface][rpc]
on nodes that enable it. The [list of supported event types][event-types] can
be found in the tendermint/types Go package.
[List of events](https://godoc.org/github.com/tendermint/tendermint/types#pkg-constants)
In Tendermint v0.36 there are two APIs to query events:
To connect to a node via websocket from the CLI, you can use a tool such as
[wscat](https://github.com/websockets/wscat) and run:
- The [**legacy streaming API**](#legacy-streaming-api), comprising the
`subscribe`, `unsubscribe`, and `unsubscribe_all` RPC methods over websocket.
- The [**event log API**](#event-log-api), comprising the `events` RPC method.
The legacy streaming API is deprecated in Tendermint v0.36, and will be removed
in Tendermint v0.37. Clients are strongly encouraged to migrate to the new
event log API as soon as is practical.
[rpc]: https://docs.tendermint.com/master/rpc
[event-types]: https://godoc.org/github.com/tendermint/tendermint/types#EventNewBlockValue
## Filter Queries
Event requests take a [filter query][query] parameter. A filter query is a
string that describes a subset of available event items to return. An empty
query matches all events; otherwise a query comprises one or more *terms*
comparing event metadata to target values.
For example, to select new block events, use the term:
```
tm.event = 'NewBlock'
```
Multiple terms can be combined with `AND` (case matters), for example to match
the transaction event with a given hash, use the query:
```
tm.event = 'Tx' AND tx.hash = 'EA7B33F'
```
Operands may be strings in single quotes (`'Tx'`), numbers (`45`), dates, or
timestamps.
The comparison operators include `=`, `<`, `<=`, `>`, `>=`, and `CONTAINS` (for
substring match). In addition, the `EXISTS` operator checks for the presence
of an attribute regardless of its value.
### Attributes
Tendermint implicitly defines a string-valued `tm.event` attribute for all
event types. Transaction items (type `Tx`) are also assigned `tx.hash`
(string), giving the hash of the transaction, and and `tx.height` (number)
giving the height of the block containing the transaction. For `NewBlock` and
`NewBlockHeader` events, Tendermint defines a `block.height` attribute giving
the height of the block.
Additional attributes can be provided by the application as [ABCI `Event`
records][abci-event] in response to the `FinalizeBlock` request. The full name
of the attribute in the query is formed by combining the `type` and attribute
`key` with a period.
For example, given the events
```go
[]abci.Event{{
Type: "reward",
Attributes: []abci.EventAttribute{
{Key: "address", Value: "cosmos1xyz012pdq"},
{Key: "amount", Value: "45.62"},
{Key: "balance", Value: "100.390001"},
},
}}
```
a query may refer to the names `reward.address`, `reward.amount`, and `reward.balance`, as in:
```
reward.address EXISTS AND reward.balance > 45
```
Certain application-specific metadata are also indexed for offline queries.
See [Indexing transactions](../app-dev/indexing-transactions.md) for more details.
[query]: https://godoc.org/github.com/tendermint/tendermint/internal/pubsub/query/syntax
[abci-event]: https://github.com/tendermint/tendermint/blob/master/proto/tendermint/abci/types.proto#L397
## Event Log API
Starting in Tendermint v0.36, when the `rpc.event-log-window-size`
configuration is enabled, the node maintains maintains a log of all events
within this operator-defined time window. This API supersedes the websocket
subscription API described below.
Clients can query these events can by long-polling the `/events` RPC method,
which returns the most recent items from the log that match the [request
parameters][reqevents]. Each item returned includes a cursor that marks its
location in the log. Cursors can be passed via the `before` and `after`
parameters to fetch events earlier in the log.
For example, this request:
```json
{
"jsonrpc": "2.0",
"id": 1,
"method": "events",
"params": {
"filter": {
"query": "tm.event = 'Tx' AND app.key = 'applesauce'"
},
"maxItems": 1,
"after": ""
}
}
```
will return a result similar to the following:
```json
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"items": [
{
"cursor": "16ee3d5e65be53d8-03d5",
"event": "Tx",
"data": {
"type": "tendermint/event/Tx",
"value": {
"height": 70,
"tx": "YXBwbGVzYXVjZT1zeXJ1cA==",
"result": {
"events": [
{
"type": "app",
"attributes": [
{
"key": "creator",
"value": "Cosmoshi Netowoko",
"index": true
},
{
"key": "key",
"value": "applesauce",
"index": true
},
{
"key": "index_key",
"value": "index is working",
"index": true
},
{
"key": "noindex_key",
"value": "index is working",
"index": false
}
]
}
]
}
}
}
}
],
"more": false,
"oldest": "16ee3d4c471c3b00-0001",
"newest": "16ee3d5f2e05a4e0-0400"
}
}
```
The `"items"` array gives the matching items (up to the requested
`"maxResults"`) in decreasing time order (i.e., newest to oldest). In this
case, there is only one result, but if there are additional results that were
not returned, the `"more"` flag will be true. Calling `/events` again with the
same query and `"after"` set to the cursor of the newest result (in this
example, `"16ee3d5e65be53d8-03d5"`) will fetch newer results.
Go clients can use the [`eventstream`][eventstream] package to simplify the use
of this method. The `eventstream.Stream` automatically handles polling for new
events, updating the cursor, and reporting any missed events.
[reqevents]: https://pkg.go.dev/github.com/tendermint/tendermint@master/rpc/coretypes#RequestEvents
[eventstream]: https://godoc.org/github.com/tendermint/tendermint/rpc/client/eventstream
## Legacy Streaming API
- **Note:** This API is deprecated in Tendermint v0.36, and will be removed in
Tendermint v0.37. New clients and existing use should use the [event log
API](#event-log-api) instead. See [ADR 075][adr075] for more details.
To subscribe to events in the streaming API, you must connect to the node RPC
service using a [websocket][ws]. From the command line you can use a tool such
as [wscat][wscat], for example:
```sh
wscat ws://127.0.0.1:26657/websocket
```
You can subscribe to any of the events above by calling the `subscribe` RPC
method via Websocket along with a valid query.
[ws]: https://en.wikipedia.org/wiki/WebSocket
[wscat]: https://github.com/websockets/wscat
To subscribe to events, call the `subscribe` JSON-RPC method method passing in
a [filter query][query] for the events you wish to receive:
```json
{
"jsonrpc": "2.0",
"method": "subscribe",
"id": 0,
"id": 1,
"params": {
"query": "tm.event='NewBlock'"
}
}
```
Check out [API docs](https://docs.tendermint.com/master/rpc/) for
more information on query syntax and other options.
The subscribe method returns an initial response confirming the subscription,
then sends additional JSON-RPC response messages containing the matching events
as they are published. The subscription continues until either the client
explicitly cancels the subscription (by calling `unsubscribe` or
`unsubscribe_all`) or until the websocket connection is terminated.
You can also use tags, given you had included them into DeliverTx
response, to query transaction results. See [Indexing
transactions](../app-dev/indexing-transactions.md) for details.
## ValidatorSetUpdates
When validator set changes, ValidatorSetUpdates event is published. The
event carries a list of pubkey/power pairs. The list is the same
Tendermint receives from ABCI application (see [EndBlock
section](https://github.com/tendermint/tendermint/blob/master/spec/abci/abci.md#endblock) in
the ABCI spec).
Response:
```json
{
"jsonrpc": "2.0",
"id": 0,
"result": {
"query": "tm.event='ValidatorSetUpdates'",
"data": {
"type": "tendermint/event/ValidatorSetUpdates",
"value": {
"validator_updates": [
{
"address": "09EAD022FD25DE3A02E64B0FE9610B1417183EE4",
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "ww0z4WaZ0Xg+YI10w43wTWbBmM3dpVza4mmSQYsd0ck="
},
"voting_power": "10",
"proposer_priority": "0"
}
]
}
}
}
}
```
[adr075]: https://tinyurl.com/adr075

6
go.mod
View File

@@ -21,7 +21,7 @@ require (
github.com/mroth/weightedrand v0.4.1
github.com/oasisprotocol/curve25519-voi v0.0.0-20210609091139-0a56a4bca00b
github.com/ory/dockertest v3.3.5+incompatible
github.com/prometheus/client_golang v1.12.1
github.com/prometheus/client_golang v1.12.2
github.com/rs/cors v1.8.2
github.com/rs/zerolog v1.26.1
github.com/snikch/goodman v0.0.0-20171125024755-10e37e294daa
@@ -173,8 +173,8 @@ require (
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/polyfloyd/go-errorlint v0.0.0-20211125173453-6d6d39c5bb8b // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/client_model v0.2.0
github.com/prometheus/common v0.32.1
github.com/prometheus/procfs v0.7.3 // indirect
github.com/quasilyte/go-ruleguard v0.3.16-0.20220213074421-6aa060fab41a // indirect
github.com/quasilyte/gogrep v0.0.0-20220120141003-628d8b3623b5 // indirect

3
go.sum
View File

@@ -873,8 +873,9 @@ github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5Fsn
github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0=
github.com/prometheus/client_golang v1.12.1 h1:ZiaPsmm9uiBeaSMRznKsCDNtPCS0T3JVDGF+06gjBzk=
github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
github.com/prometheus/client_golang v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34=
github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=

248
internal/consensus/metrics.gen.go Normal file
View File

@@ -0,0 +1,248 @@
// Code generated by metricsgen. DO NOT EDIT.
package consensus
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Height: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "height",
Help: "Height of the chain.",
}, labels).With(labelsAndValues...),
ValidatorLastSignedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_last_signed_height",
Help: "Last height signed by this validator if the node is a validator.",
}, append(labels, "validator_address")).With(labelsAndValues...),
Rounds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "rounds",
Help: "Number of rounds.",
}, labels).With(labelsAndValues...),
RoundDuration: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "round_duration",
Help: "Histogram of round duration.",
Buckets: stdprometheus.ExponentialBucketsRange(0.1, 100, 8),
}, labels).With(labelsAndValues...),
Validators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators",
Help: "Number of validators.",
}, labels).With(labelsAndValues...),
ValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators_power",
Help: "Total power of all validators.",
}, labels).With(labelsAndValues...),
ValidatorPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_power",
Help: "Power of a validator.",
}, append(labels, "validator_address")).With(labelsAndValues...),
ValidatorMissedBlocks: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_missed_blocks",
Help: "Amount of blocks missed per validator.",
}, append(labels, "validator_address")).With(labelsAndValues...),
MissingValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators",
Help: "Number of validators who did not sign.",
}, labels).With(labelsAndValues...),
MissingValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators_power",
Help: "Total power of the missing validators.",
}, labels).With(labelsAndValues...),
ByzantineValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators",
Help: "Number of validators who tried to double sign.",
}, labels).With(labelsAndValues...),
ByzantineValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators_power",
Help: "Total power of the byzantine validators.",
}, labels).With(labelsAndValues...),
BlockIntervalSeconds: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_interval_seconds",
Help: "Time between this and the last block.",
}, labels).With(labelsAndValues...),
NumTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_txs",
Help: "Number of transactions.",
}, labels).With(labelsAndValues...),
BlockSizeBytes: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_size_bytes",
Help: "Size of the block.",
}, labels).With(labelsAndValues...),
TotalTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "total_txs",
Help: "Total number of transactions.",
}, labels).With(labelsAndValues...),
CommittedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "latest_block_height",
Help: "The latest block height.",
}, labels).With(labelsAndValues...),
BlockSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_syncing",
Help: "Whether or not a node is block syncing. 1 if yes, 0 if no.",
}, labels).With(labelsAndValues...),
StateSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "state_syncing",
Help: "Whether or not a node is state syncing. 1 if yes, 0 if no.",
}, labels).With(labelsAndValues...),
BlockParts: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_parts",
Help: "Number of block parts transmitted by each peer.",
}, append(labels, "peer_id")).With(labelsAndValues...),
StepDuration: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "step_duration",
Help: "Histogram of durations for each step in the consensus protocol.",
Buckets: stdprometheus.ExponentialBucketsRange(0.1, 100, 8),
}, append(labels, "step")).With(labelsAndValues...),
BlockGossipReceiveLatency: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_gossip_receive_latency",
Help: "Histogram of time taken to receive a block in seconds, measured between when a new block is first discovered to when the block is completed.",
Buckets: stdprometheus.ExponentialBucketsRange(0.1, 100, 8),
}, labels).With(labelsAndValues...),
BlockGossipPartsReceived: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_gossip_parts_received",
Help: "Number of block parts received by the node, separated by whether the part was relevant to the block the node is trying to gather or not.",
}, append(labels, "matches_current")).With(labelsAndValues...),
QuorumPrevoteDelay: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "quorum_prevote_delay",
Help: "Interval in seconds between the proposal timestamp and the timestamp of the earliest prevote that achieved a quorum.",
}, append(labels, "proposer_address")).With(labelsAndValues...),
FullPrevoteDelay: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "full_prevote_delay",
Help: "Interval in seconds between the proposal timestamp and the timestamp of the latest prevote in a round where all validators voted.",
}, append(labels, "proposer_address")).With(labelsAndValues...),
ProposalTimestampDifference: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "proposal_timestamp_difference",
Help: "Difference between the timestamp in the proposal message and the local time of the validator at the time it received the message.",
Buckets: []float64{-10, -.5, -.025, 0, .1, .5, 1, 1.5, 2, 10},
}, append(labels, "is_timely")).With(labelsAndValues...),
VoteExtensionReceiveCount: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "vote_extension_receive_count",
Help: "Number of vote extensions received labeled by application response status.",
}, append(labels, "status")).With(labelsAndValues...),
ProposalReceiveCount: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "proposal_receive_count",
Help: "Total number of proposals received by the node since process start labeled by application response status.",
}, append(labels, "status")).With(labelsAndValues...),
ProposalCreateCount: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "proposal_create_count",
Help: "Total number of proposals created by the node since process start.",
}, labels).With(labelsAndValues...),
RoundVotingPowerPercent: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "round_voting_power_percent",
Help: "A value between 0 and 1.0 representing the percentage of the total voting power per vote type received within a round.",
}, append(labels, "vote_type")).With(labelsAndValues...),
LateVotes: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "late_votes",
Help: "Number of votes received by the node since process start that correspond to earlier heights and rounds than this node is currently in.",
}, append(labels, "vote_type")).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
Height: discard.NewGauge(),
ValidatorLastSignedHeight: discard.NewGauge(),
Rounds: discard.NewGauge(),
RoundDuration: discard.NewHistogram(),
Validators: discard.NewGauge(),
ValidatorsPower: discard.NewGauge(),
ValidatorPower: discard.NewGauge(),
ValidatorMissedBlocks: discard.NewGauge(),
MissingValidators: discard.NewGauge(),
MissingValidatorsPower: discard.NewGauge(),
ByzantineValidators: discard.NewGauge(),
ByzantineValidatorsPower: discard.NewGauge(),
BlockIntervalSeconds: discard.NewHistogram(),
NumTxs: discard.NewGauge(),
BlockSizeBytes: discard.NewHistogram(),
TotalTxs: discard.NewGauge(),
CommittedHeight: discard.NewGauge(),
BlockSyncing: discard.NewGauge(),
StateSyncing: discard.NewGauge(),
BlockParts: discard.NewCounter(),
StepDuration: discard.NewHistogram(),
BlockGossipReceiveLatency: discard.NewHistogram(),
BlockGossipPartsReceived: discard.NewCounter(),
QuorumPrevoteDelay: discard.NewGauge(),
FullPrevoteDelay: discard.NewGauge(),
ProposalTimestampDifference: discard.NewHistogram(),
VoteExtensionReceiveCount: discard.NewCounter(),
ProposalReceiveCount: discard.NewCounter(),
ProposalCreateCount: discard.NewCounter(),
RoundVotingPowerPercent: discard.NewGauge(),
LateVotes: discard.NewCounter(),
}
}

313
internal/consensus/metrics.go
View File

@@ -5,14 +5,10 @@ import (
"time"
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
cstypes "github.com/tendermint/tendermint/internal/consensus/types"
tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
"github.com/tendermint/tendermint/types"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -21,28 +17,30 @@ const (
MetricsSubsystem = "consensus"
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains metrics exposed by this package.
type Metrics struct {
// Height of the chain.
Height metrics.Gauge
// ValidatorLastSignedHeight of a validator.
ValidatorLastSignedHeight metrics.Gauge
// Last height signed by this validator if the node is a validator.
ValidatorLastSignedHeight metrics.Gauge `metrics_labels:"validator_address"`
// Number of rounds.
Rounds metrics.Gauge
// Histogram of round duration.
RoundDuration metrics.Histogram
RoundDuration metrics.Histogram `metrics_buckettype:"exprange" metrics_bucketsizes:"0.1, 100, 8"`
// Number of validators.
Validators metrics.Gauge
// Total power of all validators.
ValidatorsPower metrics.Gauge
// Power of a validator.
ValidatorPower metrics.Gauge
// Amount of blocks missed by a validator.
ValidatorMissedBlocks metrics.Gauge
ValidatorPower metrics.Gauge `metrics_labels:"validator_address"`
// Amount of blocks missed per validator.
ValidatorMissedBlocks metrics.Gauge `metrics_labels:"validator_address"`
// Number of validators who did not sign.
MissingValidators metrics.Gauge
// Total power of the missing validators.
@@ -62,27 +60,27 @@ type Metrics struct {
// Total number of transactions.
TotalTxs metrics.Gauge
// The latest block height.
CommittedHeight metrics.Gauge
CommittedHeight metrics.Gauge `metrics_name:"latest_block_height"`
// Whether or not a node is block syncing. 1 if yes, 0 if no.
BlockSyncing metrics.Gauge
// Whether or not a node is state syncing. 1 if yes, 0 if no.
StateSyncing metrics.Gauge
// Number of blockparts transmitted by peer.
BlockParts metrics.Counter
// Number of block parts transmitted by each peer.
BlockParts metrics.Counter `metrics_labels:"peer_id"`
// Histogram of step duration.
StepDuration metrics.Histogram
// Histogram of durations for each step in the consensus protocol.
StepDuration metrics.Histogram `metrics_labels:"step" metrics_buckettype:"exprange" metrics_bucketsizes:"0.1, 100, 8"`
stepStart time.Time
// Histogram of time taken to receive a block in seconds, measured between when a new block is first
// discovered to when the block is completed.
BlockGossipReceiveLatency metrics.Histogram
BlockGossipReceiveLatency metrics.Histogram `metrics_buckettype:"exprange" metrics_bucketsizes:"0.1, 100, 8"`
blockGossipStart time.Time
// Number of block parts received by the node, separated by whether the part
// was relevant to the block the node is trying to gather or not.
BlockGossipPartsReceived metrics.Counter
BlockGossipPartsReceived metrics.Counter `metrics_labels:"matches_current"`
// QuroumPrevoteMessageDelay is the interval in seconds between the proposal
// timestamp and the timestamp of the earliest prevote that achieved a quorum
@@ -93,301 +91,50 @@ type Metrics struct {
// be above 2/3 of the total voting power of the network defines the endpoint
// the endpoint of the interval. Subtract the proposal timestamp from this endpoint
// to obtain the quorum delay.
QuorumPrevoteDelay metrics.Gauge
//metrics:Interval in seconds between the proposal timestamp and the timestamp of the earliest prevote that achieved a quorum.
QuorumPrevoteDelay metrics.Gauge `metrics_labels:"proposer_address"`
// FullPrevoteDelay is the interval in seconds between the proposal
// timestamp and the timestamp of the latest prevote in a round where 100%
// of the voting power on the network issued prevotes.
FullPrevoteDelay metrics.Gauge
//metrics:Interval in seconds between the proposal timestamp and the timestamp of the latest prevote in a round where all validators voted.
FullPrevoteDelay metrics.Gauge `metrics_labels:"proposer_address"`
// ProposalTimestampDifference is the difference between the timestamp in
// the proposal message and the local time of the validator at the time
// that the validator received the message.
ProposalTimestampDifference metrics.Histogram
//metrics:Difference between the timestamp in the proposal message and the local time of the validator at the time it received the message.
ProposalTimestampDifference metrics.Histogram `metrics_labels:"is_timely" metrics_bucketsizes:"-10, -.5, -.025, 0, .1, .5, 1, 1.5, 2, 10"`
// VoteExtensionReceiveCount is the number of vote extensions received by this
// node. The metric is annotated by the status of the vote extension from the
// application, either 'accepted' or 'rejected'.
VoteExtensionReceiveCount metrics.Counter
//metrics:Number of vote extensions received labeled by application response status.
VoteExtensionReceiveCount metrics.Counter `metrics_labels:"status"`
// ProposalReceiveCount is the total number of proposals received by this node
// since process start.
// The metric is annotated by the status of the proposal from the application,
// either 'accepted' or 'rejected'.
ProposalReceiveCount metrics.Counter
//metrics:Total number of proposals received by the node since process start labeled by application response status.
ProposalReceiveCount metrics.Counter `metrics_labels:"status"`
// ProposalCreationCount is the total number of proposals created by this node
// since process start.
// The metric is annotated by the status of the proposal from the application,
// either 'accepted' or 'rejected'.
//metrics:Total number of proposals created by the node since process start.
ProposalCreateCount metrics.Counter
// RoundVotingPowerPercent is the percentage of the total voting power received
// with a round. The value begins at 0 for each round and approaches 1.0 as
// additional voting power is observed. The metric is labeled by vote type.
RoundVotingPowerPercent metrics.Gauge
//metrics:A value between 0 and 1.0 representing the percentage of the total voting power per vote type received within a round.
RoundVotingPowerPercent metrics.Gauge `metrics_labels:"vote_type"`
// LateVotes stores the number of votes that were received by this node that
// correspond to earlier heights and rounds than this node is currently
// in.
LateVotes metrics.Counter
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Height: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "height",
Help: "Height of the chain.",
}, labels).With(labelsAndValues...),
Rounds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "rounds",
Help: "Number of rounds.",
}, labels).With(labelsAndValues...),
RoundDuration: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "round_duration",
Help: "Time spent in a round.",
Buckets: stdprometheus.ExponentialBucketsRange(0.1, 100, 8),
}, labels).With(labelsAndValues...),
Validators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators",
Help: "Number of validators.",
}, labels).With(labelsAndValues...),
ValidatorLastSignedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_last_signed_height",
Help: "Last signed height for a validator",
}, append(labels, "validator_address")).With(labelsAndValues...),
ValidatorMissedBlocks: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_missed_blocks",
Help: "Total missed blocks for a validator",
}, append(labels, "validator_address")).With(labelsAndValues...),
ValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators_power",
Help: "Total power of all validators.",
}, labels).With(labelsAndValues...),
ValidatorPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_power",
Help: "Power of a validator",
}, append(labels, "validator_address")).With(labelsAndValues...),
MissingValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators",
Help: "Number of validators who did not sign.",
}, labels).With(labelsAndValues...),
MissingValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators_power",
Help: "Total power of the missing validators.",
}, labels).With(labelsAndValues...),
ByzantineValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators",
Help: "Number of validators who tried to double sign.",
}, labels).With(labelsAndValues...),
ByzantineValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators_power",
Help: "Total power of the byzantine validators.",
}, labels).With(labelsAndValues...),
BlockIntervalSeconds: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_interval_seconds",
Help: "Time between this and the last block.",
}, labels).With(labelsAndValues...),
NumTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_txs",
Help: "Number of transactions.",
}, labels).With(labelsAndValues...),
BlockSizeBytes: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_size_bytes",
Help: "Size of the block.",
}, labels).With(labelsAndValues...),
TotalTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "total_txs",
Help: "Total number of transactions.",
}, labels).With(labelsAndValues...),
CommittedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "latest_block_height",
Help: "The latest block height.",
}, labels).With(labelsAndValues...),
BlockSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_syncing",
Help: "Whether or not a node is block syncing. 1 if yes, 0 if no.",
}, labels).With(labelsAndValues...),
StateSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "state_syncing",
Help: "Whether or not a node is state syncing. 1 if yes, 0 if no.",
}, labels).With(labelsAndValues...),
BlockParts: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_parts",
Help: "Number of blockparts transmitted by peer.",
}, append(labels, "peer_id")).With(labelsAndValues...),
BlockGossipReceiveLatency: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_gossip_receive_latency",
Help: "Difference in seconds between when the validator learns of a new block" +
"and when the validator receives the last piece of the block.",
Buckets: stdprometheus.ExponentialBucketsRange(0.1, 100, 8),
}, labels).With(labelsAndValues...),
BlockGossipPartsReceived: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_gossip_parts_received",
Help: "Number of block parts received by the node, labeled by whether the " +
"part was relevant to the block the node was currently gathering or not.",
}, append(labels, "matches_current")).With(labelsAndValues...),
StepDuration: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "step_duration",
Help: "Time spent per step.",
Buckets: stdprometheus.ExponentialBucketsRange(0.1, 100, 8),
}, append(labels, "step")).With(labelsAndValues...),
QuorumPrevoteDelay: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "quorum_prevote_delay",
Help: "Difference in seconds between the proposal timestamp and the timestamp " +
"of the latest prevote that achieved a quorum in the prevote step.",
}, append(labels, "proposer_address")).With(labelsAndValues...),
FullPrevoteDelay: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "full_prevote_delay",
Help: "Difference in seconds between the proposal timestamp and the timestamp " +
"of the latest prevote that achieved 100% of the voting power in the prevote step.",
}, append(labels, "proposer_address")).With(labelsAndValues...),
ProposalTimestampDifference: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "proposal_timestamp_difference",
Help: "Difference in seconds between the timestamp in the proposal " +
"message and the local time when the message was received. " +
"Only calculated when a new block is proposed.",
Buckets: []float64{-10, -.5, -.025, 0, .1, .5, 1, 1.5, 2, 10},
}, append(labels, "is_timely")).With(labelsAndValues...),
VoteExtensionReceiveCount: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "vote_extension_receive_count",
Help: "Number of vote extensions received by the node since process start, labeled by " +
"the application's response to VerifyVoteExtension, either accept or reject.",
}, append(labels, "status")).With(labelsAndValues...),
ProposalReceiveCount: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "proposal_receive_count",
Help: "Number of vote proposals received by the node since process start, labeled by " +
"the application's response to ProcessProposal, either accept or reject.",
}, append(labels, "status")).With(labelsAndValues...),
ProposalCreateCount: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "proposal_create_count",
Help: "Number of proposals created by the node since process start.",
}, labels).With(labelsAndValues...),
RoundVotingPowerPercent: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "round_voting_power_percent",
Help: "Percentage of the total voting power received with a round. " +
"The value begins at 0 for each round and approaches 1.0 as additional " +
"voting power is observed.",
}, append(labels, "vote_type")).With(labelsAndValues...),
LateVotes: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "late_votes",
Help: "Number of votes received by the node since process start that correspond to earlier heights and rounds than this node is currently in.",
}, append(labels, "vote_type")).With(labelsAndValues...),
}
}
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
Height: discard.NewGauge(),
ValidatorLastSignedHeight: discard.NewGauge(),
Rounds: discard.NewGauge(),
RoundDuration: discard.NewHistogram(),
StepDuration: discard.NewHistogram(),
Validators: discard.NewGauge(),
ValidatorsPower: discard.NewGauge(),
ValidatorPower: discard.NewGauge(),
ValidatorMissedBlocks: discard.NewGauge(),
MissingValidators: discard.NewGauge(),
MissingValidatorsPower: discard.NewGauge(),
ByzantineValidators: discard.NewGauge(),
ByzantineValidatorsPower: discard.NewGauge(),
BlockIntervalSeconds: discard.NewHistogram(),
NumTxs: discard.NewGauge(),
BlockSizeBytes: discard.NewHistogram(),
TotalTxs: discard.NewGauge(),
CommittedHeight: discard.NewGauge(),
BlockSyncing: discard.NewGauge(),
StateSyncing: discard.NewGauge(),
BlockParts: discard.NewCounter(),
BlockGossipReceiveLatency: discard.NewHistogram(),
BlockGossipPartsReceived: discard.NewCounter(),
QuorumPrevoteDelay: discard.NewGauge(),
FullPrevoteDelay: discard.NewGauge(),
ProposalTimestampDifference: discard.NewHistogram(),
VoteExtensionReceiveCount: discard.NewCounter(),
ProposalReceiveCount: discard.NewCounter(),
ProposalCreateCount: discard.NewCounter(),
RoundVotingPowerPercent: discard.NewGauge(),
LateVotes: discard.NewCounter(),
}
//metrics:Number of votes received by the node since process start that correspond to earlier heights and rounds than this node is currently in.
LateVotes metrics.Counter `metrics_labels:"vote_type"`
}
// RecordConsMetrics uses for recording the block related metrics during fast-sync.

1
internal/consensus/mocks/cons_sync_reactor.go
View File

@@ -6,7 +6,6 @@ import (
testing "testing"
mock "github.com/stretchr/testify/mock"
state "github.com/tendermint/tendermint/internal/state"
)

18
internal/eventlog/eventlog.go
View File

@@ -24,9 +24,9 @@ import (
// any number of readers.
type Log struct {
// These values do not change after construction.
windowSize time.Duration
maxItems int
numItemsGauge gauge
windowSize time.Duration
maxItems int
metrics *Metrics
// Protects access to the fields below. Lock to modify the values of these
// fields, or to read or snapshot the values.
@@ -45,14 +45,14 @@ func New(opts LogSettings) (*Log, error) {
return nil, errors.New("window size must be positive")
}
lg := &Log{
windowSize: opts.WindowSize,
maxItems: opts.MaxItems,
numItemsGauge: discard{},
ready: make(chan struct{}),
source: opts.Source,
windowSize: opts.WindowSize,
maxItems: opts.MaxItems,
metrics: NopMetrics(),
ready: make(chan struct{}),
source: opts.Source,
}
if opts.Metrics != nil {
lg.numItemsGauge = opts.Metrics.numItemsGauge
lg.metrics = opts.Metrics
}
return lg, nil
}

30
internal/eventlog/metrics.gen.go Normal file
View File

@@ -0,0 +1,30 @@
// Code generated by metricsgen. DO NOT EDIT.
package eventlog
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
numItems: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_items",
Help: "Number of items currently resident in the event log.",
}, labels).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
numItems: discard.NewGauge(),
}
}

39
internal/eventlog/metrics.go
View File

@@ -1,39 +1,14 @@
package eventlog
import (
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
import "github.com/go-kit/kit/metrics"
// gauge is the subset of the Prometheus gauge interface used here.
type gauge interface {
Set(float64)
}
const MetricsSubsystem = "eventlog"
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics define the metrics exported by the eventlog package.
type Metrics struct {
numItemsGauge gauge
}
// discard is a no-op implementation of the gauge interface.
type discard struct{}
func (discard) Set(float64) {}
const eventlogSubsystem = "eventlog"
// PrometheusMetrics returns a collection of eventlog metrics for Prometheus.
func PrometheusMetrics(ns string, fields ...string) *Metrics {
var labels []string
for i := 0; i < len(fields); i += 2 {
labels = append(labels, fields[i])
}
return &Metrics{
numItemsGauge: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: ns,
Subsystem: eventlogSubsystem,
Name: "num_items",
Help: "Number of items currently resident in the event log.",
}, labels).With(fields...),
}
// Number of items currently resident in the event log.
numItems metrics.Gauge
}

4
internal/eventlog/prune.go
View File

@@ -12,7 +12,7 @@ func (lg *Log) checkPrune(head *logEntry, size int, age time.Duration) error {
const windowSlop = 30 * time.Second
if age < (lg.windowSize+windowSlop) && (lg.maxItems <= 0 || size <= lg.maxItems) {
lg.numItemsGauge.Set(float64(lg.numItems))
lg.metrics.numItems.Set(float64(lg.numItems))
return nil // no pruning is needed
}
@@ -46,7 +46,7 @@ func (lg *Log) checkPrune(head *logEntry, size int, age time.Duration) error {
lg.mu.Lock()
defer lg.mu.Unlock()
lg.numItems = newState.size
lg.numItemsGauge.Set(float64(newState.size))
lg.metrics.numItems.Set(float64(newState.size))
lg.oldestCursor = newState.oldest
lg.head = newState.head
return err

30
internal/evidence/metrics.gen.go Normal file
View File

@@ -0,0 +1,30 @@
// Code generated by metricsgen. DO NOT EDIT.
package evidence
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
NumEvidence: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_evidence",
Help: "Number of pending evidence in the evidence pool.",
}, labels).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
NumEvidence: discard.NewGauge(),
}
}

33
internal/evidence/metrics.go
View File

@@ -2,9 +2,6 @@ package evidence
import (
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -13,35 +10,11 @@ const (
MetricsSubsystem = "evidence_pool"
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains metrics exposed by this package.
// see MetricsProvider for descriptions.
type Metrics struct {
// Number of evidence in the evidence pool
// Number of pending evidence in the evidence pool.
NumEvidence metrics.Gauge
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
NumEvidence: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_evidence",
Help: "Number of pending evidence in evidence pool.",
}, labels).With(labelsAndValues...),
}
}
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
NumEvidence: discard.NewGauge(),
}
}

67
internal/mempool/metrics.gen.go Normal file
View File

@@ -0,0 +1,67 @@
// Code generated by metricsgen. DO NOT EDIT.
package mempool
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Size: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "size",
Help: "Number of uncommitted transactions in the mempool.",
}, labels).With(labelsAndValues...),
TxSizeBytes: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "tx_size_bytes",
Help: "Histogram of transaction sizes in bytes.",
Buckets: stdprometheus.ExponentialBuckets(1, 3, 7),
}, labels).With(labelsAndValues...),
FailedTxs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "failed_txs",
Help: "Number of failed transactions.",
}, labels).With(labelsAndValues...),
RejectedTxs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "rejected_txs",
Help: "Number of rejected transactions.",
}, labels).With(labelsAndValues...),
EvictedTxs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "evicted_txs",
Help: "Number of evicted transactions.",
}, labels).With(labelsAndValues...),
RecheckTimes: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "recheck_times",
Help: "Number of times transactions are rechecked in the mempool.",
}, labels).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
Size: discard.NewGauge(),
TxSizeBytes: discard.NewHistogram(),
FailedTxs: discard.NewCounter(),
RejectedTxs: discard.NewCounter(),
EvictedTxs: discard.NewCounter(),
RecheckTimes: discard.NewCounter(),
}
}

79
internal/mempool/metrics.go
View File

@@ -2,9 +2,6 @@ package mempool
import (
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -13,14 +10,16 @@ const (
MetricsSubsystem = "mempool"
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains metrics exposed by this package.
// see MetricsProvider for descriptions.
type Metrics struct {
// Size of the mempool.
// Number of uncommitted transactions in the mempool.
Size metrics.Gauge
// Histogram of transaction sizes, in bytes.
TxSizeBytes metrics.Histogram
// Histogram of transaction sizes in bytes.
TxSizeBytes metrics.Histogram `metrics_buckettype:"exp" metrics_bucketsizes:"1,3,7"`
// Number of failed transactions.
FailedTxs metrics.Counter
@@ -29,80 +28,16 @@ type Metrics struct {
// transactions that passed CheckTx but failed to make it into the mempool
// due to resource limits, e.g. mempool is full and no lower priority
// transactions exist in the mempool.
//metrics:Number of rejected transactions.
RejectedTxs metrics.Counter
// EvictedTxs defines the number of evicted transactions. These are valid
// transactions that passed CheckTx and existed in the mempool but were later
// evicted to make room for higher priority valid transactions that passed
// CheckTx.
//metrics:Number of evicted transactions.
EvictedTxs metrics.Counter
// Number of times transactions are rechecked in the mempool.
RecheckTimes metrics.Counter
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Size: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "size",
Help: "Size of the mempool (number of uncommitted transactions).",
}, labels).With(labelsAndValues...),
TxSizeBytes: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "tx_size_bytes",
Help: "Transaction sizes in bytes.",
Buckets: stdprometheus.ExponentialBuckets(1, 3, 17),
}, labels).With(labelsAndValues...),
FailedTxs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "failed_txs",
Help: "Number of failed transactions.",
}, labels).With(labelsAndValues...),
RejectedTxs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "rejected_txs",
Help: "Number of rejected transactions.",
}, labels).With(labelsAndValues...),
EvictedTxs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "evicted_txs",
Help: "Number of evicted transactions.",
}, labels).With(labelsAndValues...),
RecheckTimes: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "recheck_times",
Help: "Number of times transactions are rechecked in the mempool.",
}, labels).With(labelsAndValues...),
}
}
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
Size: discard.NewGauge(),
TxSizeBytes: discard.NewHistogram(),
FailedTxs: discard.NewCounter(),
RejectedTxs: discard.NewCounter(),
EvictedTxs: discard.NewCounter(),
RecheckTimes: discard.NewCounter(),
}
}

9
internal/mempool/reactor.go
View File

@@ -153,6 +153,15 @@ func (r *Reactor) handleMempoolMessage(ctx context.Context, envelope *p2p.Envelo
// problem.
continue
}
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
// Do not propagate context
// cancellation errors, but do
// not continue to check
// transactions from this
// message if we are shutting down.
return nil
}
logger.Error("checktx failed for tx",
"tx", fmt.Sprintf("%X", types.Tx(tx).Hash()),
"err", err)

86
internal/p2p/metrics.gen.go Normal file
View File

@@ -0,0 +1,86 @@
// Code generated by metricsgen. DO NOT EDIT.
package p2p
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Peers: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peers",
Help: "Number of peers.",
}, labels).With(labelsAndValues...),
PeerReceiveBytesTotal: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_receive_bytes_total",
Help: "Number of bytes per channel received from a given peer.",
}, append(labels, "peer_id", "chID", "message_type")).With(labelsAndValues...),
PeerSendBytesTotal: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_send_bytes_total",
Help: "Number of bytes per channel sent to a given peer.",
}, append(labels, "peer_id", "chID", "message_type")).With(labelsAndValues...),
PeerPendingSendBytes: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_pending_send_bytes",
Help: "Number of bytes pending being sent to a given peer.",
}, append(labels, "peer_id")).With(labelsAndValues...),
RouterPeerQueueRecv: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_peer_queue_recv",
Help: "The time taken to read off of a peer's queue before sending on the connection.",
}, labels).With(labelsAndValues...),
RouterPeerQueueSend: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_peer_queue_send",
Help: "The time taken to send on a peer's queue which will later be read and sent on the connection.",
}, labels).With(labelsAndValues...),
RouterChannelQueueSend: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_channel_queue_send",
Help: "The time taken to send on a p2p channel's queue which will later be consued by the corresponding reactor/service.",
}, labels).With(labelsAndValues...),
PeerQueueDroppedMsgs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_channel_queue_dropped_msgs",
Help: "The number of messages dropped from a peer's queue for a specific p2p Channel.",
}, append(labels, "ch_id")).With(labelsAndValues...),
PeerQueueMsgSize: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_queue_msg_size",
Help: "The size of messages sent over a peer's queue for a specific p2p Channel.",
}, append(labels, "ch_id")).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
Peers: discard.NewGauge(),
PeerReceiveBytesTotal: discard.NewCounter(),
PeerSendBytesTotal: discard.NewCounter(),
PeerPendingSendBytes: discard.NewGauge(),
RouterPeerQueueRecv: discard.NewHistogram(),
RouterPeerQueueSend: discard.NewHistogram(),
RouterChannelQueueSend: discard.NewHistogram(),
PeerQueueDroppedMsgs: discard.NewCounter(),
PeerQueueMsgSize: discard.NewGauge(),
}
}

131
internal/p2p/metrics.go
View File

@@ -7,9 +7,6 @@ import (
"sync"
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -25,140 +22,55 @@ var (
valueToLabelRegexp = regexp.MustCompile(`\*?(\w+)\.(.*)`)
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains metrics exposed by this package.
type Metrics struct {
// Number of peers.
Peers metrics.Gauge
// Number of bytes received from a given peer.
PeerReceiveBytesTotal metrics.Counter
// Number of bytes sent to a given peer.
PeerSendBytesTotal metrics.Counter
// Pending bytes to be sent to a given peer.
PeerPendingSendBytes metrics.Gauge
// Number of bytes per channel received from a given peer.
PeerReceiveBytesTotal metrics.Counter `metrics_labels:"peer_id, chID, message_type"`
// Number of bytes per channel sent to a given peer.
PeerSendBytesTotal metrics.Counter `metrics_labels:"peer_id, chID, message_type"`
// Number of bytes pending being sent to a given peer.
PeerPendingSendBytes metrics.Gauge `metrics_labels:"peer_id"`
// RouterPeerQueueRecv defines the time taken to read off of a peer's queue
// before sending on the connection.
//metrics:The time taken to read off of a peer's queue before sending on the connection.
RouterPeerQueueRecv metrics.Histogram
// RouterPeerQueueSend defines the time taken to send on a peer's queue which
// will later be read and sent on the connection (see RouterPeerQueueRecv).
//metrics:The time taken to send on a peer's queue which will later be read and sent on the connection.
RouterPeerQueueSend metrics.Histogram
// RouterChannelQueueSend defines the time taken to send on a p2p channel's
// queue which will later be consued by the corresponding reactor/service.
//metrics:The time taken to send on a p2p channel's queue which will later be consued by the corresponding reactor/service.
RouterChannelQueueSend metrics.Histogram
// PeerQueueDroppedMsgs defines the number of messages dropped from a peer's
// queue for a specific flow (i.e. Channel).
PeerQueueDroppedMsgs metrics.Counter
//metrics:The number of messages dropped from a peer's queue for a specific p2p Channel.
PeerQueueDroppedMsgs metrics.Counter `metrics_labels:"ch_id" metrics_name:"router_channel_queue_dropped_msgs"`
// PeerQueueMsgSize defines the average size of messages sent over a peer's
// queue for a specific flow (i.e. Channel).
PeerQueueMsgSize metrics.Gauge
//metrics:The size of messages sent over a peer's queue for a specific p2p Channel.
PeerQueueMsgSize metrics.Gauge `metrics_labels:"ch_id" metric_name:"router_channel_queue_msg_size"`
}
type metricsLabelCache struct {
mtx *sync.RWMutex
messageLabelNames map[reflect.Type]string
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Peers: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peers",
Help: "Number of peers.",
}, labels).With(labelsAndValues...),
PeerReceiveBytesTotal: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_receive_bytes_total",
Help: "Number of bytes received from a given peer.",
}, append(labels, "peer_id", "chID", "message_type")).With(labelsAndValues...),
PeerSendBytesTotal: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_send_bytes_total",
Help: "Number of bytes sent to a given peer.",
}, append(labels, "peer_id", "chID", "message_type")).With(labelsAndValues...),
PeerPendingSendBytes: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "peer_pending_send_bytes",
Help: "Number of pending bytes to be sent to a given peer.",
}, append(labels, "peer_id")).With(labelsAndValues...),
RouterPeerQueueRecv: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_peer_queue_recv",
Help: "The time taken to read off of a peer's queue before sending on the connection.",
}, labels).With(labelsAndValues...),
RouterPeerQueueSend: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_peer_queue_send",
Help: "The time taken to send on a peer's queue which will later be read and sent on the connection (see RouterPeerQueueRecv).",
}, labels).With(labelsAndValues...),
RouterChannelQueueSend: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_channel_queue_send",
Help: "The time taken to send on a p2p channel's queue which will later be consued by the corresponding reactor/service.",
}, labels).With(labelsAndValues...),
PeerQueueDroppedMsgs: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_channel_queue_dropped_msgs",
Help: "The number of messages dropped from a peer's queue for a specific p2p Channel.",
}, append(labels, "ch_id")).With(labelsAndValues...),
PeerQueueMsgSize: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "router_channel_queue_msg_size",
Help: "The size of messages sent over a peer's queue for a specific p2p Channel.",
}, append(labels, "ch_id")).With(labelsAndValues...),
mtx: &sync.RWMutex{},
messageLabelNames: map[reflect.Type]string{},
}
}
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
Peers: discard.NewGauge(),
PeerReceiveBytesTotal: discard.NewCounter(),
PeerSendBytesTotal: discard.NewCounter(),
PeerPendingSendBytes: discard.NewGauge(),
RouterPeerQueueRecv: discard.NewHistogram(),
RouterPeerQueueSend: discard.NewHistogram(),
RouterChannelQueueSend: discard.NewHistogram(),
PeerQueueDroppedMsgs: discard.NewCounter(),
PeerQueueMsgSize: discard.NewGauge(),
mtx: &sync.RWMutex{},
messageLabelNames: map[reflect.Type]string{},
}
}
// ValueToMetricLabel is a method that is used to produce a prometheus label value of the golang
// type that is passed in.
// This method uses a map on the Metrics struct so that each label name only needs
// to be produced once to prevent expensive string operations.
func (m *Metrics) ValueToMetricLabel(i interface{}) string {
func (m *metricsLabelCache) ValueToMetricLabel(i interface{}) string {
t := reflect.TypeOf(i)
m.mtx.RLock()
@@ -176,3 +88,10 @@ func (m *Metrics) ValueToMetricLabel(i interface{}) string {
m.messageLabelNames[t] = l
return l
}
func newMetricsLabelCache() *metricsLabelCache {
return &metricsLabelCache{
mtx: &sync.RWMutex{},
messageLabelNames: map[reflect.Type]string{},
}
}

6
internal/p2p/metrics_test.go
View File

@@ -9,12 +9,12 @@ import (
)
func TestValueToMetricsLabel(t *testing.T) {
m := NopMetrics()
lc := newMetricsLabelCache()
r := &p2p.PexResponse{}
str := m.ValueToMetricLabel(r)
str := lc.ValueToMetricLabel(r)
assert.Equal(t, "p2p_PexResponse", str)
// subsequent calls to the function should produce the same result
str = m.ValueToMetricLabel(r)
str = lc.ValueToMetricLabel(r)
assert.Equal(t, "p2p_PexResponse", str)
}

5
internal/p2p/pqueue.go
View File

@@ -70,6 +70,7 @@ var _ queue = (*pqScheduler)(nil)
type pqScheduler struct {
logger log.Logger
metrics *Metrics
lc *metricsLabelCache
size uint
sizes map[uint]uint // cumulative priority sizes
pq *priorityQueue
@@ -88,6 +89,7 @@ type pqScheduler struct {
func newPQScheduler(
logger log.Logger,
m *Metrics,
lc *metricsLabelCache,
chDescs []*ChannelDescriptor,
enqueueBuf, dequeueBuf, capacity uint,
) *pqScheduler {
@@ -117,6 +119,7 @@ func newPQScheduler(
return &pqScheduler{
logger: logger.With("router", "scheduler"),
metrics: m,
lc: lc,
chDescs: chDescsCopy,
capacity: capacity,
chPriorities: chPriorities,
@@ -251,7 +254,7 @@ func (s *pqScheduler) process(ctx context.Context) {
s.metrics.PeerSendBytesTotal.With(
"chID", chIDStr,
"peer_id", string(pqEnv.envelope.To),
"message_type", s.metrics.ValueToMetricLabel(pqEnv.envelope.Message)).Add(float64(pqEnv.size))
"message_type", s.lc.ValueToMetricLabel(pqEnv.envelope.Message)).Add(float64(pqEnv.size))
s.metrics.PeerPendingSendBytes.With(
"peer_id", string(pqEnv.envelope.To)).Add(float64(-pqEnv.size))
select {

2
internal/p2p/pqueue_test.go
View File

@@ -17,7 +17,7 @@ func TestCloseWhileDequeueFull(t *testing.T) {
chDescs := []*ChannelDescriptor{
{ID: 0x01, Priority: 1},
}
pqueue := newPQScheduler(log.NewNopLogger(), NopMetrics(), chDescs, uint(enqueueLength), 1, 120)
pqueue := newPQScheduler(log.NewNopLogger(), NopMetrics(), newMetricsLabelCache(), chDescs, uint(enqueueLength), 1, 120)
for i := 0; i < enqueueLength; i++ {
pqueue.enqueue() <- Envelope{

9
internal/p2p/router.go
View File

@@ -148,7 +148,9 @@ type Router struct {
*service.BaseService
logger log.Logger
metrics *Metrics
metrics *Metrics
lc *metricsLabelCache
options RouterOptions
privKey crypto.PrivKey
peerManager *PeerManager
@@ -193,6 +195,7 @@ func NewRouter(
router := &Router{
logger: logger,
metrics: metrics,
lc: newMetricsLabelCache(),
privKey: privKey,
nodeInfoProducer: nodeInfoProducer,
connTracker: newConnTracker(
@@ -226,7 +229,7 @@ func (r *Router) createQueueFactory(ctx context.Context) (func(int) queue, error
size++
}
q := newPQScheduler(r.logger, r.metrics, r.chDescs, uint(size)/2, uint(size)/2, defaultCapacity)
q := newPQScheduler(r.logger, r.metrics, r.lc, r.chDescs, uint(size)/2, uint(size)/2, defaultCapacity)
q.start(ctx)
return q
}, nil
@@ -839,7 +842,7 @@ func (r *Router) receivePeer(ctx context.Context, peerID types.NodeID, conn Conn
r.metrics.PeerReceiveBytesTotal.With(
"chID", fmt.Sprint(chID),
"peer_id", string(peerID),
"message_type", r.metrics.ValueToMetricLabel(msg)).Add(float64(proto.Size(msg)))
"message_type", r.lc.ValueToMetricLabel(msg)).Add(float64(proto.Size(msg)))
r.metrics.RouterChannelQueueSend.Observe(time.Since(start).Seconds())
r.logger.Debug("received message", "peer", peerID, "message", msg)

32
internal/proxy/metrics.gen.go Normal file
View File

@@ -0,0 +1,32 @@
// Code generated by metricsgen. DO NOT EDIT.
package proxy
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
MethodTiming: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "method_timing",
Help: "Timing for each ABCI method.",
Buckets: []float64{.0001, .0004, .002, .009, .02, .1, .65, 2, 6, 25},
}, append(labels, "method", "type")).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
MethodTiming: discard.NewHistogram(),
}
}

36
internal/proxy/metrics.go
View File

@@ -2,9 +2,6 @@ package proxy
import (
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -13,35 +10,10 @@ const (
MetricsSubsystem = "abci_connection"
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains the prometheus metrics exposed by the proxy package.
type Metrics struct {
MethodTiming metrics.Histogram
}
// PrometheusMetrics constructs a Metrics instance that collects metrics samples.
// The resulting metrics will be prefixed with namespace and labeled with the
// defaultLabelsAndValues. defaultLabelsAndValues must be a list of string pairs
// where the first of each pair is the label and the second is the value.
func PrometheusMetrics(namespace string, defaultLabelsAndValues ...string) *Metrics {
defaultLabels := []string{}
for i := 0; i < len(defaultLabelsAndValues); i += 2 {
defaultLabels = append(defaultLabels, defaultLabelsAndValues[i])
}
return &Metrics{
MethodTiming: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "method_timing",
Help: "ABCI Method Timing",
Buckets: []float64{.0001, .0004, .002, .009, .02, .1, .65, 2, 6, 25},
}, append(defaultLabels, []string{"method", "type"}...)).With(defaultLabelsAndValues...),
}
}
// NopMetrics constructs a Metrics instance that discards all samples and is suitable
// for testing.
func NopMetrics() *Metrics {
return &Metrics{
MethodTiming: discard.NewHistogram(),
}
// Timing for each ABCI method.
MethodTiming metrics.Histogram `metrics_bucketsizes:".0001,.0004,.002,.009,.02,.1,.65,2,6,25" metrics_labels:"method, type"`
}

2
internal/state/indexer/block/kv/kv.go
View File

@@ -65,7 +65,7 @@ func (idx *BlockerIndexer) Index(bh types.EventDataNewBlockHeader) error {
}
// 2. index FinalizeBlock events
if err := idx.indexEvents(batch, bh.ResultFinalizeBlock.Events, types.EventTypeFinalizeBlock, height); err != nil {
if err := idx.indexEvents(batch, bh.ResultFinalizeBlock.Events, "finalize_block", height); err != nil {
return fmt.Errorf("failed to index FinalizeBlock events: %w", err)
}

2
internal/state/indexer/metrics.go
View File

@@ -4,7 +4,7 @@ import (
"github.com/go-kit/kit/metrics"
)
//go:generate go run github.com/tendermint/tendermint/scripts/metricsgen -struct=Metrics
//go:generate go run ../../../scripts/metricsgen -struct=Metrics
// MetricsSubsystem is a the subsystem label for the indexer package.
const MetricsSubsystem = "indexer"

1
internal/state/indexer/mocks/event_sink.go
View File

@@ -6,7 +6,6 @@ import (
context "context"
mock "github.com/stretchr/testify/mock"
indexer "github.com/tendermint/tendermint/internal/state/indexer"
query "github.com/tendermint/tendermint/internal/pubsub/query"

46
internal/state/metrics.gen.go Normal file
View File

@@ -0,0 +1,46 @@
// Code generated by metricsgen. DO NOT EDIT.
package state
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
BlockProcessingTime: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_processing_time",
Help: "Time between BeginBlock and EndBlock.",
Buckets: stdprometheus.LinearBuckets(1, 10, 10),
}, labels).With(labelsAndValues...),
ConsensusParamUpdates: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "consensus_param_updates",
Help: "Number of consensus parameter updates returned by the application since process start.",
}, labels).With(labelsAndValues...),
ValidatorSetUpdates: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_set_updates",
Help: "Number of validator set updates returned by the application since process start.",
}, labels).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
BlockProcessingTime: discard.NewHistogram(),
ConsensusParamUpdates: discard.NewCounter(),
ValidatorSetUpdates: discard.NewCounter(),
}
}

52
internal/state/metrics.go
View File

@@ -2,9 +2,6 @@ package state
import (
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -13,59 +10,20 @@ const (
MetricsSubsystem = "state"
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains metrics exposed by this package.
type Metrics struct {
// Time between BeginBlock and EndBlock.
BlockProcessingTime metrics.Histogram
BlockProcessingTime metrics.Histogram `metrics_buckettype:"lin" metrics_bucketsizes:"1,10,10"`
// ConsensusParamUpdates is the total number of times the application has
// udated the consensus params since process start.
//metrics:Number of consensus parameter updates returned by the application since process start.
ConsensusParamUpdates metrics.Counter
// ValidatorSetUpdates is the total number of times the application has
// udated the validator set since process start.
//metrics:Number of validator set updates returned by the application since process start.
ValidatorSetUpdates metrics.Counter
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
BlockProcessingTime: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_processing_time",
Help: "Time between BeginBlock and EndBlock in ms.",
Buckets: stdprometheus.LinearBuckets(1, 10, 10),
}, labels).With(labelsAndValues...),
ConsensusParamUpdates: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "consensus_param_updates",
Help: "The total number of times the application as updated the consensus " +
"parameters since process start.",
}, labels).With(labelsAndValues...),
ValidatorSetUpdates: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validator_set_updates",
Help: "The total number of times the application as updated the validator " +
"set since process start.",
}, labels).With(labelsAndValues...),
}
}
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
BlockProcessingTime: discard.NewHistogram(),
ConsensusParamUpdates: discard.NewCounter(),
ValidatorSetUpdates: discard.NewCounter(),
}
}

1
internal/state/mocks/evidence_pool.go
View File

@@ -6,7 +6,6 @@ import (
context "context"
mock "github.com/stretchr/testify/mock"
state "github.com/tendermint/tendermint/internal/state"
testing "testing"

1
internal/state/mocks/store.go
View File

@@ -4,7 +4,6 @@ package mocks
import (
mock "github.com/stretchr/testify/mock"
state "github.com/tendermint/tendermint/internal/state"
tendermintstate "github.com/tendermint/tendermint/proto/tendermint/state"

72
internal/statesync/metrics.gen.go Normal file
View File

@@ -0,0 +1,72 @@
// Code generated by metricsgen. DO NOT EDIT.
package statesync
import (
"github.com/go-kit/kit/metrics/discard"
prometheus "github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
TotalSnapshots: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "total_snapshots",
Help: "The total number of snapshots discovered.",
}, labels).With(labelsAndValues...),
ChunkProcessAvgTime: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "chunk_process_avg_time",
Help: "The average processing time per chunk.",
}, labels).With(labelsAndValues...),
SnapshotHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "snapshot_height",
Help: "The height of the current snapshot the has been processed.",
}, labels).With(labelsAndValues...),
SnapshotChunk: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "snapshot_chunk",
Help: "The current number of chunks that have been processed.",
}, labels).With(labelsAndValues...),
SnapshotChunkTotal: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "snapshot_chunk_total",
Help: "The total number of chunks in the current snapshot.",
}, labels).With(labelsAndValues...),
BackFilledBlocks: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "back_filled_blocks",
Help: "The current number of blocks that have been back-filled.",
}, labels).With(labelsAndValues...),
BackFillBlocksTotal: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "back_fill_blocks_total",
Help: "The total number of blocks that need to be back-filled.",
}, labels).With(labelsAndValues...),
}
}
func NopMetrics() *Metrics {
return &Metrics{
TotalSnapshots: discard.NewCounter(),
ChunkProcessAvgTime: discard.NewGauge(),
SnapshotHeight: discard.NewGauge(),
SnapshotChunk: discard.NewCounter(),
SnapshotChunkTotal: discard.NewGauge(),
BackFilledBlocks: discard.NewCounter(),
BackFillBlocksTotal: discard.NewGauge(),
}
}

89
internal/statesync/metrics.go
View File

@@ -2,9 +2,6 @@ package statesync
import (
"github.com/go-kit/kit/metrics"
"github.com/go-kit/kit/metrics/discard"
"github.com/go-kit/kit/metrics/prometheus"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
@@ -12,80 +9,22 @@ const (
MetricsSubsystem = "statesync"
)
//go:generate go run ../../scripts/metricsgen -struct=Metrics
// Metrics contains metrics exposed by this package.
type Metrics struct {
TotalSnapshots metrics.Counter
// The total number of snapshots discovered.
TotalSnapshots metrics.Counter
// The average processing time per chunk.
ChunkProcessAvgTime metrics.Gauge
SnapshotHeight metrics.Gauge
SnapshotChunk metrics.Counter
SnapshotChunkTotal metrics.Gauge
BackFilledBlocks metrics.Counter
// The height of the current snapshot the has been processed.
SnapshotHeight metrics.Gauge
// The current number of chunks that have been processed.
SnapshotChunk metrics.Counter
// The total number of chunks in the current snapshot.
SnapshotChunkTotal metrics.Gauge
// The current number of blocks that have been back-filled.
BackFilledBlocks metrics.Counter
// The total number of blocks that need to be back-filled.
BackFillBlocksTotal metrics.Gauge
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
TotalSnapshots: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "total_snapshots",
Help: "The total number of snapshots discovered.",
}, labels).With(labelsAndValues...),
ChunkProcessAvgTime: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "chunk_process_avg_time",
Help: "The average processing time per chunk.",
}, labels).With(labelsAndValues...),
SnapshotHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "snapshot_height",
Help: "The height of the current snapshot the has been processed.",
}, labels).With(labelsAndValues...),
SnapshotChunk: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "snapshot_chunk",
Help: "The current number of chunks that have been processed.",
}, labels).With(labelsAndValues...),
SnapshotChunkTotal: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "snapshot_chunks_total",
Help: "The total number of chunks in the current snapshot.",
}, labels).With(labelsAndValues...),
BackFilledBlocks: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "backfilled_blocks",
Help: "The current number of blocks that have been back-filled.",
}, labels).With(labelsAndValues...),
BackFillBlocksTotal: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "backfilled_blocks_total",
Help: "The total number of blocks that need to be back-filled.",
}, labels).With(labelsAndValues...),
}
}
// NopMetrics returns no-op Metrics.
func NopMetrics() *Metrics {
return &Metrics{
TotalSnapshots: discard.NewCounter(),
ChunkProcessAvgTime: discard.NewGauge(),
SnapshotHeight: discard.NewGauge(),
SnapshotChunk: discard.NewCounter(),
SnapshotChunkTotal: discard.NewGauge(),
BackFilledBlocks: discard.NewCounter(),
BackFillBlocksTotal: discard.NewGauge(),
}
}

1
internal/statesync/mocks/state_provider.go
View File

@@ -6,7 +6,6 @@ import (
context "context"
mock "github.com/stretchr/testify/mock"
state "github.com/tendermint/tendermint/internal/state"
testing "testing"

23
rpc/openapi/openapi.yaml
View File

@@ -260,10 +260,9 @@ paths:
operationId: events
description: |
Fetch a batch of events posted by the consensus node and matching a
specified query.
specified query string.
The query grammar is defined in
https://godoc.org/github.com/tendermint/tendermint/internal/pubsub/query/syntax.
The query grammar is defined in [pubsub/query/syntax](https://godoc.org/github.com/tendermint/tendermint/internal/pubsub/query/syntax).
An empty query matches all events; otherwise a query comprises one or
more terms comparing event metadata to target values. For example, to
select new block events:
@@ -275,13 +274,13 @@ paths:
tm.event = 'Tx' AND tx.hash = 'EA7B33F'
The comparison operators include "=", "<", "<=", ">", ">=", and
"CONTAINS". Operands may be strings (in single quotes), numbers, dates,
or timestamps. In addition, the "EXISTS" operator allows you to check
The comparison operators include `=`, `<`, `<=`, `>`, `>=`, and
`CONTAINS`. Operands may be strings (in single quotes), numbers, dates,
or timestamps. In addition, the `EXISTS` operator allows you to check
for the presence of an attribute regardless of its value.
Tendermint defines a tm.event attribute for all events. Transactions
are also assigned tx.hash and tx.height attributes. Other attributes
Tendermint defines a `tm.event` attribute for all events. Transactions
are also assigned `tx.hash` and `tx.height` attributes. Other attributes
are provided by the application as ABCI Event records. The name of the
event in the query is formed by combining the type and attribute key
with a period. For example, given:
@@ -295,16 +294,16 @@ paths:
},
}}
the query may refer to the names "reward.address", "reward.amount", and
"reward.balance", as in:
the query may refer to the names`"reward.address`,`"reward.amount`, and
`reward.balance`, as in:
reward.address EXISTS AND reward.balance > 45
The node maintains a log of all events within an operator-defined time
window. The /events method returns the most recent items from the log
that match the query. Each item returned includes a cursor that marks
its location in the log. Cursors can be passed via the "before" and
"after" parameters to fetch events earlier in the log.
its location in the log. Cursors can be passed via the `before` and
`after` parameters to fetch events earlier in the log.
parameters:
- in: query
name: filter

197
scripts/metricsgen/metricsdiff/metricsdiff.go Normal file
View File

@@ -0,0 +1,197 @@
// metricsdiff is a tool for generating a diff between two different files containing
// prometheus metrics. metricsdiff outputs which metrics have been added, removed,
// or have different sets of labels between the two files.
package main
import (
"flag"
"fmt"
"io"
"log"
"os"
"path/filepath"
"sort"
"strings"
dto "github.com/prometheus/client_model/go"
"github.com/prometheus/common/expfmt"
)
func init() {
flag.Usage = func() {
fmt.Fprintf(os.Stderr, `Usage: %[1]s <path1> <path2>
Generate the diff between the two files of Prometheus metrics.
The input should have the format output by a Prometheus HTTP endpoint.
The tool indicates which metrics have been added, removed, or use different
label sets from path1 to path2.
`, filepath.Base(os.Args[0]))
flag.PrintDefaults()
}
}
// Diff contains the set of metrics that were modified between two files
// containing prometheus metrics output.
type Diff struct {
Adds []string
Removes []string
Changes []LabelDiff
}
// LabelDiff describes the label changes between two versions of the same metric.
type LabelDiff struct {
Metric string
Adds []string
Removes []string
}
type parsedMetric struct {
name string
labels []string
}
type metricsList []parsedMetric
func main() {
flag.Parse()
if flag.NArg() != 2 {
log.Fatalf("Usage is '%s <path1> <path2>', got %d arguments",
filepath.Base(os.Args[0]), flag.NArg())
}
fa, err := os.Open(flag.Arg(0))
if err != nil {
log.Fatalf("Open: %v", err)
}
defer fa.Close()
fb, err := os.Open(flag.Arg(1))
if err != nil {
log.Fatalf("Open: %v", err)
}
defer fb.Close()
md, err := DiffFromReaders(fa, fb)
if err != nil {
log.Fatalf("Generating diff: %v", err)
}
fmt.Print(md)
}
// DiffFromReaders parses the metrics present in the readers a and b and
// determines which metrics were added and removed in b.
func DiffFromReaders(a, b io.Reader) (Diff, error) {
var parser expfmt.TextParser
amf, err := parser.TextToMetricFamilies(a)
if err != nil {
return Diff{}, err
}
bmf, err := parser.TextToMetricFamilies(b)
if err != nil {
return Diff{}, err
}
md := Diff{}
aList := toList(amf)
bList := toList(bmf)
i, j := 0, 0
for i < len(aList) || j < len(bList) {
for j < len(bList) && (i >= len(aList) || bList[j].name < aList[i].name) {
md.Adds = append(md.Adds, bList[j].name)
j++
}
for i < len(aList) && j < len(bList) && aList[i].name == bList[j].name {
adds, removes := listDiff(aList[i].labels, bList[j].labels)
if len(adds) > 0 || len(removes) > 0 {
md.Changes = append(md.Changes, LabelDiff{
Metric: aList[i].name,
Adds: adds,
Removes: removes,
})
}
i++
j++
}
for i < len(aList) && (j >= len(bList) || aList[i].name < bList[j].name) {
md.Removes = append(md.Removes, aList[i].name)
i++
}
}
return md, nil
}
func toList(l map[string]*dto.MetricFamily) metricsList {
r := make([]parsedMetric, len(l))
var idx int
for name, family := range l {
r[idx] = parsedMetric{
name: name,
labels: labelsToStringList(family.Metric[0].Label),
}
idx++
}
sort.Sort(metricsList(r))
return r
}
func labelsToStringList(ls []*dto.LabelPair) []string {
r := make([]string, len(ls))
for i, l := range ls {
r[i] = l.GetName()
}
return sort.StringSlice(r)
}
func listDiff(a, b []string) ([]string, []string) {
adds, removes := []string{}, []string{}
i, j := 0, 0
for i < len(a) || j < len(b) {
for j < len(b) && (i >= len(a) || b[j] < a[i]) {
adds = append(adds, b[j])
j++
}
for i < len(a) && j < len(b) && a[i] == b[j] {
i++
j++
}
for i < len(a) && (j >= len(b) || a[i] < b[j]) {
removes = append(removes, a[i])
i++
}
}
return adds, removes
}
func (m metricsList) Len() int { return len(m) }
func (m metricsList) Less(i, j int) bool { return m[i].name < m[j].name }
func (m metricsList) Swap(i, j int) { m[i], m[j] = m[j], m[i] }
func (m Diff) String() string {
var s strings.Builder
if len(m.Adds) > 0 || len(m.Removes) > 0 {
fmt.Fprintln(&s, "Metric changes:")
}
if len(m.Adds) > 0 {
for _, add := range m.Adds {
fmt.Fprintf(&s, "+++ %s\n", add)
}
}
if len(m.Removes) > 0 {
for _, rem := range m.Removes {
fmt.Fprintf(&s, "--- %s\n", rem)
}
}
if len(m.Changes) > 0 {
fmt.Fprintln(&s, "Label changes:")
for _, ld := range m.Changes {
fmt.Fprintf(&s, "Metric: %s\n", ld.Metric)
for _, add := range ld.Adds {
fmt.Fprintf(&s, "+++ %s\n", add)
}
for _, rem := range ld.Removes {
fmt.Fprintf(&s, "--- %s\n", rem)
}
}
}
return s.String()
}

62
scripts/metricsgen/metricsdiff/metricsdiff_test.go Normal file
View File

@@ -0,0 +1,62 @@
package main_test
import (
"bytes"
"io"
"testing"
"github.com/stretchr/testify/require"
metricsdiff "github.com/tendermint/tendermint/scripts/metricsgen/metricsdiff"
)
func TestDiff(t *testing.T) {
for _, tc := range []struct {
name string
aContents string
bContents string
want string
}{
{
name: "labels",
aContents: `
metric_one{label_one="content", label_two="content"} 0
`,
bContents: `
metric_one{label_three="content", label_four="content"} 0
`,
want: `Label changes:
Metric: metric_one
+++ label_three
+++ label_four
--- label_one
--- label_two
`,
},
{
name: "metrics",
aContents: `
metric_one{label_one="content"} 0
`,
bContents: `
metric_two{label_two="content"} 0
`,
want: `Metric changes:
+++ metric_two
--- metric_one
`,
},
} {
t.Run(tc.name, func(t *testing.T) {
bufA := bytes.NewBuffer([]byte{})
bufB := bytes.NewBuffer([]byte{})
_, err := io.WriteString(bufA, tc.aContents)
require.NoError(t, err)
_, err = io.WriteString(bufB, tc.bContents)
require.NoError(t, err)
md, err := metricsdiff.DiffFromReaders(bufA, bufB)
require.NoError(t, err)
require.Equal(t, tc.want, md.String())
})
}
}

33
scripts/metricsgen/metricsgen.go
View File

@@ -88,8 +88,8 @@ func PrometheusMetrics(namespace string, labelsAndValues...string) *Metrics {
{{- if eq (len $metric.Labels) 0 }}
}, labels).With(labelsAndValues...),
{{ else }}
}, append(labels, {{$metric.Labels | printf "%q" }})).With(labelsAndValues...),
{{- end }}
}, append(labels, {{$metric.Labels}})).With(labelsAndValues...),
{{ end }}
{{- end }}
}
}
@@ -249,14 +249,8 @@ func findMetricsStruct(files map[string]*ast.File, structName string) (*ast.Stru
}
func parseMetricField(f *ast.Field) ParsedMetricField {
var comment string
if f.Doc != nil {
for _, c := range f.Doc.List {
comment += strings.TrimPrefix(c.Text, "// ")
}
}
pmf := ParsedMetricField{
Description: comment,
Description: extractHelpMessage(f.Doc),
MetricName: extractFieldName(f.Names[0].String(), f.Tag),
FieldName: f.Names[0].String(),
TypeName: extractTypeName(f.Type),
@@ -272,6 +266,21 @@ func extractTypeName(e ast.Expr) string {
return strings.TrimPrefix(path.Ext(types.ExprString(e)), ".")
}
func extractHelpMessage(cg *ast.CommentGroup) string {
if cg == nil {
return ""
}
var help []string //nolint: prealloc
for _, c := range cg.List {
mt := strings.TrimPrefix(c.Text, "//metrics:")
if mt != c.Text {
return strings.TrimSpace(mt)
}
help = append(help, strings.TrimSpace(strings.TrimPrefix(c.Text, "//")))
}
return strings.Join(help, " ")
}
func isMetric(e ast.Expr, mPkgName string) bool {
return strings.Contains(types.ExprString(e), fmt.Sprintf("%s.", mPkgName))
}
@@ -280,7 +289,11 @@ func extractLabels(bl *ast.BasicLit) string {
if bl != nil {
t := reflect.StructTag(strings.Trim(bl.Value, "`"))
if v := t.Get(labelsTag); v != "" {
return v
var res []string
for _, s := range strings.Split(v, ",") {
res = append(res, strconv.Quote(strings.TrimSpace(s)))
}
return strings.Join(res, ",")
}
}
return ""

4
scripts/metricsgen/metricsgen_test.go
View File

@@ -149,7 +149,7 @@ func TestParseMetricsStruct(t *testing.T) {
{
name: "metric labels",
metricsStruct: "type Metrics struct {\n" +
"myCounter metrics.Counter `metrics_labels:\"label1, label2\"`\n" +
"myCounter metrics.Counter `metrics_labels:\"label1,label2\"`\n" +
"}",
expected: metricsgen.TemplateData{
Package: pkgName,
@@ -158,7 +158,7 @@ func TestParseMetricsStruct(t *testing.T) {
TypeName: "Counter",
FieldName: "myCounter",
MetricName: "my_counter",
Labels: "label1, label2",
Labels: "\"label1\",\"label2\"",
},
},
},

2
scripts/metricsgen/testdata/commented/metrics.gen.go vendored
View File

@@ -18,7 +18,7 @@ func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "field",
Help: "Height of the chain.We expect multi-line comments to parse correctly.",
Help: "Height of the chain. We expect multi-line comments to parse correctly.",
}, labels).With(labelsAndValues...),
}
}

3
scripts/metricsgen/testdata/tags/metrics.gen.go vendored
View File

@@ -19,7 +19,8 @@ func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
Subsystem: MetricsSubsystem,
Name: "with_labels",
Help: "",
}, append(labels, "step,time")).With(labelsAndValues...), WithExpBuckets: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
}, append(labels, "step", "time")).With(labelsAndValues...),
WithExpBuckets: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "with_exp_buckets",

23
scripts/proto-gen.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/bin/sh
#
# Update the generated code for protocol buffers in the Tendermint repository.
# This must be run from inside a Tendermint working directory.
#
set -euo pipefail
# Work from the root of the repository.
cd "$(git rev-parse --show-toplevel)"
# Run inside Docker to install the correct versions of the required tools
# without polluting the local system.
docker run --rm -i -v "$PWD":/w --workdir=/w golang:1.18-alpine sh <<"EOF"
apk add curl git make
readonly buf_release='https://github.com/bufbuild/buf/releases/latest/download'
readonly OS="$(uname -s)" ARCH="$(uname -m)"
curl -sSL "${buf_release}/buf-${OS}-${ARCH}.tar.gz" \
| tar -xzf - -C /usr/local --strip-components=1
go install github.com/gogo/protobuf/protoc-gen-gogofaster@latest
make proto-gen
EOF

15
types/events.go
View File

@@ -131,7 +131,10 @@ type EventDataNewBlock struct {
func (EventDataNewBlock) TypeTag() string { return "tendermint/event/NewBlock" }
// ABCIEvents implements the eventlog.ABCIEventer interface.
func (e EventDataNewBlock) ABCIEvents() []abci.Event { return e.ResultFinalizeBlock.Events }
func (e EventDataNewBlock) ABCIEvents() []abci.Event {
base := []abci.Event{eventWithAttr(BlockHeightKey, fmt.Sprint(e.Block.Header.Height))}
return append(base, e.ResultFinalizeBlock.Events...)
}
type EventDataNewBlockHeader struct {
Header Header `json:"header"`
@@ -144,7 +147,10 @@ type EventDataNewBlockHeader struct {
func (EventDataNewBlockHeader) TypeTag() string { return "tendermint/event/NewBlockHeader" }
// ABCIEvents implements the eventlog.ABCIEventer interface.
func (e EventDataNewBlockHeader) ABCIEvents() []abci.Event { return e.ResultFinalizeBlock.Events }
func (e EventDataNewBlockHeader) ABCIEvents() []abci.Event {
base := []abci.Event{eventWithAttr(BlockHeightKey, fmt.Sprint(e.Header.Height))}
return append(base, e.ResultFinalizeBlock.Events...)
}
type EventDataNewEvidence struct {
Evidence Evidence `json:"evidence"`
@@ -262,18 +268,17 @@ func (EventDataEvidenceValidated) TypeTag() string { return "tendermint/event/Ev
const (
// EventTypeKey is a reserved composite key for event name.
EventTypeKey = "tm.event"
// TxHashKey is a reserved key, used to specify transaction's hash.
// see EventBus#PublishEventTx
TxHashKey = "tx.hash"
// TxHeightKey is a reserved key, used to specify transaction block's height.
// see EventBus#PublishEventTx
TxHeightKey = "tx.height"
// BlockHeightKey is a reserved key used for indexing FinalizeBlock events.
BlockHeightKey = "block.height"
// EventTypeFinalizeBlock is a reserved key used for indexing FinalizeBlock events.
EventTypeFinalizeBlock = "finalize_block"
)
var (