This pull request adds an additional set of metrics targeted at providing more visibility into `abci++`.
The following set of metrics are added and exposed through the `metrics` endpoint:
```
tendermint_consensus_proposal_receive_count{chain_id="test-chain-IrF74Y",status="accepted"} 34
tendermint_consensus_proposal_create_count{chain_id="test-chain-IrF74Y"} 34
tendermint_consensus_vote_extension_receive_count{chain_id="test-chain-IrF74Y",status="accepted"} 34
tendermint_consensus_round_voting_power_percent{chain_id="test-chain-IrF74Y",vote_type="precommit"} 1
tendermint_consensus_round_voting_power_percent{chain_id="test-chain-IrF74Y",vote_type="prevote"} 1
tendermint_state_consensus_param_updates{chain_id="test-chain-IrF74Y"} 0
tendermint_state_validator_set_updates{chain_id="test-chain-IrF74Y"} 0
tendermint_consensus_late_votes{chain_id="test-chain-IrF74Y",vote_type="precommit"} 16
```
This pull request also updates the `metrics.md` file to include some metrics that were previously missed. My hope is to generate the `metrics.md` file with a future version of the tool being architected in #8479
This pull requests adds a new tool, metricsgen, for generating Tendermint metrics constructors from `Metrics` struct definitions. This tool aims to reduce the amount of boilerplate required to add additional metrics to Tendermint.
Its working is fairly simple, it parses the go ast, extracts field information, and uses this field information to execute a go template.
This pull request also adds a proof-of-concept of the tool's output and working by using it to generate the [indexer metrics](https://github.com/tendermint/tendermint/pull/8479/files#diff-4b0c597b6fa05332a2f9a8e0ce079e360602942fae99dc5485f1edfe71c0a29e) using `//go:generate` directives and a simple `make` target.
The next steps for this tool are documented in https://github.com/tendermint/tendermint/issues/8485 and https://github.com/tendermint/tendermint/issues/8486, which detail using the tool to generate the `metrics.md` documentation file and using the tool to migrate away from `go-kit`.
Per https://github.com/tendermint/docs/issues/20, it is no longer necessary to
build the static documentation out of a separate repository.
This change:
- Adds an actions workflow to build and deploy the docs to GitHub Pages.
- Updates some build settings in a compatible manner.
This change does not affect the existing site deployment. To complete this
change, we will need to update the custom domain pointer and disable the
corresponding workflow in the tendermint/docs repository. Those changes can and
must be done after this is merged.
In the future should probably also move the build rule out of the Makefile and
into the workflow directly. That will also make it easier to manage caching of
build artifacts. For now, however, I've left it as-is, so that we do not break
the active workflow on tendermint/docs, which depends on it.
Bumps [github.com/vektra/mockery/v2](https://github.com/vektra/mockery) from 2.12.1 to 2.12.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/vektra/mockery/releases">github.com/vektra/mockery/v2's releases</a>.</em></p>
<blockquote>
<h2>v2.12.2</h2>
<h2>Changelog</h2>
<ul>
<li>ea4c438 Add deprecation notice to logs</li>
<li>735bc0c Add go-get deprecation note</li>
<li>bea853e Add missing mock</li>
<li>989253d Fix *unsafe.Pointer</li>
<li>9228ad4 Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/457">#457</a> from LandonTClipp/readme_deprecation</li>
<li>1d92e73 Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/460">#460</a> from grongor/fix-unsafe-pointer</li>
<li>2fcd83d Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/462">#462</a> from LandonTClipp/deprecation</li>
<li>9f67b8a More explicit deprecation for go-get</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="1d92e7320b"><code>1d92e73</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/460">#460</a> from grongor/fix-unsafe-pointer</li>
<li><a href="2fcd83d746"><code>2fcd83d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/462">#462</a> from LandonTClipp/deprecation</li>
<li><a href="ea4c438a53"><code>ea4c438</code></a> Add deprecation notice to logs</li>
<li><a href="989253d1a4"><code>989253d</code></a> Fix *unsafe.Pointer</li>
<li><a href="bea853e93d"><code>bea853e</code></a> Add missing mock</li>
<li><a href="9f67b8afdc"><code>9f67b8a</code></a> More explicit deprecation for go-get</li>
<li><a href="9228ad4b4a"><code>9228ad4</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/457">#457</a> from LandonTClipp/readme_deprecation</li>
<li><a href="735bc0c9f8"><code>735bc0c</code></a> Add go-get deprecation note</li>
<li>See full diff in <a href="https://github.com/vektra/mockery/compare/v2.12.1...v2.12.2">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
This is a follow-up to #8352. The check for legacy evidence keys is only based
on the prefix of the key. Hashes, which are unprefixed, could easily have this
form and be misdiagnosed.
Because the conversion for evidence checks the key structure, this should not
cause corruption. The probability that a hash is a syntactically valid evidence
key is negligible. The tool will report an error rather than storing bad data.
But this does mean that such transaction hashes could cause the migration to
stop and report an error before it is complete.
To ensure we convert all the data, refine the legacy key check to filter these
keys more precisely. Update the test cases to exercise this condition.
* Update upgrading instructions.
Bumps [github.com/creachadair/tomledit](https://github.com/creachadair/tomledit) from 0.0.18 to 0.0.19.
<details>
<summary>Commits</summary>
<ul>
<li><a href="0692e4157a"><code>0692e41</code></a> Release v0.0.19</li>
<li><a href="d1160a474b"><code>d1160a4</code></a> Update default permissions.</li>
<li><a href="56f28f4ea0"><code>56f28f4</code></a> Move transform tests to that package.</li>
<li><a href="3b8b380274"><code>3b8b380</code></a> Add permissions to CI workflow.</li>
<li><a href="409951b699"><code>409951b</code></a> Add a quotation test case.</li>
<li><a href="f35c8bec5c"><code>f35c8be</code></a> parser: include line numbers in headings, mappings, and values</li>
<li><a href="26acca1df8"><code>26acca1</code></a> Regularize location formatting in diagnostics.</li>
<li><a href="3394f599e4"><code>3394f59</code></a> Add more parser test cases.</li>
<li><a href="5ce10cc05a"><code>5ce10cc</code></a> Rename test file.</li>
<li><a href="29f3eb34c8"><code>29f3eb3</code></a> Allow compliance tests to be skipped with -short.</li>
<li>See full diff in <a href="https://github.com/creachadair/tomledit/compare/v0.0.18...v0.0.19">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
In #8397 I tried to remove all the cases where we needed to keep track of the
target type of parameters for JSON encoding, but there is one case still left:
When decoding parameters from URL query terms, there is no way to tell whether
or not we need base64 encoding without knowing whether the underlying type of
the target is string or []byte.
To fix this, keep track of parameters that are []byte valued when RPCFunc is
compiling its argument map, and use that when parsing URL query terms. Update
the tests accordingly.
* Split vote verification/validation based on vote extensions
Some parts of the code need vote extensions to be verified and
validated (mostly in consensus), and other parts of the code don't
because its possible that, in some cases (as per RFC 017), we won't have
vote extensions.
This explicitly facilitates that split.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Only sign extensions in precommits, not prevotes
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update privval/file.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Apply suggestions from code review
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Temporarily disable extension requirement again for E2E testing
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Reorganize comment for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Leave vote validation and pre-call nil check up to caller of VoteToProto
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Split complex vote validation test into multiple tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Universally enforce no vote extensions on any vote type but precommits
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Make error messages more generic
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Verify with vote extensions when constructing a VoteSet
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Expand comment for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add extension check for prevotes prior to signing votes
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix supporting test code to only inject extensions into precommits
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Separate vote malleation from signing in vote tests for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add extension signature length check and corresponding test
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Perform basic vote validation in CommitToVoteSet
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
In #8339 we pointed the markdown link checker action to a patched version that
has the up-to-date version of the underlying check tool. In doing so, I missed
the periodic cron job that runs the same workflow. Update it to use the patched
version also.
Pass all parameters from JSON-RPC requests to their corresponding handlers
using struct types instead of positional parameters. This allows us to control
encoding of arguments using only the standard library, and to eliminate the
remaining special-purpose JSON encoding hooks in the server.
To support existing use, the server still allows arguments to be encoded in
JSON as either an array or an object.
Related changes:
- Rework the RPCFunc constructor to reduce reflection during RPC call service.
- Add request parameter wrappers for each RPC service method.
- Update the RPC Environment methods to use these types.
- Update the interfaces and shims derived from Environment to the new
signatures.
- Update and extend test cases.
In the conversion to Go 1.18 fuzzing in e4991fd862,
a `return 0` was converted to a panic. A `return 0` is a hint to the fuzzer, not
a failing testcase.
While here, clean up the test by folding setup code into it.
Bumps [github.com/vektra/mockery/v2](https://github.com/vektra/mockery) from 2.12.0 to 2.12.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/vektra/mockery/releases">github.com/vektra/mockery/v2's releases</a>.</em></p>
<blockquote>
<h2>v2.12.1</h2>
<h2>Changelog</h2>
<ul>
<li>facf60b Add extra test cases for increasing code coverage.</li>
<li>2e1360a Collapse if statements and rename interface in the fixtures.</li>
<li>8bdc90d Fix test on go1.18.</li>
<li>fe03b57 Fix tests.</li>
<li>b8c62f7 Fix: avoid package name collision with inPackage (<a href="https://github-redirect.dependabot.com/vektra/mockery/issues/291">#291</a>)</li>
<li>c9dc740 Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/422">#422</a> from i-sevostyanov/fix-package-collision</li>
<li>58a7f18 Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/452">#452</a> from grongor/refactor-first-letter-helper</li>
<li>749b2d6 Refactor mock name generation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="c9dc740b50"><code>c9dc740</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/422">#422</a> from i-sevostyanov/fix-package-collision</li>
<li><a href="facf60b02e"><code>facf60b</code></a> Add extra test cases for increasing code coverage.</li>
<li><a href="8bdc90da7a"><code>8bdc90d</code></a> Fix test on go1.18.</li>
<li><a href="fe03b57da5"><code>fe03b57</code></a> Fix tests.</li>
<li><a href="2e1360ae46"><code>2e1360a</code></a> Collapse if statements and rename interface in the fixtures.</li>
<li><a href="b8c62f7858"><code>b8c62f7</code></a> Fix: avoid package name collision with inPackage (<a href="https://github-redirect.dependabot.com/vektra/mockery/issues/291">#291</a>)</li>
<li><a href="58a7f185bd"><code>58a7f18</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/452">#452</a> from grongor/refactor-first-letter-helper</li>
<li><a href="749b2d6fa5"><code>749b2d6</code></a> Refactor mock name generation</li>
<li>See full diff in <a href="https://github.com/vektra/mockery/compare/v2.12.0...v2.12.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.3.1 to 1.4.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/bufbuild/buf-setup-action/releases">bufbuild/buf-setup-action's releases</a>.</em></p>
<blockquote>
<h2>v1.4.0</h2>
<ul>
<li>Set the default buf version to v1.4.0</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="b0ab760002"><code>b0ab760</code></a> Update to v1.4.0</li>
<li><a href="60debd1093"><code>60debd1</code></a> Revert "Add dependabot config (<a href="https://github-redirect.dependabot.com/bufbuild/buf-setup-action/issues/21">#21</a>)" (<a href="https://github-redirect.dependabot.com/bufbuild/buf-setup-action/issues/33">#33</a>)</li>
<li><a href="e789cb128b"><code>e789cb1</code></a> Add dependabot config (<a href="https://github-redirect.dependabot.com/bufbuild/buf-setup-action/issues/21">#21</a>)</li>
<li>See full diff in <a href="https://github.com/bufbuild/buf-setup-action/compare/v1.3.1...v1.4.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
The p2p/conn library was using a build patch to work around an old issue with
the net.Conn type that has not existed since Go 1.10. Remove the workaround and
update usage to use the standard net.Pipe directly.
The HexBytes wrapper type handles decoding byte strings from JSON. In the RPC
API, hashes are encoded as hex digits rather than the standard base64.
Simplify the implementation of this wrapper using the TextMarshaler interface,
which the encoding/json package uses for values (like these) that are meant to
be wrapped in JSON strings.
In addition, allow HexBytes values to be decoded from either hex OR base64
input. This preserves all existing use, but will allow us to remove some
reflection special cases in the RPC decoder plumbing.
Update tests to correctly tolerate empty/nil.
