This pull requests adds the protocol buffer field for the `ABCI.VoteExtensionsEnableHeight` parameter. This proto field is threaded throughout all of the relevant places where consensus params are used and referenced.
This PR also adds validation of the consensus param updates. Previous consensus param changes didn't depend on _previous_ versions of the params, so this change adds a method for validating against the old params as well.
closes: #8453
This PR makes vote extensions optional within Tendermint. A new ConsensusParams field, called ABCIParams.VoteExtensionsEnableHeight, has been added to toggle whether or not extensions should be enabled or disabled depending on the current height of the consensus engine. Related to: #8453
* Split vote verification/validation based on vote extensions
Some parts of the code need vote extensions to be verified and
validated (mostly in consensus), and other parts of the code don't
because its possible that, in some cases (as per RFC 017), we won't have
vote extensions.
This explicitly facilitates that split.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Only sign extensions in precommits, not prevotes
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update privval/file.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Apply suggestions from code review
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Temporarily disable extension requirement again for E2E testing
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Reorganize comment for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Leave vote validation and pre-call nil check up to caller of VoteToProto
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Split complex vote validation test into multiple tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Universally enforce no vote extensions on any vote type but precommits
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Make error messages more generic
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Verify with vote extensions when constructing a VoteSet
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Expand comment for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add extension check for prevotes prior to signing votes
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix supporting test code to only inject extensions into precommits
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Separate vote malleation from signing in vote tests for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add extension signature length check and corresponding test
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Perform basic vote validation in CommitToVoteSet
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Refactor so building and linting works
This is the first step towards implementing vote extensions: generating
the relevant proto stubs and getting the build and linter to pass.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix typo
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Better describe method given vote extensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix types tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Move CanonicalVoteExtension to canonical types proto defs
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Regenerate protos including latest PBTS synchrony params update
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Inject vote extensions into proposal
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Thread vote extensions through code and fix tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove extraneous empty value initialization
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix lint
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix missing VerifyVoteExtension request data
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Explicitly ensure length > 0 to sign vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Explicitly ensure length > 0 to sign vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove extraneous comment
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update privval/file.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update types/vote_test.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Format
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix ABCI proto generation scripts for Linux
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync intermediate and goal protos
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update internal/consensus/common_test.go
Co-authored-by: Sergio Mena <sergio@informal.systems>
* Use dummy value with clearer meaning
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Rewrite loop for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Panic on ABCI++ method call failure
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add strong correctness guarantees when constructing extended commit info for ABCI++
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add strong guarantee in extendedCommitInfo that the number of votes corresponds
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Make extendedCommitInfo function more robust
At first extendedCommitInfo expected votes to be in the same order as
their corresponding validators in the supplied CommitInfo struct, but
this proved to be rather difficult since when a validator set's loaded
from state it's first sorted by voting power and then by address.
Instead of sorting the votes in the same way, this approach simply maps
votes to their corresponding validator's address prior to constructing
the extended commit info. This way it's easy to look up the
corresponding vote and we don't need to care about vote order.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove extraneous validator address assignment
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sign over canonical vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Validate vote extension signature against canonical vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update privval tests for more meaningful dummy value
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add vote extension capability to E2E test app
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Disable lint for weak RNG usage for test app
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Use parseVoteExtension instead of custom parsing in PrepareProposal
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Only include extension if we have received txs
It's unclear at this point why this is necessary to ensure that the
application's local app_hash matches that committed in the previous
block.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Require app_hash from app to match that from last block
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add contrived (possibly flaky) test to check that vote extensions code works
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove workaround for problem now solved by #8229
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* add tests for vote extension cases
* Fix spelling mistake to appease linter
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Collapse redundant if statement
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Formatting
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Always expect an extension signature, regardless of whether an extension is present
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Votes constructed from commits cannot include extensions or signatures
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Pass through vote extension in test helpers
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Temporarily disable vote extension signature requirement
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Expand on vote equality test errors for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Expand on vote matching error messages in testing
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Allow for selective subscription by vote type
This is an attempt to fix the intermittently failing
`TestPrepareProposalReceivesVoteExtensions` test in the internal
consensus package.
Occasionally we get prevote messages via the subscription channel, and
we're not interested in those. This change allows us to specify what
types of votes we're interested in (i.e. precommits) and discard the
rest.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Read lock consensus state mutex in test helper to avoid data race
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Revert BlockIDFlag parameter in node test
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Perform additional check in ProcessProposal for special txs generated by vote extensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: check that our added tx does not cause all txs to exceed req.MaxTxBytes
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Only set vote extension signatures when signing is successful
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove channel capacity constraint in test helper to avoid missing messages
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add TODO to always require extension signatures in vote validation
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: reject vote extensions if the request height does not match what we expect
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* types: remove extraneous call to voteWithoutExtension in test
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove unnecessary address parameter from CanonicalVoteExtension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval: change test vote type to precommit since we use an extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval: update signing logic to cater for vote extensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* proto: update field descriptions for vote message
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* proto: update field description for vote extension sig in vote message
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* proto/types: use fixed-length 64-bit integers for rounds in CanonicalVoteExtension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* consensus: fix flaky TestPrepareProposalReceivesVoteExtensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* consensus: remove previously added test helper functionality
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: add error logs when we get an unexpected height in ExtendVote or VerifyVoteExtension requests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* node_test: get validator addresses from privvals
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval/file_test: optimize filepv creation in tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval: add test to check that vote extensions are always signed
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add a script to check documentation for ToC entries. (#8356)
This script verifies that each document in the docs and architecture directory
has a corresponding table-of-contents entry in its README file. It can be run
manually from the command line.
- Hook up this script to run in CI (optional workflow).
- Update ADR ToC to include missing entries this script found.
* build(deps): Bump async from 2.6.3 to 2.6.4 in /docs (#8357)
Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4.
- [Release notes](https://github.com/caolan/async/releases)
- [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md)
- [Commits](https://github.com/caolan/async/compare/v2.6.3...v2.6.4)
---
updated-dependencies:
- dependency-name: async
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* privval/file_test: reset vote ext sig before signing
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Co-authored-by: Sergio Mena <sergio@informal.systems>
Co-authored-by: William Banfield <wbanfield@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Changed the spec text to agreed VoteExtension solution
* Revert "Removed protobufs related to vote extensions"
This reverts commit 4566f1e302.
* Changes to ABCI protocol buffers
* Update spec/core/data_structures.md
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update spec/core/data_structures.md
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Fix dangling link in ABCI++ readme
* Addressed comments
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Our documentation build process is rather finicky. I noticed yet another build failure that doesn't actually cause the docs site build to fail - see https://github.com/tendermint/tendermint/actions/runs/3578638807/jobs/6019002654
Basically the `docs/post.sh` script deletes the `.vuepress/public/rpc` directory after generating the docs for a specific version of Tendermint, and then the `docs/pre.sh` script attempts to copy the next version's OpenAPI script into `.vuepress/public/rpc` but that directory doesn't exist.
For some reason this doesn't cause the npm build process to fail - it just results in the OpenAPI doc not being available.
This is hard to test locally because our docs build process currently relies on checking out specific branches to do the full docs site build: 4290ad2982/Makefile (L290-L300)
---
#### PR checklist
- [x] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
List of typos in comments and functions.
* L26: `conflictinge` seems to be `conflicting`
* L77: `melevolent` seems to be `malevolent`
* L218, L226: `challendingBlock` seems to be `challengingBlock`
* L375: `heght` seems to be `height`
* L395: `avoud` seems to be `avoid`
---
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
Follows from #9734, porting to `main`.
---
#### PR checklist
- [x] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
This change reduces the number of Precommit messages sent to peers by 50%.
During the `ApplyNewRoundStepMessage`, we update the known state of the peer sending us the message.
We set the value of `ps.PRS.Precommits` to nil in this method if the peer is entering a new height or round.
34ca3fb474/consensus/reactor.go (L1368)
We then assign `ps.PRS.LastCommit = ps.PRS.Precommits` if the peer is entering a new height only - this does not happen during just a round change. This therefore results in `ps.PRS.LastCommit` having the value `nil`.
When the `LastCommit` bit field is seen as `nil` in the reactor, an empty bit field is initialized.
34ca3fb474/consensus/reactor.go (L1273)
The code responsible for gossiping votes from the previous height uses this `LastCommit` value and, seeing an empty `LastCommit` will resend every `Precommit` from the previous height since it lost the information it previously had detailing which precommits from the previous height the peer had.
This can be seen in the code responsible for gossiping precommits from the previous height:
34ca3fb474/consensus/reactor.go (L773)
Where this code grabs the, previously `nil`, `LastCommit` bit field:
34ca3fb474/consensus/reactor.go (L1204-L1212)
---
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
Fixes the tendermint OSS-Fuzz build by adjusting for some infra changes that were made in OSS-Fuzz.
---
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
## NOTE: this pr exclusively runs commands from the makefile found here
This PR ONLY runs `make format` ... then `make mockery`
Its purpose is to ensure that the review scope of other PR's, which changed .go files and thus triggered the linter that only runs conditionally, have smaller review
scopes, and should be merged before:
https://github.com/tendermint/tendermint/pull/9738https://github.com/tendermint/tendermint/pull/9739https://github.com/tendermint/tendermint/pull/9742
---
#### PR checklist
- [x] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
Since starting off as a wee validator, I've been mystified by the volume of p2p logspam, which often makes it impossible to monitor other tasks. Thus, routine p2p events, have been cast into the land of debug.
---
#### PR checklist
- [x] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
