Commit Graph

60 Commits

Author SHA1 Message Date
Sam Kleinman
d2ca0b868d consensus: gossip catchup sleeping protocol (#8652) 2022-06-01 10:17:17 +00:00
William Banfield
0cceadf4d4 abci++: add consensus parameter logic to control vote extension require height (#8547)
This PR makes vote extensions optional within Tendermint. A new ConsensusParams field, called ABCIParams.VoteExtensionsEnableHeight, has been added to toggle whether or not extensions should be enabled or disabled depending on the current height of the consensus engine. Related to: #8453
2022-05-20 17:46:52 -04:00
William Banfield
ad73e6da2f consensus: update state from store before use in reactor (#8576)
Closes: #8575

This PR aims to fix the `LastCommitRound can only be negative for initial height 0` issue we see in the e2e tests by initializing the `state` object before starting the receive routines in the consensus reactor. This is somewhat inelegant, but it should fix the issue.
2022-05-19 19:35:30 +00:00
Thane Thomson
06b1812094 abci++: Propagate vote extensions (RFC 017) (#8433)
* Add protos for ExtendedCommit

Cherry-pick from e73f0178b72a16ee81f8e856aadf651f2c62ec6e just the
changes to the .proto files, since we have deleted the .intermediate
files.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make proto-gen

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* BlockStore holds extended commit

Cherry-pick 8d504d4b50ec6afbdffe2df7ababbef30e15053d and fix conflicts.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Reshuffle ExtendedCommit and ExtendedCommitSig

Separate the data structures and functions from their Commit-oriented
counterparts to adhere to the current coding style.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix exit condition in blocksync

* Add note to remove TxResult proto

As Sergio pointed out in 3e31aa6f583cdc71e208ed03a82f1d804ec0de49, this
proto message can probably be removed. We should do this in a separate
PR.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Lift termination condition into for loop

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Enforce vote extension signature requirement

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand on comment for PeekTwoBlocks for posterity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Isolate TODO more clearly

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make mockery

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix comment

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Make panic output from BlockStore.SaveBlock more readable

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add helper methods to ExtendedCommitSig and ExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix most tests except TestHandshake*

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix store prefix collision

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix TestBlockFetchAtHeight

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove global state from store tests

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Co-authored-by: Sergio Mena <sergio@informal.systems>

* blocksync: Just return error

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove unused/commented-out code

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Change pool AddBlock function signature to return errors

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Improve legibility of switch statements

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Expand on extended commit requirement in AddBlock description

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Return error without also logging it

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* consensus: Rename short-lived local variable

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* consensus: Allocate TODO to Sergio

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* evidence/pool_test: Inline slice construction

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* state: Rename LoadBlockExtCommit to LoadBlockExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* proto: Remove TODO on TxResult

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Minor format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Reformat ExtendedCommitSig.BlockID

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove NewExtendedCommit constructor

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove NewCommit constructor

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Shorten receiver names for ExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Convert ExtendedCommit.Copy to a deep clone

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Assign TODO to Sergio

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Fix legibility nits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Improve legibility

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* store/state: Add TODO to move prefixes to common package

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Propagate validator info to PrepareProposal

In order to propagate validator voting power through to PrepareProposal,
we need to load the validator set info from the height corresponding to
the extended commit that we're passing through to PrepareProposal as the
"LocalLastCommit".

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename local var for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix TestMaxProposalBlockSize

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename local var for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove debug log

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove CommigSig.ForBlock helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove CommigSig.Absent helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove ExtendedCommitSig.ForBlock helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove ExtendedCommitSig.Absent helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* There are no extended commits below the initial height

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix comment grammar

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove JSON encoding from ExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Embed CommitSig into ExtendedCommitSig instead of duplicating fields

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename ExtendedCommit vote_extension field to extension for consistency with domain types

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Panic if we peek a block without an extended commit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Remove Sergio from TODO

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Increase hard-coded vote extension max size to 1MB

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* state: Remove unnecessary comment

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* state: Ensure no of commit sigs equals validator set length

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Minor legibility improvements

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Improve legibility

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove unused GetVotes function on VoteSet

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor TestMaxProposalBlockSize to construct more realistic extended commit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor buildExtendedCommitInfo to resemble buildLastCommitInfo

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* abci++: Disable VerifyVoteExtension call on nil precommits (#8491)

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Require vote extensions on non-nil precommits and not otherwise

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Disable lint

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Increase timeout for TestReactorVotingPowerChange to counter flakiness

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Only sign and verify vote extensions in non-nil precommits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Revert "Disable lint"

This reverts commit 6fffbf9402.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add missing non-nil check uncovered non-deterministically in TestHandshakeReplayAll

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand error message for accuracy

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Only call ExtendVote when we make non-nil precommits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Revert "Increase timeout for TestReactorVotingPowerChange to counter flakiness"

This reverts commit af514939db.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor ValidateBasic for ExtendedCommitSig for legibility

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Co-authored-by: Sergio Mena <sergio@informal.systems>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-05-11 07:10:32 -04:00
Sam Kleinman
cf2a00b398 p2p: avoid using p2p.Channel internals (#8444) 2022-04-29 17:21:36 -04:00
Sam Kleinman
c372390fea eventbus: publish without contexts (#8369) 2022-04-18 16:28:31 -04:00
William Banfield
6ff07a1633 switch to consensus change startup ordering (#8290) 2022-04-08 18:11:20 -04:00
Sam Kleinman
631ef7aae0 consensus: add nil check to gossip routine (#8288) 2022-04-08 17:13:10 -04:00
Sam Kleinman
9d20e06900 statesync+blocksync: move event publications into the sync operations (#8274) 2022-04-07 16:23:36 -04:00
Sam Kleinman
2304ea70f7 consensus: remove string indented function (#8257) 2022-04-06 08:07:21 -07:00
Sam Kleinman
9d1e8eaad4 node: remove channel and peer update initialization from construction (#8238) 2022-04-05 13:26:53 +00:00
Sam Kleinman
97f7021712 statesync: merge channel processing (#8240) 2022-04-04 12:31:15 -04:00
William Banfield
485c96b0d3 consensus: change lock handling in reactor and handleMsg for RoundState (forward-port #7994 #7992) (#8139)
Related to #8157
2022-03-18 18:59:44 +00:00
Sam Kleinman
1db41663c7 consensus: avoid race in accessing channel (#8149) 2022-03-18 09:54:40 -04:00
Sam Kleinman
7a0b05f22d libs/events: remove unneccessary unsubscription code (#8135)
The events switch code is largely vestigal and is responsible for
wiring between the consensus state machine and the consensus
reactor. While there might have been a need, historicallly to managed
these subscriptions at runtime, it's nolonger used: subscriptions are
registered during startup, and then the switch shuts down at at the
end. 

Eventually the EventSwitch should be replaced by a much smaller
implementation of an eventloop in the consensus state machine, but
cutting down on the scope of the event switch will help clarify the
requirements from the consensus side.
2022-03-17 13:02:02 +00:00
Sam Kleinman
c35d6d6e2c node: pass eventbus at construction time (#8084)
* node: pass eventbus at construction time

* remove cruft
2022-03-08 13:04:59 +00:00
JayT106
ce898a738c consensus: HasVoteMessage index boundary check (#7720)
Went through #2871, there are several issues, this PR tries to tackle the `HasVoteMessage` with an invalid validator index sent by a bad peer and it prevents the bad vote goes to the peerMsgQueue.

Future work, check other bad message cases and plumbing the reactor errors with the peer manager and then can disconnect the peer sending the bad messages.
2022-02-21 13:21:24 +00:00
Sam Kleinman
28d34d635c service: change stop interface (#7816) 2022-02-17 11:23:32 -05:00
Sam Kleinman
eed617c2d9 consensus: refactor operations in consensus queryMaj23Routine (#7791) 2022-02-09 14:47:50 -05:00
Sam Kleinman
c555226d2b consensus: tie peer threads to peer lifecylce context (#7792) 2022-02-09 12:01:12 -05:00
Sam Kleinman
cb98d515dd consensus: delay start of peer routines (#7753) 2022-02-02 14:20:56 -05:00
Sam Kleinman
1d3ecf37ee consensus: remove unused closer construct (#7734)
This is clearly a cob-web in the code, and may predict a solution to #7729, though this is difficult to backport because we don't have contexts in 0.35
2022-01-31 16:03:20 +00:00
William Banfield
0aa3b0b6fc Proposer-Based Timestamps Merge (#7605)
This pull request merges in the changes for implementing Proposer-based timestamps into `master`. The power was primarily being done in the `wb/proposer-based-timestamps` branch, with changes being merged into that branch during development. This pull request represents an amalgamation of the changes made into that development branch. All of the changes that were placed into that branch have been cleanly rebased on top of the latest `master`. The changes compile and the tests pass insofar as our tests in general pass.

### Note To Reviewers
 These changes have been extensively reviewed during development. There is not much new here. In the interest of making effective use of time, I would recommend against trying to perform a complete audit of the changes presented and instead examine for mistakes that may have occurred during the process of rebasing the changes. I gave the complete change set a first pass for any issues, but additional eyes would be very appreciated. 

In sum, this change set does the following:
closes #6942 
merges in #6849
2022-01-26 16:00:23 +00:00
Sam Kleinman
5bf1bdcfb4 reactors: skip log on some routine cancels (#7556) 2022-01-11 12:56:52 -05:00
Sam Kleinman
10402b728f consensus+p2p: change how consensus reactor is constructed (#7525) 2022-01-07 10:59:10 -05:00
Sam Kleinman
74a8941854 consensus: remove reactor options (#7526) 2022-01-07 09:15:35 -05:00
Sam Kleinman
d0e03f01fc sync: remove special mutexes (#7438) 2021-12-13 13:35:32 -05:00
Sam Kleinman
65c0aaee5e p2p: use recieve for channel iteration (#7425) 2021-12-13 11:04:44 -05:00
Sam Kleinman
bd6dc3ca88 p2p: refactor channel Send/out (#7414) 2021-12-09 14:03:41 -05:00
Sam Kleinman
cb88bd3941 p2p: migrate to use new interface for channel errors (#7403)
* p2p: migrate to use new interface for channel errors

* Update internal/p2p/p2ptest/require.go

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>

* rename

* feedback

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2021-12-08 14:05:01 +00:00
Sam Kleinman
892f5d9524 service: cleanup mempool and peer update shutdown (#7401) 2021-12-08 08:44:32 -05:00
Sam Kleinman
0ff3d4b89d service: cleanup close channel in reactors (#7399) 2021-12-07 11:40:59 -05:00
Sam Kleinman
6b35cc1a47 p2p: remove unneeded close channels from p2p layer (#7392) 2021-12-07 10:40:07 -05:00
Sam Kleinman
a62ac27047 service: remove exported logger from base implemenation (#7381) 2021-12-06 10:16:42 -05:00
Sam Kleinman
8a991e288c service: plumb contexts to all (most) threads (#7363)
This continues the push of plumbing contexts through tendermint. I
attempted to find all goroutines in the production code (non-test) and
made sure that these threads would exit when their contexts were
canceled, and I believe this PR does that.
2021-12-02 21:38:38 +00:00
Sam Kleinman
a823d167bc service: cleanup base implementation and some caller implementations (#7301) 2021-12-01 09:28:06 -05:00
Sam Kleinman
4af2dbd03b eventbus: plumb contexts (#7337)
* eventbus: plumb contexts

* fix lint
2021-11-30 14:24:11 +00:00
Sam Kleinman
6ab62fe7b6 service: remove stop method and use contexts (#7292) 2021-11-18 17:56:21 -05:00
M. J. Fromberger
54d7030510 pubsub: Move indexing out of the primary subscription path (#7231)
This is part of the work described by #7156.

Remove "unbuffered subscriptions" from the pubsub service.
Replace them with a dedicated blocking "observer" mechanism.
Use the observer mechanism for indexing.

Add a SubscribeWithArgs method and deprecate the old Subscribe
method. Remove SubscribeUnbuffered entirely (breaking).

Rework the Subscription interface to eliminate exposed channels.
Subscriptions now use a context to manage lifecycle notifications.

Internalize the eventbus package.
2021-11-05 10:25:25 -07:00
Sam Kleinman
ca8f004112 p2p: remove final shims from p2p package (#7136)
This is, perhaps, the trival final piece of #7075 that I've been
working on.

There's more work to be done: 
- push more of the setup into the pacakges themselves
- move channel-based sending/filtering out of the 
- simplify the buffering throuhgout the p2p stack.
2021-10-15 20:08:09 +00:00
Sam Kleinman
cbe6ad6cd5 p2p: flatten channel descriptor (#7132) 2021-10-15 13:03:10 -04:00
Sam Kleinman
0900ea8396 p2p: channel shim cleanup (#7129) 2021-10-15 12:31:33 -04:00
Sam Kleinman
f4a56f4034 p2p: refactor channel description (#7130)
This is another small sliver of #7075, with the intention of removing
the legacy shim layer related to channel registration.
2021-10-15 15:45:12 +00:00
William Banfield
f5b9c210ca consensus: wait until peerUpdates channel is closed to close remaining peers (#7058)
The race occurred as a result of a goroutine launched by `processPeerUpdate` racing with the `OnStop` method. The `processPeerUpdates` goroutine deletes from the map as `OnStop` is reading from it. This change updates the `OnStop` method to wait for the peer updates channel to be done before closing the peers. It also copies the map contents to a new map so that it will not conflict with the view of the map that the goroutine created in `processPeerUpdate` sees.
2021-10-04 22:37:18 +00:00
Sam Kleinman
1c4950dbd2 state: move package to internal (#6964) 2021-09-22 13:04:25 -04:00
Callum Waters
bda948e814 statesync: implement p2p state provider (#6807) 2021-09-02 13:19:18 +02:00
William Banfield
4e96c6b234 tools: add mockery to tools.go and remove mockery version strings (#6787)
This change aims to keep versions of mockery consistent across developer laptops.

This change adds mockery to the `tools.go` file so that its version can be managed consistently in the `go.mod` file.

Additionally, this change temporarily disables adding mockery's version number to generated files. There is an outstanding issue against the mockery project related to the version string behavior when running from `go get`. I have created a pull request to fix this issue in the mockery project.
see: https://github.com/vektra/mockery/issues/397
2021-07-30 20:47:15 +00:00
Callum Waters
6ff4c3139c blockchain: rename to blocksync service (#6755) 2021-07-28 17:25:42 +02:00
JayT106
e70445f942 statesync/event: emit statesync start/end event (#6700) 2021-07-22 08:16:50 +02:00
JayT106
c4f77ab6d1 fastsync/event: emit fastsync status event when switching consensus/fastsync (#6619)
closes #2498 
solves part of #3365

Note: difficult to test the event emit in SwitchToFastSync part, might need to change `stateSyncReactor` to an interface in the `nodeImpl` struct
2021-07-20 15:36:47 +00:00