107 Commits

Author SHA1 Message Date
Thane Thomson
06b1812094 abci++: Propagate vote extensions (RFC 017) (#8433)
* Add protos for ExtendedCommit

Cherry-pick from e73f0178b72a16ee81f8e856aadf651f2c62ec6e just the
changes to the .proto files, since we have deleted the .intermediate
files.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make proto-gen

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* BlockStore holds extended commit

Cherry-pick 8d504d4b50ec6afbdffe2df7ababbef30e15053d and fix conflicts.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Reshuffle ExtendedCommit and ExtendedCommitSig

Separate the data structures and functions from their Commit-oriented
counterparts to adhere to the current coding style.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix exit condition in blocksync

* Add note to remove TxResult proto

As Sergio pointed out in 3e31aa6f583cdc71e208ed03a82f1d804ec0de49, this
proto message can probably be removed. We should do this in a separate
PR.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Lift termination condition into for loop

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Enforce vote extension signature requirement

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand on comment for PeekTwoBlocks for posterity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Isolate TODO more clearly

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make mockery

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix comment

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Make panic output from BlockStore.SaveBlock more readable

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add helper methods to ExtendedCommitSig and ExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix most tests except TestHandshake*

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix store prefix collision

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix TestBlockFetchAtHeight

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove global state from store tests

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Co-authored-by: Sergio Mena <sergio@informal.systems>

* blocksync: Just return error

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove unused/commented-out code

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Change pool AddBlock function signature to return errors

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Improve legibility of switch statements

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Expand on extended commit requirement in AddBlock description

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Return error without also logging it

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* consensus: Rename short-lived local variable

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* consensus: Allocate TODO to Sergio

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* evidence/pool_test: Inline slice construction

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* state: Rename LoadBlockExtCommit to LoadBlockExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* proto: Remove TODO on TxResult

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Minor format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Reformat ExtendedCommitSig.BlockID

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove NewExtendedCommit constructor

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove NewCommit constructor

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Shorten receiver names for ExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Convert ExtendedCommit.Copy to a deep clone

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Assign TODO to Sergio

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Fix legibility nits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Improve legibility

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* store/state: Add TODO to move prefixes to common package

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Propagate validator info to PrepareProposal

In order to propagate validator voting power through to PrepareProposal,
we need to load the validator set info from the height corresponding to
the extended commit that we're passing through to PrepareProposal as the
"LocalLastCommit".

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename local var for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix TestMaxProposalBlockSize

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename local var for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove debug log

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove CommigSig.ForBlock helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove CommigSig.Absent helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove ExtendedCommitSig.ForBlock helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove ExtendedCommitSig.Absent helper

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* There are no extended commits below the initial height

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix comment grammar

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove JSON encoding from ExtendedCommit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Embed CommitSig into ExtendedCommitSig instead of duplicating fields

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename ExtendedCommit vote_extension field to extension for consistency with domain types

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* blocksync: Panic if we peek a block without an extended commit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Remove Sergio from TODO

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Increase hard-coded vote extension max size to 1MB

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* state: Remove unnecessary comment

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* state: Ensure no of commit sigs equals validator set length

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Minor legibility improvements

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Improve legibility

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Remove unused GetVotes function on VoteSet

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor TestMaxProposalBlockSize to construct more realistic extended commit

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor buildExtendedCommitInfo to resemble buildLastCommitInfo

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* abci++: Disable VerifyVoteExtension call on nil precommits (#8491)

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* types: Require vote extensions on non-nil precommits and not otherwise

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Disable lint

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Increase timeout for TestReactorVotingPowerChange to counter flakiness

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Only sign and verify vote extensions in non-nil precommits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Revert "Disable lint"

This reverts commit 6fffbf9402.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add missing non-nil check uncovered non-deterministically in TestHandshakeReplayAll

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand error message for accuracy

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Only call ExtendVote when we make non-nil precommits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Revert "Increase timeout for TestReactorVotingPowerChange to counter flakiness"

This reverts commit af514939db.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor ValidateBasic for ExtendedCommitSig for legibility

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Co-authored-by: Sergio Mena <sergio@informal.systems>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-05-11 07:10:32 -04:00
Sam Kleinman
3d448e1ef8 crypto: cleanup tmhash package (#8434) 2022-04-28 22:21:43 -04:00
Thane Thomson
d69bd64702 abci++: Vote extension cleanup (#8402)
* Split vote verification/validation based on vote extensions

Some parts of the code need vote extensions to be verified and
validated (mostly in consensus), and other parts of the code don't
because its possible that, in some cases (as per RFC 017), we won't have
vote extensions.

This explicitly facilitates that split.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Only sign extensions in precommits, not prevotes

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update privval/file.go

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Temporarily disable extension requirement again for E2E testing

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Reorganize comment for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Leave vote validation and pre-call nil check up to caller of VoteToProto

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Split complex vote validation test into multiple tests

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Universally enforce no vote extensions on any vote type but precommits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Make error messages more generic

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Verify with vote extensions when constructing a VoteSet

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand comment for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add extension check for prevotes prior to signing votes

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix supporting test code to only inject extensions into precommits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Separate vote malleation from signing in vote tests for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add extension signature length check and corresponding test

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Perform basic vote validation in CommitToVoteSet

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-04-28 13:53:44 -04:00
William Banfield
969690d81c abci++: only include meaningful header fields in data passed-through to application (#8216)
closes: #7950
2022-04-11 20:27:50 +00:00
William Banfield
c80734e5af state: synchronize the ProcessProposal implementation with the latest version of the spec (#7961)
This change implements the spec for `ProcessProposal`. It first calls the Tendermint block validation logic to check that all of the proposed block fields are well formed and do not violate any of the rules for Tendermint to consider the block valid and then passes the validated block the `ProcessProposal`.

This change also adds additional fixtures to test the change. It adds the `baseMock` types that holds a mock as well as a reference to `BaseApplication`. If the function was not setup by the test on the contained mock Application, the type delegates to the `BaseApplication` and returns what `BaseApplication` returns. 

The change also switches the `makeState` helper to take an arg struct so that an ABCI application can be plumbed through when needed.

closes: #7656
2022-02-25 18:56:34 +00:00
Jasmina Malicevic
cd875c8a2c types: remove nested evidence field from block (#7765)
* types: replaced EvidenceData in block structure with EvidenceList

* types: introduced toProto, fromProto functions to EvidenceList

* updated Changelog

* Removed comments from tests
2022-02-07 13:57:52 +01:00
M. J. Fromberger
f9c6cc9306 rpc: use encoding/json rather than tmjson (#7670)
The main change here is to use encoding/json to encode and decode RPC 
parameters, rather than the custom tmjson package. This includes:

- Update the HTTP POST handler parameter handling.
- Add field tags to 64-bit integer types to get string encoding (to match amino/tmjson).
- Add marshalers to struct types that mention interfaces.
- Inject wrappers to decode interface arguments in RPC handlers.
2022-01-21 15:10:28 -08:00
M. J. Fromberger
dbe2146d0a rpc: simplify the encoding of interface-typed arguments in JSON (#7600)
Add package jsontypes that implements a subset of the custom libs/json 
package. Specifically it handles encoding and decoding of interface types
wrapped in "tagged" JSON objects. It omits the deep reflection on arbitrary
types, preserving only the handling of type tags wrapper encoding.

- Register interface types (Evidence, PubKey, PrivKey) for tagged encoding.
- Update the existing implementations to satisfy the type.
- Register those types with the jsontypes registry.
- Add string tags to 64-bit integer fields where needed.
- Add marshalers to structs that export interface-typed fields.
2022-01-14 18:14:09 -08:00
Sam Kleinman
332163ede6 testing: remove background contexts (#7509) 2022-01-05 12:42:57 -05:00
Callum Waters
8fe651ba30 e2e: clean up generation of evidence (#6904) 2021-09-07 12:20:43 +02:00
Callum Waters
5ae3c24d55 evidence: update ADR 59 and add comments to the use of common height (#6628) 2021-06-29 18:15:11 +02:00
Callum Waters
72ee5aab26 evidence: separate abci specific validation (#6473) 2021-05-25 16:53:14 +02:00
Callum Waters
5bafedff17 evidence: fix bug with hashes (#6375) 2021-04-21 18:50:39 +02:00
Marko
efd2fde474 privval: add ctx to privval interface (#6240)
## Description

- Add `context.Context` to Privval interface

This pr does not introduce context into our custom privval connection protocol because this will be removed in the next release. When this pr is released.
2021-03-16 14:41:03 +00:00
Callum Waters
3922dde05d evidence: structs can independently form abci evidence (#5610) 2020-11-04 17:14:48 +01:00
Callum Waters
302aec6dcc evidence: use bytes instead of quantity to limit size (#5449)
## Description

Closes: #5432
2020-10-07 09:29:52 +00:00
Callum Waters
ed002cea7e evidence: introduction of LightClientAttackEvidence and refactor of evidence lifecycle (#5361)
evidence: modify evidence types (#5342)

light: detect light client attacks (#5344)

evidence: refactor evidence pool (#5345)

abci: application evidence prepared by evidence pool (#5354)
2020-09-22 10:22:54 +02:00
Marko
710a97d850 evidence: remove amnesia & POLC (#5319)
## Description

remove unneeded types 

![](https://media1.giphy.com/media/fSAyceY3BCgtiQGnJs/giphy.gif)

ref #5288
2020-09-02 13:05:15 +00:00
Marko
82011ee7ab evidence: remove lunatic (#5318)
## Description

remove lunaticEvidence
2020-09-01 16:56:26 +00:00
Marko
e0140e4beb evidence: remove ConflictingHeaders type (#5317)
## Description

Remove ConflictingHeaders & compositeEvidence types


Ref #5288
2020-09-01 16:34:37 +00:00
Callum Waters
b7f6e47a42 evidence: modularise evidence by moving verification function into evidence package (#5234) 2020-08-20 18:11:21 +02:00
Marko
9e98c74e3c crypto: API modifications (#5236)
## Description

This PR aims to make the crypto.PubKey interface more intuitive. 

Changes: 

- `VerfiyBytes` -> `VerifySignature`

Before `Bytes()` was amino encoded, now since it is the byte representation should we get rid of it entirely?

EDIT: decided to keep `Bytes()` as it is useful if you are using the interface instead of the concrete key

Closes: #XXX
2020-08-13 12:29:16 +00:00
Callum Waters
bfcf45a461 evidence: remove validator index verification (#5225) 2020-08-11 15:32:32 +02:00
Callum Waters
312c4f8fe1 evidence: change evidence time to block time (#5219)
adds blockstore interface to evidence and adds fix to byzantine test
2020-08-11 14:39:07 +02:00
Callum Waters
3c21c3546c evidence: remove phantom validator evidence (#5181) 2020-07-31 12:23:58 +02:00
Callum Waters
b5a5f9274d evidence: minor correction to potential amnesia ev validate basic (#5151)
ValidateBasic() for PotentialAmnesiaEvidence checks that the rounds of the two votes are different and does not check Vote Type

ValidateBasic() now also ensures that the first block is not a nil block (else the validator hasn't actually locked onto a block)
2020-07-27 16:12:42 +02:00
Callum Waters
a97d05be4d evidence: check lunatic vote matches header (#5093) 2020-07-08 17:58:03 +02:00
Anton Kaliaev
42be533129 types: verify commit fully
Since the light client work introduced in v0.33 it appears full nodes
are no longer fully verifying commit signatures during block execution -
they stop after +2/3. See in VerifyCommit:
0c7fd316eb/types/validator_set.go (L700-L703)

This means proposers can propose blocks that contain valid +2/3
signatures and then the rest of the signatures can be whatever they
want. They can claim that all the other validators signed just by
including a CommitSig with arbitrary signature data. While this doesn't
seem to impact safety of Tendermint per se, it means that Commits may
contain a lot of invalid data. This is already true of blocks, since
they can include invalid txs filled with garbage, but in that case the
application knows they they are invalid and can punish the proposer. But
since applications dont verify commit signatures directly (they trust
tendermint to do that), they won't be able to detect it.

This can impact incentivization logic in the application that depends on
the LastCommitInfo sent in BeginBlock, which includes which validators
signed. For instance, Gaia incentivizes proposers with a bonus for
including more than +2/3 of the signatures. But a proposer can now claim
that bonus just by including arbitrary data for the final -1/3 of
validators without actually waiting for their signatures. There may be
other tricks that can be played because of this.

In general, the full node should be a fully verifying machine. While
it's true that the light client can avoid verifying all signatures by
stopping after +2/3, the full node can not. Thus the light client and
full node should use distinct VerifyCommit functions if one is going to
stop after +2/3 or otherwise perform less validation (for instance light
clients can also skip verifying votes for nil while full nodes can not).

See a commit with a bad signature that verifies here: 56367fd. From what
I can tell, Tendermint will go on to think this commit is valid and
forward this data to the app, so the app will think the second validator
actually signed when it clearly did not.
2020-07-02 15:41:49 +02:00
Callum Waters
3ecc0ffe7e evidence: replace mock evidence with mocked duplicate vote evidence (#5036) 2020-06-24 07:24:30 +02:00
Callum Waters
65d7ce9c9c evidence: improve amnesia evidence handling (#5003)
fix bug so that PotentialAmnesiaEvidence is being gossiped

handle inbound amnesia evidence correctly

add method to check if potential amnesia evidence is on trial

fix a bug with the height when we upgrade to amnesia evidence

change evidence to using just pointers.

More logging in the evidence module

Co-authored-by: Marko <marbar3778@yahoo.com>
2020-06-23 17:09:14 +02:00
Anton Kaliaev
0b13059216 types: fix evidence timestamp calculation (#5032)
depending on the votes order in DuplicateVoteEvidence is arbitrary.
we should always choose the latest timestamp.

Closes #5030
2020-06-23 07:10:57 +00:00
Marko
dedf0d2350 proto: folder structure adhere to buf (#5025) 2020-06-22 10:00:51 +02:00
Marko
74cae49c3b proto: leftover amino (#4986) 2020-06-15 11:14:36 +02:00
Marko
f6243d8b9e privval: migrate to protobuf (#4985) 2020-06-11 11:54:02 +02:00
Marko
46f6d17601 crypto/merkle: remove simple prefix (#4989)
## Description

This PR removes simple prefix from all types in the crypto/merkle directory.

The two proto types `Proof` & `ProofOp` have been moved to the `proto/crypto/merkle` directory.

proto messge `Proof` was renamed to `ProofOps` and `SimpleProof` message to `Proof`. 

Closes: #2755
2020-06-10 14:57:38 +00:00
Callum Waters
5697e144a7 evidence: adr56 form amnesia evidence (#4821)
Creates Amnesia Evidence which is formed from Potential Amnesia Evidence with either a matching proof or after a period of time denoted as the Amnesia Trial Period. This also adds the code necessary so that Amnesia Evidence can be validated and committed on a block
2020-06-10 13:53:55 +02:00
Marko
99985278d4 evidence: migrate reactor to proto (#4949)
## Description

migration of evidence reactor to proto

Closes: #XXX
2020-06-09 07:54:47 +00:00
Erik Grinaker
ba3a2dde37 rpc: replace Amino with new JSON encoder (#4968)
Migrates the `rpc` package to use new JSON encoder in #4955. Branched off of that PR.

Tests pass, but I haven't done any manual testing beyond that. This should be handled as part of broader 0.34 testing.
2020-06-08 12:04:05 +00:00
Marko
4a87d60736 light: migrate to proto (#4964) 2020-06-08 09:14:58 +02:00
Marko
f9f4916bc3 store: proto migration (#4974)
## Description

migrate store to protobuf

Closes: #XXX
2020-06-08 06:43:29 +00:00
Marko
dc49dcc1c1 evidence: json tags for DuplicateVoteEvidence (#4959)
## Description

Add json tags to duplicate evidence 

Closes: #4958
2020-06-04 22:41:16 +00:00
Marko
a88537bb88 ints: stricter numbers (#4939) 2020-06-04 16:34:56 +02:00
Callum Waters
2014357a19 evidence: remove header from phantom evidence (#4892) 2020-05-27 09:20:45 +02:00
Marko
e03b61abd2 proto: add proto files for ibc unblock (#4853)
## Description

these proto files are meant to help unblock ibc in their quest of migrating the ibc module to proto.

Closes: #XXX
2020-05-25 15:52:34 +00:00
Callum Waters
c0682a3bed evidence: prevent proposer from proposing duplicate pieces of evidence (#4839)
prevent proposer from proposing duplicate pieces of evidence
2020-05-13 18:46:30 +02:00
Anton Kaliaev
b7b721c484 change use of errors.Wrap to fmt.Errorf with %w verb
Closes #4603

Commands used (VIM):

```
:args `rg -l errors.Wrap`
:argdo normal @q | update
```

where q is a macros rewriting the `errors.Wrap` to `fmt.Errorf`.
2020-05-12 03:35:47 +00:00
Callum Waters
7cb3d90e01 evidence: create proof of lock change and implement it in evidence store (#4746)
creates a proof of lock change which is an array of votes that proves the validator was permitted to change locked block and vote again. This proof is stored in the evidence pool and is used as a part of amnesia evidence
2020-05-11 17:06:01 +02:00
Callum Waters
a620e5fd96 evidence: cap evidence to an absolute number (#4780)
The number of evidence that can be committed in a single block is capped by a new evidence parameter called MaxNum
2020-05-11 15:28:08 +02:00
Anton Kaliaev
d202fab478 types: simplify VerifyCommitTrusting
Closes #4783 

It looks like we're validating Commit twice. Also, height and blockID params were coming from the commit, so no need to pass them separately.
2020-05-07 09:24:31 +00:00
Marko
a5a84e11f1 evidence: remove pubkey from duplicate vote evidence
this pr brings over the removal of pubkey from duplicatevote evidence from proto-breakage

ref #4580
2020-04-28 05:47:20 +00:00