Commit Graph

124 Commits

Author SHA1 Message Date
William Banfield
3324f49fb7 inspect: resurrect the inspect command (#9655)
resurrect the inspect command from #6785

Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: Callum Waters <cmwaters19@gmail.com>
2022-12-07 12:15:55 -05:00
Jacob Gadikian
8fd66a6e8d make format (#9744)
## NOTE: this pr exclusively runs commands from the makefile found here


This PR ONLY runs `make format` ... then `make mockery`



Its purpose is to ensure that the review scope of other PR's, which changed .go files and thus triggered the linter that only runs conditionally, have smaller review 
scopes, and should be merged before:

https://github.com/tendermint/tendermint/pull/9738
https://github.com/tendermint/tendermint/pull/9739
https://github.com/tendermint/tendermint/pull/9742






---

#### PR checklist

- [x] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed
2022-11-24 14:53:57 +00:00
JayT106
db26cff58f state: move pruneBlocks from consensus/state to state/execution (#9443) 2022-09-20 11:30:22 +02:00
Sergio Mena
50b5c23d88 Merge branch 'feature/abci++ppp' 2022-08-22 17:16:17 +02:00
Callum Waters
b37f062619 e2e: add evidence tests (#9292) 2022-08-22 13:33:47 +02:00
samricotta
aa303edaef Forward port discard abci responses change (#9286)
* Backport of sam/abci-responses (#9090) (#9159)

Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: Thane Thomson <connect@thanethomson.com>
2022-08-22 12:53:01 +02:00
Callum Waters
2d8df1bd4e proto: deduplicate consensus params (#9287) 2022-08-22 10:50:21 +02:00
William Banfield
1a4d9397e9 abci++: update PrepareProposal API to accept just a list of bytes (#9283)
This changes the ResponsePrepareProposal type, substituting the []TxRecord for just []bytes. This change is made in the .proto file and in all necessary additional places in the code.
2022-08-18 16:33:19 -04:00
Sergio Mena
7dc4f934b0 Merge branch main into feature/abci++ppp 2022-08-12 13:59:19 +02:00
Sergio Mena
25b0c7c78e PrepareProposal/ProcessProposal: Dealing with block data exposed to App (#9219)
* [cherrypicked] abci++: only include meaningful header fields in data passed-through to application (#8216)

* make proto-gen

* Fix kvstore tests

* Small changes to abci protobufs taken from v0.36.x

* make proto-gen (again)

* [Partial cherrypick] Restore `Commit` to the ABCI++ spec, and other late modifications (backport #8796) (#8936)

* Restore `Commit` to the ABCI++ spec, and other late modifications (#8796)

* Added VoteExtensionsEnableHeight

* Fix reference to `modified`

* Removed old pseudo-code, now included in spec

* Removed markdown warnings in abci++_basic_concepts_002_draft.md

* Restored `Commit` in the Methods section

* Addressed remaining markdown warnings

* Revisited intro and basic concepts section

* Extra pass at all spec sections to recover Commit, and other ABCI++ spec modifications

* Fixed links

* make proto-gen

* Remove _primes_ from spec notation

* Update proto/tendermint/abci/types.proto

Co-authored-by: Callum Waters <cmwaters19@gmail.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: Callum Waters <cmwaters19@gmail.com>

* Addressed @cmwaters' comments

* Addressed @angbrav's and @mpoke's comments on spec

* make proto-gen

* Fix MD anchor reference

* Clarify throughout the spec when `ProcessProposal` and `VerifyVoteExtension` are called

* Update spec/abci++/abci++_app_requirements_002_draft.md

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update spec/abci++/abci++_app_requirements_002_draft.md

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update spec/abci++/abci++_app_requirements_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_basic_concepts_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_basic_concepts_002_draft.md

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update spec/abci++/abci++_basic_concepts_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_methods_002_draft.md

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addresed comments

* Renamed 'draft' files

* Adatped links to new filenames

* Fixed links and minor cosmetic changes

* Renamed 'byzantine_validators' to 'misbehavior' in ABCI++ spec and protobufs

* make proto-gen

* Renamed 'byzantine_validators' to 'misbehavior' in the code

* Fixed link

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_basic_concepts.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Update spec/abci++/abci++_methods.md

Co-authored-by: Daniel <daniel.cason@usi.ch>

* Addressed @cason's comments

* Clarified conditions for `ProcessProposal` call at proposer

Co-authored-by: Callum Waters <cmwaters19@gmail.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: Daniel <daniel.cason@usi.ch>
(cherry picked from commit 331860c2a8)

* Fixed merge conflicts

Co-authored-by: Sergio Mena <sergio@informal.systems>

* make proto-gen (and again)

* make build

* fix UTs

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-11 11:21:43 +02:00
Callum Waters
f861062ee2 abci: implement process proposal to spec (#9122) 2022-08-09 15:15:18 +02:00
William Banfield
b92a19b2ce proxy: add initial set of abci metrics (port of #7115) (#9169)
* internal/proxy: add initial set of abci metrics (#7115)

This PR adds an initial set of metrics for use ABCI. The initial metrics enable the calculation of timing histograms and call counts for each of the ABCI methods. The metrics are also labeled as either 'sync' or 'async' to determine if the method call was performed using ABCI's `*Async` methods.

An example of these metrics is included here for reference:
```
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.0001"} 0
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.0004"} 5
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.002"} 12
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.009"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.02"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.1"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.65"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="2"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="6"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="25"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="+Inf"} 13
tendermint_abci_connection_method_timing_sum{chain_id="ci",method="commit",type="sync"} 0.007802058000000001
tendermint_abci_connection_method_timing_count{chain_id="ci",method="commit",type="sync"} 13
```

These metrics can easily be graphed using prometheus's `histogram_quantile(...)` method to pick out a particular quantile to graph or examine. I chose buckets that were somewhat of an estimate of expected range of times for ABCI operations. They start at .0001 seconds and range to 25 seconds. The hope is that this range captures enough possible times to be useful for us and operators.

* fixup

* fixups

* docs: add abci timing metrics to the metrics docs (#7311)

* format table
2022-08-05 13:29:00 -04:00
Sergio Mena
b2409b3345 Follow-up fixes to main PrepareProposal implementation (#9162)
* -----start------

* [cherrypicked] state: panic on ResponsePrepareProposal validation error (#8145)

* state: panic on ResponsePrepareProposal validation error

* lint++

Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* [cherrypicked] abci++: remove CheckTx call from PrepareProposal flow (#8176)

* [cherrypicked] abci++: correct max-size check to only operate on added and unmodified (#8242)

* [cherrypicked] Remove `ModifiedTxStatus` from the spec and the code (#8210)

* Outstanding abci-gen changes to 'pb.go' files

* Removed modified_tx_status from spec and protobufs

* Fix sed for OSX

* Regenerated abci protobufs with 'abci-proto-gen'

* Code changes. UTs e2e tests passing

* Recovered UT: TestPrepareProposalModifiedTxStatusFalse

* Adapted UT

* Fixed UT

* Revert "Fix sed for OSX"

This reverts commit e576708c61.

* Update internal/state/execution_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update internal/state/execution_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed some comments

* Added one test that tests error at the ABCI client + Fixed some mock calls

* Addressed remaining comments

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed William's latest comments

* Adressed Michael's comment

* Fixed UT

* Some md fixes

* More md fixes

* gofmt

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* make proto-gen

* Fixed testcase on PrepareProposal error

* mockery

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-08-03 19:39:17 +02:00
Sergio Mena
d268e56383 Sync PrepareProposal with Spec. Main part (#9158)
* ----start----

* [PARTIAL cherry-pick] ABCI Vote Extension 2 (#6885)

* Cherry-picked #6567: state/types: refactor makeBlock, makeBlocks and makeTxs (#6567)

* [Cherrypicked] types: remove panic from block methods (#7501)

* [cherrypicked] abci++: synchronize PrepareProposal with the newest version of the spec (#8094)

This change implements the logic for the PrepareProposal ABCI++ method call. The main logic for creating and issuing the PrepareProposal request lives in execution.go and is tested in a set of new tests in execution_test.go. This change also updates the mempool mock to use a mockery generated version and removes much of the plumbing for the no longer used ABCIResponses.

* make proto-gen

* Backported EvidenceList's method ToABCI from #7961

* make build

* Fix mockery for Mempool

* mockery

* Backported abci Application mocks from #7961

* mockery2

* Fixed new PrepareProposal test cases in state/execution_test.go

* Fixed returned errors in consensus/state.go

* lint

* Addressed @cmwaters' comment

Co-authored-by: mconcat <monoidconcat@gmail.com>
Co-authored-by: JayT106 <JayT106@users.noreply.github.com>
Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
2022-08-03 17:24:24 +02:00
Sergio Mena
2164883501 Old PrepareProposal and vote extension integration on feature/abci++ppp (#9117)
* abci: PrepareProposal-VoteExtension integration [2nd try] (#7821)

* PrepareProposal-VoteExtension integration (#6915)

* make proto-gen

* Fix protobuf crash in e2e nightly tests

* Update types/vote.go

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Addressed @creachadair's comments

Co-authored-by: mconcat <monoidconcat@gmail.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Proto changes

* make proto-gen

* Fixed UTs

* bump

* lint

* lint2

* lint3

* lint4

* lint5

* lint6

* no_lint

Co-authored-by: mconcat <monoidconcat@gmail.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-08-03 13:29:15 +02:00
Callum Waters
49ec3b9780 chore: lint repo (use american english) (#9144) 2022-08-01 14:24:49 +02:00
Callum Waters
9d354c842e evidence: structs can independently form abci evidence (#5610) 2020-11-05 10:38:42 +01:00
Callum Waters
ed002cea7e evidence: introduction of LightClientAttackEvidence and refactor of evidence lifecycle (#5361)
evidence: modify evidence types (#5342)

light: detect light client attacks (#5344)

evidence: refactor evidence pool (#5345)

abci: application evidence prepared by evidence pool (#5354)
2020-09-22 10:22:54 +02:00
Marko
56911ee352 state: define interface for state store (#5348)
## Description

Make an interface for the state store. 

Closes: #5213
2020-09-15 07:45:48 +00:00
Erik Grinaker
cc247c091b genesis: add support for arbitrary initial height (#5191)
Adds a genesis parameter `initial_height` which specifies the initial block height, as well as ABCI `RequestInitChain.InitialHeight` to pass it to the ABCI application, and `State.InitialHeight` to keep track of the initial height throughout the code. Fixes #2543, based on [RFC-002](https://github.com/tendermint/spec/pull/119). Spec changes in https://github.com/tendermint/spec/pull/135.
2020-08-11 17:03:28 +00:00
Callum Waters
68468fb024 evidence: fix usage of time field in abci evidence (#5201)
* fix usage of time in abci evidence

* update changelong and upgrading

* add test cases
2020-08-04 12:58:48 +02:00
Marko
7e2cc1db5e linter: (1/2) enable errcheck (#5064)
## Description

partially cleanup in preparation for errcheck

i ignored a bunch of defer errors in tests but with the update to go 1.14 we can use `t.Cleanup(func() { if err := <>; err != nil {..}}` to cover those errors, I will do this in pr number two of enabling errcheck.

ref #5059
2020-07-01 15:13:11 +00:00
Callum Waters
3ecc0ffe7e evidence: replace mock evidence with mocked duplicate vote evidence (#5036) 2020-06-24 07:24:30 +02:00
Anton Kaliaev
ceac02b891 types: add AppVersion to ConsensusParams (#5031)
Co-authored-by: JamesRay <66258875@qq.com>

making it possible to change app version via EndBlock
2020-06-23 12:14:16 +04:00
Marko
dedf0d2350 proto: folder structure adhere to buf (#5025) 2020-06-22 10:00:51 +02:00
Marko
51da4fe356 types: rename partsheader to partsetheader (#5029)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-06-22 09:38:03 +02:00
Marko
31a361d119 proto: move keys to oneof (#4983) 2020-06-11 11:10:37 +02:00
Marko
9ef266b88f types: migrate params to protobuf (#4962) 2020-06-05 15:29:53 +02:00
Marko
a88537bb88 ints: stricter numbers (#4939) 2020-06-04 16:34:56 +02:00
Callum Waters
47cfadb0aa evidence: refactor evidence mocks throughout packages (#4787)
Predominantly following the discussions regarding the conventions of using mocks, I have decided to revert back to the previous state where mocks were specialized and stored in the separate packages that used them rather then have a generalized mock in the evidence package.

This also was a problem as the state package were running tests too slow and occasionally timing out unnecessarily.

For the replay file I renamed mockEvidencePool to emptyEvidencePool to illustrate that it was intentionally like this and not give the impression that testing software was being used in production

Closes: #4786
2020-05-05 13:11:44 +04:00
Callum
15a9f1760d move mempool mock directory 2020-04-29 15:25:01 +02:00
Callum
6dc971052a replace old mocks with new mock 2020-04-28 12:59:18 +02:00
Marko
7a6ca0c1bf evidence: remove unused param (#4726)
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
2020-04-22 17:18:24 +02:00
Erik Grinaker
4298bbcc4e add support for block pruning via ABCI Commit response (#4588)
* Added BlockStore.DeleteBlock()

* Added initial block pruner prototype

* wip

* Added BlockStore.PruneBlocks()

* Added consensus setting for block pruning

* Added BlockStore base

* Error on replay if base does not have blocks

* Handle missing blocks when sending VoteSetMaj23Message

* Error message tweak

* Properly update blockstore state

* Error message fix again

* blockchain: ignore peer missing blocks

* Added FIXME

* Added test for block replay with truncated history

* Handle peer base in blockchain reactor

* Improved replay error handling

* Added tests for Store.PruneBlocks()

* Fix non-RPC handling of truncated block history

* Panic on missing block meta in needProofBlock()

* Updated changelog

* Handle truncated block history in RPC layer

* Added info about earliest block in /status RPC

* Reorder height and base in blockchain reactor messages

* Updated changelog

* Fix tests

* Appease linter

* Minor review fixes

* Non-empty BlockStores should always have base > 0

* Update code to assume base > 0 invariant

* Added blockstore tests for pruning to 0

* Make sure we don't prune below the current base

* Added BlockStore.Size()

* config: added retain_blocks recommendations

* Update v1 blockchain reactor to handle blockstore base

* Added state database pruning

* Propagate errors on missing validator sets

* Comment tweaks

* Improved error message

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* use ABCI field ResponseCommit.retain_height instead of retain-blocks config option

* remove State.RetainHeight, return value instead

* fix minor issues

* rename pruneHeights() to pruneBlocks()

* noop to fix GitHub borkage

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-04-03 08:38:32 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
Marko
6d91c1faf4 evidence: introduce time.Duration to evidence params (#4254)
* evidence: introduce time.Duration to evidence params

- add time.duration to evidence
- this pr is taking pr #2606 and updating it to use both time and height

- closes #2565

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing and genesis cfg in signer harness

* remove debugging fmt

* change maxageheight to maxagenumblocks, rename other things to block instead of height

* further check of duration

* check duration to not send peers outdated evidence

* change some lines, onward and upward

* refactor evidence package

* add a changelog pending entry

* make mockbadevidence have time and use it

* add what could possibly be called a test case

* remove mockbadevidence and mockgoodevidence in favor of mockevidence

* add a comment for err that is returned

* add a changelog for removal of good & bad evidence

* add a test for adding evidence

* fix test

* add ev to types in testcase

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix tests

* fix linting

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-01-08 10:46:37 +01:00
Marko
3e2751d274 lint: Enable Golint (#4212)
* Fix many golint errors

* Fix golint errors in the 'lite' package

* Don't export Pool.store

* Fix typo

* Revert unwanted changes

* Fix errors in counter package

* Fix linter errors in kvstore package

* Fix linter error in example package

* Fix error in tests package

* Fix linter errors in v2 package

* Fix linter errors in consensus package

* Fix linter errors in evidence package

* Fix linter error in fail package

* Fix linter errors in query package

* Fix linter errors in core package

* Fix linter errors in node package

* Fix linter errors in mempool package

* Fix linter error in conn package

* Fix linter errors in pex package

* Rename PEXReactor export to Reactor

* Fix linter errors in trust package

* Fix linter errors in upnp package

* Fix linter errors in p2p package

* Fix linter errors in proxy package

* Fix linter errors in mock_test package

* Fix linter error in client_test package

* Fix linter errors in coretypes package

* Fix linter errors in coregrpc package

* Fix linter errors in rpcserver package

* Fix linter errors in rpctypes package

* Fix linter errors in rpctest package

* Fix linter error in json2wal script

* Fix linter error in wal2json script

* Fix linter errors in kv package

* Fix linter error in state package

* Fix linter error in grpc_client

* Fix linter errors in types package

* Fix linter error in version package

* Fix remaining errors

* Address review comments

* Fix broken tests

* Reconcile package coregrpc

* Fix golangci bot error

* Fix new golint errors

* Fix broken reference

* Enable golint linter

* minor changes to bring golint into line

* fix failing test

* fix pex reactor naming

* address PR comments
2019-12-05 10:12:08 +01:00
Anton Kaliaev
ad715fe966 types: change Commit to consist of just signatures (#4146)
* types: change `Commit` to consist of just signatures

These are final changes towards removing votes from commit and leaving
only signatures (see ADR-25)

Fixes #1648

* bring back TestCommitToVoteSetWithVotesForAnotherBlockOrNilBlock

+ add absent flag to Vote to indicate that it's for another block

* encode nil votes as CommitSig with BlockIDFlagAbsent

+ make Commit#Precommits array of non-pointers
because precommit will never be nil

* add NewCommitSigAbsent and Absent() funcs

* uncomment validation in CommitSig#ValidateBasic

* add comments to ValidatorSet funcs

* add a changelog entry

* break instead of continue

continue does not make sense in these cases

* types: rename Commit#Precommits to Signatures

* swagger: fix /commit response

* swagger: change block_id_flag type

* fix merge conflicts
2019-11-26 14:10:38 +04:00
Phil Salant
bc572217c0 Fix linter errors thrown by lll (#3970)
* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter
2019-10-17 10:42:28 +02:00
Phil Salant
d1d517a9b7 linters: enable scopelint (#3963)
* Pin range scope vars

* Don't disable scopelint

This PR repairs linter errors seen when running the following commands:
golangci-lint run --no-config --disable-all=true --enable=scopelint

Contributes to #3262
2019-09-11 09:15:18 +04:00
Thane Thomson
228bba799d state: add more tests for block validation (#3674)
* Expose priv validators for use in testing

* Generalize block header validation test past height 1

* Remove ineffectual assignment

* Remove redundant SaveState call

* Reorder comment for clarity

* Use the block executor ApplyBlock function instead of implementing a stripped-down version of it

* Remove commented-out code

* Remove unnecessary test

The required tests already appear to be implemented (implicitly) through
the TestValidateBlockHeader test.

* Allow for catching of specific error types during TestValidateBlockCommit

* Make return error testable

* Clean up and add TestValidateBlockCommit code

* Fix formatting

* Extract function to create a new mock test app

* Update comment for clarity

* Fix comment

* Add skeleton code for evidence-related test

* Allow for addressing priv val by address

* Generalize test beyond a single validator

* Generalize TestValidateBlockEvidence past first height

* Reorder code to clearly separate tests and utility code

* Use a common constant for stop height for testing in state/validation_test.go

* Refactor errors to resemble existing conventions

* Fix formatting

* Extract common helper functions

Having the tests littered with helper functions makes them less easily
readable imho, so I've pulled them out into a separate file. This also
makes it easier to see what helper functions are available during
testing, so we minimize the chance of duplication when writing new
tests.

* Remove unused parameter

* Remove unused parameters

* Add field keys

* Remove unused height constant

* Fix typo

* Fix incorrect return error

* Add field keys

* Use separate package for tests

This refactors all of the state package's tests into a state_test
package, so as to keep any usage of the state package's internal methods
explicit.

Any internal methods/constants used by tests are now explicitly exported
in state/export_test.go

* Refactor: extract helper function to make, validate, execute and commit a block

* Rename state function to makeState

* Remove redundant constant for number of validators

* Refactor mock evidence registration into TestMain

* Remove extraneous nVals variable

* Replace function-level TODOs with file-level TODO and explanation

* Remove extraneous comment

* Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)
2019-06-21 17:29:29 -04:00
Marko
866b343c0c Changes to files that had linting issue (#3731)
- Govet issues fixed
- 1 gosec issue solved using nolint

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
2019-06-21 09:58:32 +04:00
Thane Thomson
9d5ba576ee abci: Refactor ABCI CheckTx and DeliverTx signatures (#3735)
* Refactor signature of Application.CheckTx

* Refactor signature of Application.DeliverTx

* Refactor example variable names for clarity and consistency

* Rename method variables for consistency

* Rename method variables for consistency

* add a changelog entry

* update docs
2019-06-21 09:56:27 +04:00
Alexander Bezobchuk
ab0835463f abci: Refactor tagging events using list of lists (#3643)
## PR

This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events.

    Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like:

"rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}]
"sender": [{"address": "...", "balance": "100uatom"}]

The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...'

    ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair.
    PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings.

closes: #1859
closes: #2905

## Commits:

* Implement Event ABCI type and updates responses to use events

* Update messages_test.go

* Update kvstore.go

* Update event_bus.go

* Update subscription.go

* Update pubsub.go

* Update kvstore.go

* Update query logic to handle slice of strings in events

* Update Empty#Matches and unit tests

* Update pubsub logic

* Update EventBus#Publish

* Update kv tx indexer

* Update godocs

* Update ResultEvent to use slice of strings; update RPC

* Update more tests

* Update abci.md

* Check for key in validateAndStringifyEvents

* Fix KV indexer to skip empty keys

* Fix linting errors

* Update CHANGELOG_PENDING.md

* Update docs/spec/abci/abci.md

Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* Update abci/types/types.proto

Co-Authored-By: Ethan Buchman <ethan@coinculture.info>

* Update docs/spec/abci/abci.md

Co-Authored-By: Ethan Buchman <ethan@coinculture.info>

* Update libs/pubsub/query/query.go

Co-Authored-By: Ethan Buchman <ethan@coinculture.info>

* Update match function to match if ANY value matches

* Implement TestSubscribeDuplicateKeys

* Update TestMatches to include multi-key test cases

* Update events.go

* Update Query interface godoc

* Update match godoc

* Add godoc for matchValue

* DRY-up tx indexing

* Return error from PublishWithEvents in EventBus#Publish

* Update PublishEventNewBlockHeader to return an error

* Fix build

* Update events doc in ABCI

* Update ABCI events godoc

* Implement TestEventBusPublishEventTxDuplicateKeys

* Update TestSubscribeDuplicateKeys to be table-driven

* Remove mod file

* Remove markdown from events godoc

* Implement TestTxSearchDeprecatedIndexing test
2019-06-12 14:03:45 +02:00
Anton Kaliaev
5051a1f7bc mempool: move interface into mempool package (#3524)
## Description

Refs #2659

Breaking changes in the mempool package:

[mempool] #2659 Mempool now an interface
    old Mempool renamed to CListMempool
    NewMempool renamed to NewCListMempool
    Option renamed to CListOption
    MempoolReactor renamed to Reactor
    NewMempoolReactor renamed to NewReactor
    unexpose TxID method
    TxInfo.PeerID renamed to SenderID
    unexpose MempoolReactor.Mempool

Breaking changes in the state package:

[state] #2659 Mempool interface moved to mempool package
    MockMempool moved to top-level mock package and renamed to Mempool

Non Breaking changes in the node package:

[node] #2659 Add Mempool method, which allows you to access mempool

## Commits

* move Mempool interface into mempool package

Refs #2659

Breaking changes in the mempool package:

- Mempool now an interface
- old Mempool renamed to CListMempool

Breaking changes to state package:

- MockMempool moved to mempool/mock package and renamed to Mempool
- Mempool interface moved to mempool package

* assert CListMempool impl Mempool

* gofmt code

* rename MempoolReactor to Reactor

- combine everything into one interface
- rename TxInfo.PeerID to TxInfo.SenderID
- unexpose MempoolReactor.Mempool

* move mempool mock into top-level mock package

* add a fixme

TxsFront should not be a part of the Mempool interface
because it leaks implementation details. Instead, we need to come up
with general interface for querying the mempool so the MempoolReactor
can fetch and broadcast txs to peers.

* change node#Mempool to return interface

* save commit = new reactor arch

* Revert "save commit = new reactor arch"

This reverts commit 1bfceacd9d.

* require CListMempool in mempool.Reactor

* add two changelog entries

* fixes after my own review

* quote interfaces, structs and functions

* fixes after Ismail's review

* make node's mempool an interface

* make InitWAL/CloseWAL methods a part of Mempool interface

* fix merge conflicts

* make node's mempool an interface
2019-05-04 10:41:31 +04:00
JamesRay
2c26d95ab9 cs/replay: execCommitBlock should not read from state.lastValidators (#3067)
* execCommitBlock should not read from state.lastValidators

* fix height 1

* fix blockchain/reactor_test

* fix consensus/mempool_test

* fix consensus/reactor_test

* fix consensus/replay_test

* add CHANGELOG

* fix consensus/reactor_test

* fix consensus/replay_test

* add a test for replay validators change

* fix mem_pool test

* fix byzantine test

* remove a redundant code

* reduce validator change blocks to 6

* fix

* return peer0 config

* seperate testName

* seperate testName 1

* seperate testName 2

* seperate app db path

* seperate app db path 1

* add a lock before startNet

* move the lock to reactor_test

* simulate just once

* try to find problem

* handshake only saveState when app version changed

* update gometalinter to 3.0.0 (#3233)

in the attempt to fix https://circleci.com/gh/tendermint/tendermint/43165

also

    code is simplified by running gofmt -s .
    remove unused vars
    enable linters we're currently passing
    remove deprecated linters
(cherry picked from commit d470945503)

* gofmt code

* goimport code

* change the bool name to testValidatorsChange

* adjust receive kvstore.ProtocolVersion

* adjust receive kvstore.ProtocolVersion 1

* adjust receive kvstore.ProtocolVersion 3

* fix merge execution.go

* fix merge develop

* fix merge develop 1

* fix run cleanupFunc

* adjust code according to reviewers' opinion

* modify the func name match the convention

* simplify simulate a chain containing some validator change txs 1

* test CI error

* Merge remote-tracking branch 'upstream/develop' into fixReplay 1

* fix pubsub_test

* subscribeUnbuffered vote channel
2019-05-01 17:15:53 -04:00
Anton Kaliaev
b6a510a3e7 make ineffassign linter pass (#3386)
Refs #3262

This fixes two small bugs:

1) lite/dbprovider: return `ok` instead of true in parse* functions. It's weird that we're ignoring `ok` value before.
2) consensus/state: previously because of the shadowing we almost never output "Error with msg". Now we declare both `added` and `err` in the beginning of the function, so there's no shadowing.
2019-03-08 09:46:09 +04:00
Anton Kaliaev
2137ecc130 pubsub 2.0 (#3227)
* green pubsub tests :OK:

* get rid of clientToQueryMap

* Subscribe and SubscribeUnbuffered

* start adapting other pkgs to new pubsub

* nope

* rename MsgAndTags to Message

* remove TagMap

it does not bring any additional benefits

* bring back EventSubscriber

* fix test

* fix data race in TestStartNextHeightCorrectly

```
Write at 0x00c0001c7418 by goroutine 796:
  github.com/tendermint/tendermint/consensus.TestStartNextHeightCorrectly()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1296 +0xad
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Previous read at 0x00c0001c7418 by goroutine 858:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1631 +0x1366
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1476 +0x8f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:667 +0xa1e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:628 +0x794

Goroutine 796 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1117 +0x4ee
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:214 +0x332

Goroutine 858 (running) created at:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:334 +0x221
  github.com/tendermint/tendermint/consensus.startTestRound()
      /go/src/github.com/tendermint/tendermint/consensus/common_test.go:122 +0x63
  github.com/tendermint/tendermint/consensus.TestStateFullRound1()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:255 +0x397
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
```

* fixes after my own review

* fix formatting

* wait 100ms before kicking a subscriber out

+ a test for indexer_service

* fixes after my second review

* no timeout

* add changelog entries

* fix merge conflicts

* fix typos after Thane's review

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* reformat code

* rewrite indexer service in the attempt to fix failing test

https://github.com/tendermint/tendermint/pull/3227/#issuecomment-462316527

* Revert "rewrite indexer service in the attempt to fix failing test"

This reverts commit 0d9107a098.

* another attempt to fix indexer

* fixes after Ethan's review

* use unbuffered channel when indexing transactions

Refs https://github.com/tendermint/tendermint/pull/3227#discussion_r258786716

* add a comment for EventBus#SubscribeUnbuffered

* format code
2019-02-22 23:11:27 -05:00
Anca Zamfir
8a9eecce7f test blockExec does not panic if all vals removed (#3241)
Fix for #3224
Also address #2084
2019-02-12 09:02:44 +04:00
Ethan Buchman
4f2ef36701 types.NewCommit (#3275)
* types.NewCommit

* use types.NewCommit everywhere

* fix log in unsafe_reset

* memoize height and round in constructor

* notes about deprecating toVote

* bring back memoizeHeightRound
2019-02-08 18:40:41 -05:00