Commit Graph

128 Commits

Author SHA1 Message Date
Marko
a12bfca93d proto: move all proto dirs to /proto (#5012)
## Description

This PR moves all proto files under one dir (`/proto`). The script to generate adding functionality to copy the files that still need to be in the same place. (abci & rpc)

renames every proto package from `tendermint.proto.<pkg_name>` to `tendermint.<pkg_name>`

Removes unneeded types in privval proto directory

Closes: #XXX
2020-06-16 14:41:36 +00:00
Ismail Khoffi
4774a8ec61 non breaking signbytes (#5008)
* test-vectors for backwards compatibility:

 - copy & paste test-vectors from v0.33.5 to ensure
 backwards compatibility for vote's SignBytes

* WIP: everything besides time seems to match :-/

* almost

* Found the culprit: field nums weren't consecutive ints ...

* fix order of partset header too

* this last votes-related test can easily be fixed

* some minor changes and fix last failing test

* move proto types back to stdtime, fix various linting

* use libs/protoio

* remvoe commented code

* add comments

* fix tests

* uncomment testscases

* dont ignore error panic

* fix signable test

* fix happy path testing

* fix comment

Co-authored-by: Marko Baricevic <marbar3778@yahoo.com>
2020-06-15 21:54:58 +02:00
Marko
31a361d119 proto: move keys to oneof (#4983) 2020-06-11 11:10:37 +02:00
Marko
46f6d17601 crypto/merkle: remove simple prefix (#4989)
## Description

This PR removes simple prefix from all types in the crypto/merkle directory.

The two proto types `Proof` & `ProofOp` have been moved to the `proto/crypto/merkle` directory.

proto messge `Proof` was renamed to `ProofOps` and `SimpleProof` message to `Proof`. 

Closes: #2755
2020-06-10 14:57:38 +00:00
Marko
9ef266b88f types: migrate params to protobuf (#4962) 2020-06-05 15:29:53 +02:00
Marko
b4aca980e9 proto: change to use gogofaster (#4957) 2020-06-05 14:18:44 +02:00
Marko
b9af87c4ea state: proto migration (#4951) 2020-06-05 10:47:16 +02:00
Marko
7c576f02ab keys: change to []bytes (#4950) 2020-06-04 15:32:42 +02:00
Marko
99d88cbe2f crypto: remove key suffixes (#4941)
## Description

- remove keyname suffix from keys


Closes: #2228
2020-06-03 05:46:29 +00:00
Erik Grinaker
0566646373 test/abci: use random socket names to avoid collisions (#4885)
Fixes these test failures:

```
=== RUN   TestKVStore
### Testing KVStore 
I[2020-05-25|13:32:31.303] Starting ABCIServer service                  module=abci-server impl=ABCIServer
    TestKVStore: example_test.go:48: 
                Error Trace:    example_test.go:48
                                                        example_test.go:28
                Error:          Received unexpected error:
                                listen unix test.sock: bind: address already in use
                Test:           TestKVStore
                Messages:       Error starting socket server
--- FAIL: TestKVStore (0.00s) 
```
2020-05-26 11:33:49 +00:00
Marko
89a3cb2b0a libs: remove kv (#4874)
## Description

In https://github.com/tendermint/tendermint/pull/4466 a new type was created (EventAttribute) that replaced KV.Pair. This made the type Pair deadcode, this pr removes that type.

I don't think a changelog entry is needed since it is not being used anymore. But will add a section to the upgrading.md to let users know that it was replaced

Closes: #XXX
2020-05-26 06:37:24 +00:00
Marko
243dfbd585 proto: remove test files
## Description

remove test files for proto stubs

Closes: #XXX
2020-05-13 14:30:33 +00:00
Marko
cf4e056a33 abci: remove python examples
## Description

remove unused and unmaintained python examples of abci

Closes: #XXX
2020-05-12 09:52:04 +00:00
Erik Grinaker
1af80b44fc abci: remove protoreplace script
gogoproto is here to stay.
2020-05-12 09:03:20 +00:00
Marko
8d63d7192f abci: regenerate proto files
## Description

regenerate proto stubs

I do find max_num to be vague. possibly renaming to max_entries or max_evidence_entries may be more descriptive?

Closes: #XXX
2020-05-11 16:54:54 +00:00
Callum Waters
a620e5fd96 evidence: cap evidence to an absolute number (#4780)
The number of evidence that can be committed in a single block is capped by a new evidence parameter called MaxNum
2020-05-11 15:28:08 +02:00
Erik Grinaker
81c2798df0 abci: fix protobuf lint issues
Fix some linter issues to conform with the Protobuf style guide.

The state sync enum changes are ok to break since it's not released yet. Personally I find the uppercase kind of ugly, but that's what the guide says. Couldn't find a way to generate camel case in Go, short of specifying custom names for each and every enum variant.

Another option would be to simply disable the enum case lint.
2020-05-07 14:40:22 +00:00
Marko
b7c2d7a977 lint: enable nolintlinter, disable on tests
## Description
- enable nolintlint
- disable linting on tests

Closes: #XXX
2020-05-04 07:49:53 +00:00
Erik Grinaker
569981325a add ABCI interface for state sync snapshots (#4704)
Adds the ABCI interface for [state sync](https://github.com/tendermint/tendermint/issues/828) as outlined in [ADR-053](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-053-state-sync-prototype.md), and bumps ABCIVersion to `0.17.0`.

The interface adds a new ABCI connection which Tendermint can use to query and load snapshots from the app (for serving snapshots to other nodes), and to offer and apply snapshots to the app (for state syncing a local node from peers).

Split out from the original PR in #4645, state sync reactor will be submitted as a separate PR. The interface is implemented by the Cosmos SDK in https://github.com/cosmos/cosmos-sdk/pull/5803.
2020-04-29 10:32:09 +02:00
Anton Kaliaev
33a19dabd7 abci/server: print panic & stack trace to STDERR if logger is not set
Closes #4382
2020-04-22 11:42:45 +00:00
Diep Pham
843d63f935 indexer: allow indexing an event at runtime (#4466)
The PR added a new field `index` to event attribute, that will cause indexer service to index the event if set to true.
2020-04-22 12:07:03 +02:00
Anton Kaliaev
41c11ad2c1 evidence: handling evidence from light client(s) (#4532)
Closes: #4530

This PR contains logic for both submitting an evidence by the light client (lite2 package) and receiving it on the Tendermint side (/broadcast_evidence RPC and/or EvidenceReactor#Receive). Upon receiving the ConflictingHeadersEvidence (introduced by this PR), the Tendermint validates it, then breaks it down into smaller pieces (DuplicateVoteEvidence, LunaticValidatorEvidence, PhantomValidatorEvidence, PotentialAmnesiaEvidence). Afterwards, each piece of evidence is verified against the state of the full node and added to the pool, from which it's reaped upon block creation.

* rpc/client: do not pass height param if height ptr is nil

* rpc/core: validate incoming evidence!

* only accept ConflictingHeadersEvidence if one

of the headers is committed from this full node's perspective

This simplifies the code. Plus, if there are multiple forks, we'll
likely to receive multiple ConflictingHeadersEvidence anyway.

* swap CommitSig with Vote in LunaticValidatorEvidence

Vote is needed to validate signature

* no need to embed client

http is a provider and should not be used as a client
2020-04-22 11:29:05 +04:00
Erik Grinaker
4298bbcc4e add support for block pruning via ABCI Commit response (#4588)
* Added BlockStore.DeleteBlock()

* Added initial block pruner prototype

* wip

* Added BlockStore.PruneBlocks()

* Added consensus setting for block pruning

* Added BlockStore base

* Error on replay if base does not have blocks

* Handle missing blocks when sending VoteSetMaj23Message

* Error message tweak

* Properly update blockstore state

* Error message fix again

* blockchain: ignore peer missing blocks

* Added FIXME

* Added test for block replay with truncated history

* Handle peer base in blockchain reactor

* Improved replay error handling

* Added tests for Store.PruneBlocks()

* Fix non-RPC handling of truncated block history

* Panic on missing block meta in needProofBlock()

* Updated changelog

* Handle truncated block history in RPC layer

* Added info about earliest block in /status RPC

* Reorder height and base in blockchain reactor messages

* Updated changelog

* Fix tests

* Appease linter

* Minor review fixes

* Non-empty BlockStores should always have base > 0

* Update code to assume base > 0 invariant

* Added blockstore tests for pruning to 0

* Make sure we don't prune below the current base

* Added BlockStore.Size()

* config: added retain_blocks recommendations

* Update v1 blockchain reactor to handle blockstore base

* Added state database pruning

* Propagate errors on missing validator sets

* Comment tweaks

* Improved error message

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* use ABCI field ResponseCommit.retain_height instead of retain-blocks config option

* remove State.RetainHeight, return value instead

* fix minor issues

* rename pruneHeights() to pruneBlocks()

* noop to fix GitHub borkage

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-04-03 08:38:32 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
mergify[bot]
4c8e3c8145 fix: proto-breakage (#4506)
* fix: fix proto-breakage

- this is amed to fix proto breakage for consumers

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix for importing third_party everywhere

* undo change

* test breakage change

* test ssh

* test https

* change ssh to https

* fix phony
2020-03-05 08:41:36 +00:00
Erik Grinaker
c33576e72a example/kvstore: return ABCI query height (#4509)
* example/kvstore: return ABCI query height

* CHANGELOG: added PR link

* Updated tests
2020-03-01 20:03:28 +00:00
Erik Grinaker
fe11219795 Fix unexported returns (#4450) 2020-02-21 19:21:39 +01:00
Erik Grinaker
8f48c49543 Fix some golangci-lint warnings (#4448) 2020-02-20 13:43:40 +01:00
Marko
9a9e8c5bb3 proto: minor linting to proto files (#4386)
* proto: minor linting

minor linting after working with the proto files in the sdk.

there is no logic change just spacing fixes

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* hardcore linting
2020-02-11 15:07:05 +01:00
Marko
31fd99a91a proto: add buf and protogen script (#4369)
* proto: add buf and protogen script

- add buf with minimal changes
- add protogen script to easier generate proto files

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add protoc needs

* add some needed shell cmds

* remove buf from tools as it is not needed everytime

* add proto lint and breakage to ci

* add section in changelog and upgrading files

* address pr comments

* remove space in circle config

* remove spaces in makefile comment

* add section on contributing on how to work with proto

* bump buf to 0.7

* test bufbuild image

* test install make in bufbuild image

* revert to tendermintdev image

* Update Makefile

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-02-11 10:31:15 +01:00
Tess Rinearson
fa34ff96b8 abci: fix broken spec link (#4366) 2020-02-03 15:00:31 +01:00
Marko
6b71b928be dep: update tm-db to 0.4.0 (#4289)
* dep: update tm-db to 0.4.0

- update 0.4.0 as it is a breaking change and cannot be handled by depndabot

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

*  more work towards error handling

* error and emtpy bytes handling

* work on tests

* add changelog entry, change some error handling

* address some pr comments

* panic in a few more places

* move error higher up

* redo some error handling

* fix some bz == nil to len(bz) == 0

* change statebytes
2020-01-13 18:45:16 +01:00
Marko
6d91c1faf4 evidence: introduce time.Duration to evidence params (#4254)
* evidence: introduce time.Duration to evidence params

- add time.duration to evidence
- this pr is taking pr #2606 and updating it to use both time and height

- closes #2565

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing and genesis cfg in signer harness

* remove debugging fmt

* change maxageheight to maxagenumblocks, rename other things to block instead of height

* further check of duration

* check duration to not send peers outdated evidence

* change some lines, onward and upward

* refactor evidence package

* add a changelog pending entry

* make mockbadevidence have time and use it

* add what could possibly be called a test case

* remove mockbadevidence and mockgoodevidence in favor of mockevidence

* add a comment for err that is returned

* add a changelog for removal of good & bad evidence

* add a test for adding evidence

* fix test

* add ev to types in testcase

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix tests

* fix linting

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-01-08 10:46:37 +01:00
Marko
9bd0f9e634 lint: golint issue fixes (#4258)
* lint: golint issue fixes

- on my local machine golint is a lot stricter than the bot so slowly going through and fixing things.

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* more fixes from golint

* remove isPeerPersistentFn

* add changelog entry
2019-12-17 13:02:45 +01:00
Marko
7b52f51700 libs/common: Refactor libs/common 5 (#4240)
* libs/common: Refactor libs/common 5

- move mathematical functions and types out of `libs/common` to math pkg
- move net functions out of `libs/common` to net pkg
- move string functions out of `libs/common` to strings pkg
- move async functions out of `libs/common` to async pkg
- move bit functions out of `libs/common` to bits pkg
- move cmap functions out of `libs/common` to cmap pkg
- move os functions out of `libs/common` to os pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing issues

* fix tests

closes #41417

woooooooooooooooooo kill the cmn pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix goimport issues

* run gofmt
2019-12-12 09:33:27 +01:00
Marko
89f0bbbd76 libs/common: Refactor libs/common 4 (#4237)
* libs/common: Refactor libs/common 4

- move byte function out of cmn to its own pkg
- move tempfile out of cmn to its own pkg
- move throttletimer to its own pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix linting issues
2019-12-11 23:16:35 +01:00
Marko
27b00cf8d1 libs/common: refactor libs common 3 (#4232)
* libs/common: refactor libs common 3

- move nil.go into types folder and make private
- move service & baseservice out of common into service pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry
2019-12-11 09:31:25 +01:00
Prince Sinha
efc27ab0b9 abci/kvstore: return LastBlockHeight and LastBlockAppHash in Info (#4233)
Fixes #4177
2019-12-11 12:09:43 +04:00
Marko
afc4d7a61f libs/common: refactor libs/common 2 (#4231)
* libs/common: refactor libs/common 2

- move random function to there own pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* change imports and usage throughout repo

* fix goimports

* add changelog entry
2019-12-10 18:38:34 +01:00
Marko
dfebac86f7 libs/common: refactor libs/common 01 (#4230)
* libs/common: Refactor libs/common 01

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* regenerate proto files, move intslice to where its used

* update kv.KVPair(s) to kv.Pair(s)

* add changelog entry

* make intInSlice private
2019-12-10 12:40:01 +01:00
Marko
00c4b52a58 docs: fix broken ecosystem link (#4222)
I put it as the awesome repo ecosytem section as this is currently the only place where there is tendermint only ecosystem information.

- closes #4198

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
2019-12-06 19:04:44 +04:00
Marko
3e2751d274 lint: Enable Golint (#4212)
* Fix many golint errors

* Fix golint errors in the 'lite' package

* Don't export Pool.store

* Fix typo

* Revert unwanted changes

* Fix errors in counter package

* Fix linter errors in kvstore package

* Fix linter error in example package

* Fix error in tests package

* Fix linter errors in v2 package

* Fix linter errors in consensus package

* Fix linter errors in evidence package

* Fix linter error in fail package

* Fix linter errors in query package

* Fix linter errors in core package

* Fix linter errors in node package

* Fix linter errors in mempool package

* Fix linter error in conn package

* Fix linter errors in pex package

* Rename PEXReactor export to Reactor

* Fix linter errors in trust package

* Fix linter errors in upnp package

* Fix linter errors in p2p package

* Fix linter errors in proxy package

* Fix linter errors in mock_test package

* Fix linter error in client_test package

* Fix linter errors in coretypes package

* Fix linter errors in coregrpc package

* Fix linter errors in rpcserver package

* Fix linter errors in rpctypes package

* Fix linter errors in rpctest package

* Fix linter error in json2wal script

* Fix linter error in wal2json script

* Fix linter errors in kv package

* Fix linter error in state package

* Fix linter error in grpc_client

* Fix linter errors in types package

* Fix linter error in version package

* Fix remaining errors

* Address review comments

* Fix broken tests

* Reconcile package coregrpc

* Fix golangci bot error

* Fix new golint errors

* Fix broken reference

* Enable golint linter

* minor changes to bring golint into line

* fix failing test

* fix pex reactor naming

* address PR comments
2019-12-05 10:12:08 +01:00
Alex Kogon
72f39e2d1f Fixed typo (#4181) 2019-11-21 17:23:37 -05:00
Anthony
fb9e667f18 prefix proto types (#4007)
Fixes #3986

This pull request is prefixing all the types in proto to avoid conflict.

When a go application is using Tendermint as a library and also define similar types in gogo proto some conflicts might occur (as types is a common package in go).

By prefixing the types with tendermint, this highly reduces the risk of conflicts.

BREAKING CHANGE.

This modification breaks the ABCI Application endpoint.

What was accessible before with `/types.ABCIApplication/Flush` is now accessible with `/tendermint.abci.types.ABCIApplication/Flush`.
2019-11-21 12:00:33 +04:00
Anton Kaliaev
3e1516b624 linters: enable stylecheck (#4153)
Refs #3262
2019-11-16 19:35:39 +04:00
Marko
1604047c39 abci: remove TotalTxs and NumTxs from Header (#3783)
* Removal of TotalTx & NumTx

- Removed totalTx and numTx

closes #2521

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* abci proto changes

* proto number fix

* txfilter_test fix

* comments on PR

* further changes

* bring back metrics

* fix indexer

* fix TestBlockMaxDataBytes and TestBlockMaxDataBytesUnknownEvidence

* indexer service back to header

* statistics.go fix

* fix ci

* listen for blocks, not headers

to be able to record txs throughput

* fix TestNetworkNewBlock

* fix tests

* fix tests in types package

* fixes after Anton's review

* fix tests

* bring back `consensus_total_txs` metric

I mistakenly thought it was removed.

* improve changelog

* remove LastBlockTotalTx from state

* docs: remove getNumTxs from BeginBlock Java example
2019-11-14 13:56:12 +04:00
Phil Salant
bc572217c0 Fix linter errors thrown by lll (#3970)
* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter
2019-10-17 10:42:28 +02:00
Marko
7edc0601d5 New lint version upgrade (#4056)
* New lint version upgrade

- linter was upgraded

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* enable-a;; is deprecated

* minor change

* another try

* some more changes

* some more changes

* reenable prealloc

* add version till bot is fixed
2019-10-14 17:40:15 +02:00
Phil Salant
05075ea5b7 linters: modify code to pass maligned and interfacer (#3959)
* Fix maligned structs

* Fix interfacer errors

* Revert accidental go.mod and go.sum changes

* Revert P2PConfig struct maligned reorder

* Revert PeerRoundState struct maligned reordering

* Revert RoundState struct maligned reordering

* Reorder WSClient struct

* Revert accidental type change

* Clean up type change

* Clean up type changes

* Revert to types.ABCIApplicationServer in GRPCServer struct

* Revert maligned changes to BaseConfig struct

* Fix tests in io_test.go

* Fix client_test package tests

* Fix reactor tests in consensus package

* Fix new interfacer errors
2019-09-30 17:12:51 -07:00
Anton Kaliaev
4dfbaeb0c4 regenerate protobuf files with newer gogo version (#3998)
protoc 3.7.0
gogo v1.3.0
2019-09-19 11:41:57 +03:00