Commit Graph

398 Commits

Author SHA1 Message Date
Erik Grinaker
cd19471c49 bump ABCI version to 0.16.2 due to ResponseCommit.retain_height
See #4588 for original change.

I believe this is appropriate. Anything else that needs to be updated?

______

For contributor use:

- [ ] ~Wrote tests~
- [x] Updated CHANGELOG_PENDING.md
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] ~Updated relevant documentation (`docs/`) and code comments~
- [x] Re-reviewed `Files changed` in the Github PR explorer
2020-04-20 08:15:56 +00:00
Alessio Treglia
fcbce21534 cli: add command to generate shell completion scripts (#4665)
How to use it:

```
$ . <(tendermint completion)
```

Note that the completion command does not show up in the help screen,
though it comes with its own --help option.

This is a port of the feature provided by cosmos-sdk.
2020-04-13 16:08:23 +04:00
Anton Kaliaev
4f88071324 update changelog and version 2020-04-09 19:32:00 +02:00
Callum Waters
88d7007c3c check evidence hasn't already been stored (#4632)
Add Has function, create better handling of errors when adding evidence, usage of error types.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-04-07 08:42:14 +00:00
Callum Waters
6aa469d008 p2p: PEX message abuse should ban as well as disconnect (#4621)
* mark unsolicited and too frequent messaged as bad

* add tests

* update changelog and fix error

* revised error types

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-04-07 04:26:13 +00:00
Erik Grinaker
4298bbcc4e add support for block pruning via ABCI Commit response (#4588)
* Added BlockStore.DeleteBlock()

* Added initial block pruner prototype

* wip

* Added BlockStore.PruneBlocks()

* Added consensus setting for block pruning

* Added BlockStore base

* Error on replay if base does not have blocks

* Handle missing blocks when sending VoteSetMaj23Message

* Error message tweak

* Properly update blockstore state

* Error message fix again

* blockchain: ignore peer missing blocks

* Added FIXME

* Added test for block replay with truncated history

* Handle peer base in blockchain reactor

* Improved replay error handling

* Added tests for Store.PruneBlocks()

* Fix non-RPC handling of truncated block history

* Panic on missing block meta in needProofBlock()

* Updated changelog

* Handle truncated block history in RPC layer

* Added info about earliest block in /status RPC

* Reorder height and base in blockchain reactor messages

* Updated changelog

* Fix tests

* Appease linter

* Minor review fixes

* Non-empty BlockStores should always have base > 0

* Update code to assume base > 0 invariant

* Added blockstore tests for pruning to 0

* Make sure we don't prune below the current base

* Added BlockStore.Size()

* config: added retain_blocks recommendations

* Update v1 blockchain reactor to handle blockstore base

* Added state database pruning

* Propagate errors on missing validator sets

* Comment tweaks

* Improved error message

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* use ABCI field ResponseCommit.retain_height instead of retain-blocks config option

* remove State.RetainHeight, return value instead

* fix minor issues

* rename pruneHeights() to pruneBlocks()

* noop to fix GitHub borkage

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-04-03 08:38:32 +00:00
Erik Grinaker
fdf9c7ae64 rpc/client: split out client packages (#4628)
* rpc/client: initial split into directories

* lite2: split out test package

* rpc/client: simplify client constructurs

* updated docs

* updated changelog
2020-04-02 13:25:30 +00:00
Anton Kaliaev
6c88d2ba1f lite2: make maxClockDrift an option (#4616)
Closes #4607
2020-03-31 16:33:33 +04:00
Callum Waters
5c380cdacb lite2: use bisection for some of backward verification (#4575)
Closes: #4537

Uses SignedHeaderBefore to find header before unverified header and then bisection to verify the header. Only when header is between first and last trusted header height else if before the first trusted header height then regular backwards verification is used.
2020-03-31 16:20:22 +04:00
Yongwoo Lee
bcf27c2364 rpc: add codespace to ResultBroadcastTx (#4611)
Closes: #4606
2020-03-31 11:49:28 +04:00
Marko
f5651d2b61 proto: use docker to generate stubs (#4615)
* proto: use docker to generate stubs

- provide an option to developers to use docker to generate proto stubs

closes #4579

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
2020-03-30 16:48:48 +02:00
Tess Rinearson
ccf53a2276 all: name reactors when they are initialized (#4608)
Previously, many reactors were initialized with the name "Reactor," which made it difficult to log which reactor was doing what. This changes those reactors' names to something more descriptive.
2020-03-27 18:30:42 +01:00
Callum Waters
a25faed5f0 lite2: cache headers in bisection (#4562)
Closes: #4546

The algorithm uses an array to store the headers and validators and populates it at every bisection (which is an unsuccessful verification). When a successful verification finally occurs it updates the new trusted header, trims that header from the cache (the array) and sets the depth pointer back to 0. Instead of retrieving new headers it will use the cached headers, incrementing in depth until it reaches the end of the cache which by then it will start to retrieve new headers from the provider. 

Mathematically, this method doesn't properly bisect after the first round but it will always choose a pivot header that is within 1/8th of the upper header's height. I.e. if we are trying to jump 128 headers, the maximum offset from bisection height (64) is 64 + 16(128/8) = 80, therefore a better heuristic would be to obtain the new pivot header height as the middle of these two numbers which would therefore mean to multiply it by 9/16ths instead of 1/2  (sorry this might be a bit more complicated in writing but I can try better explain if someone is interested). Therefore I would also, upon consensus, propose that we change the pivot height to 9/16th's of the previous height
2020-03-26 12:38:02 +01:00
Greg Szabo
c05b2f2384 Dockerfile updated with defaults (#4577) 2020-03-17 10:31:55 +00:00
Anton Kaliaev
c917c2ddfb rpc: fix panic when Subscribe is called (#4570)
but HTTP client is not running.

`Subscribe`, `Unsubscribe(All)` methods return an error now.

Closes #4568
2020-03-16 22:02:54 +04:00
Callum
586d299991 moved to changelog_pending 2020-03-12 18:39:13 +01:00
Marko
48f073d796 privval: return error on getpubkey (#4534)
closes #3602

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-03-12 11:10:36 +04:00
Marko
c534291a56 rc1/v0.33.2(#4541)
* rc1/v0.33.2

- release candiadte for 0.33.2

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-03-11 11:57:42 +01:00
Tess Rinearson
31bea92d12 libs/kv: remove unused type KI64Pair (#4542) 2020-03-10 21:34:44 +01:00
Anton Kaliaev
bc89aad162 rpc: create buffered subscriptions on /subscribe (#4521)
Closes #3935
2020-03-06 17:23:44 +04:00
Anton Kaliaev
891afc2235 cmd: fix debug kill and change debug dump archive filename format (#4517)
* cmd: change archive filename format

before: Mar  3 08:44:43.zip
after: 2020-03-03T08:44:43Z.zip

Refs #4515

* cmd: kill pid, not the debug process itself

Fixes #4515

* update changelog

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-03-03 12:41:58 +00:00
Erik Grinaker
c33576e72a example/kvstore: return ABCI query height (#4509)
* example/kvstore: return ABCI query height

* CHANGELOG: added PR link

* Updated tests
2020-03-01 20:03:28 +00:00
Michael FIG
3f18e22c96 rpc: keep the original subscription "id" field when new RPCs come in (#4493) 2020-02-29 22:22:19 +01:00
Alessio Treglia
25d92d05f8 types: VerifyCommitX return when +2/3 sigs are verified (#4445)
Closes #4417
2020-02-25 10:04:24 +01:00
Erik Grinaker
b09cdaf1af rpc: fix tx_search pagination with ordered results (#4437) 2020-02-19 14:01:42 +01:00
Anthony
5ea1ff94d1 rpc: fix issue with multiple subscriptions (#4406)
Using the WebSocket server, when the same client calls multiple time the subscribe method, only the last subscription receives all the events of the previous ones.

example:

    subscription1 = tm.event = 'NewBlock'
    subscription2 = tm.event = 'Tx'

In this case, subscription2 will receive the new blocks but subscription1 will not.

This came from the WebSocket handler that had the declaration of the rpcrequest moved and so overridden for every request and given in the JSONReq client context (so the id of the subscription was not the right one).

This fixes the issue by simply declaring the rpcrequest inside the loop so every request will create a new object without overwriting the previous one.
2020-02-17 08:00:56 +01:00
Marko
fb5751d2c5 release: minor release 0.33.1 (#4401)
* release: minor release 0.33.1

- minor release for 0.33.1

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* remvoe wording

* version bump
2020-02-13 16:38:00 +01:00
Marko
31fd99a91a proto: add buf and protogen script (#4369)
* proto: add buf and protogen script

- add buf with minimal changes
- add protogen script to easier generate proto files

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add protoc needs

* add some needed shell cmds

* remove buf from tools as it is not needed everytime

* add proto lint and breakage to ci

* add section in changelog and upgrading files

* address pr comments

* remove space in circle config

* remove spaces in makefile comment

* add section on contributing on how to work with proto

* bump buf to 0.7

* test bufbuild image

* test install make in bufbuild image

* revert to tendermintdev image

* Update Makefile

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-02-11 10:31:15 +01:00
Anton Kaliaev
587ac3a563 node: use GRPCMaxOpenConnections when creating the gRPC server (#4349)
not MaxOpenConnections

Fixes #4311

Also, set MaxBodyBytes, MaxHeaderBytes and WriteTimeout similar to HTTP
server.
2020-01-29 10:04:42 +04:00
Anton Kaliaev
d90dc9db26 rpc: add sort_order option to tx_search (#4342)
I have added order_by which can be "asc" or "desc" (should be in string format) in the tx_search RPC method.

Fixes: #3333

Author: @princesinha19
2020-01-27 21:20:56 +04:00
Marko
b50cb26664 rpc: check nil blockmeta (#4320)
* rpc: check nil blockmeta

- fixes #4319

- check if block meta is nil

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry
2020-01-17 18:32:31 +01:00
Marko
5aa209a156 Rc1 0.33 (#4306)
* rc for 0.33

* address some pr comments

* some adds to upgrade.md

* undo version
2020-01-15 12:43:13 +01:00
Marko
6b71b928be dep: update tm-db to 0.4.0 (#4289)
* dep: update tm-db to 0.4.0

- update 0.4.0 as it is a breaking change and cannot be handled by depndabot

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

*  more work towards error handling

* error and emtpy bytes handling

* work on tests

* add changelog entry, change some error handling

* address some pr comments

* panic in a few more places

* move error higher up

* redo some error handling

* fix some bz == nil to len(bz) == 0

* change statebytes
2020-01-13 18:45:16 +01:00
Anton Kaliaev
faf783331d cmd/lite: switch to new lite2 package (#4300)
* cmd/lite: switch to new lite2 package

* update changelog

* shorten the trusting period explanation
2020-01-13 14:49:53 +04:00
Greg Szabo
15aa8d82b3 rpc/client: add basic authentication (#4291)
Closes #4248
2020-01-09 13:42:30 +04:00
Marko
6d91c1faf4 evidence: introduce time.Duration to evidence params (#4254)
* evidence: introduce time.Duration to evidence params

- add time.duration to evidence
- this pr is taking pr #2606 and updating it to use both time and height

- closes #2565

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing and genesis cfg in signer harness

* remove debugging fmt

* change maxageheight to maxagenumblocks, rename other things to block instead of height

* further check of duration

* check duration to not send peers outdated evidence

* change some lines, onward and upward

* refactor evidence package

* add a changelog pending entry

* make mockbadevidence have time and use it

* add what could possibly be called a test case

* remove mockbadevidence and mockgoodevidence in favor of mockevidence

* add a comment for err that is returned

* add a changelog for removal of good & bad evidence

* add a test for adding evidence

* fix test

* add ev to types in testcase

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix tests

* fix linting

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-01-08 10:46:37 +01:00
Peter Mrekaj
8f5d58f32e rpc: modify New* functions to return error (#4274)
The New* client functions return an error instead
of panicking when the remote address is invalid.

Fixes #3953
2020-01-07 10:07:03 +04:00
Prince Sinha
7be74c73b7 rpc: add method block_by_hash (#4257)
* added RPC method block_by_hash

* created hash => height

* changes typings

* block_by_hash changes

* Update store/store.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update store/store.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update store/store.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update store/store.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-01-06 18:47:18 +04:00
Anton Kaliaev
7f655d8e9e rpc: pass outCapacity to eventBus#Subscribe when subscribing using a l… (#4279)
* pass `outCapacity` to `eventBus#Subscribe` when subscribing using a local client

Fixes #4256

* use outCap directly
2020-01-03 12:10:36 +04:00
Marko
7368ba358b prometheus/metrics: three new metrics for consensus (#4263)
* prometheus/metrics: two new metrics for consensus

- add consensus_validator_power metric so if a node is a validator it can see its own power in prometheus
- add last_signed_height metric so if a node is a validator it can be aware of at which height the most recent time it signed.

- closes: #3773
- closes: #3083

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* check if signature is present

* minor change and check if sig is not absent

* add changelog entry

* Update consensus/state.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update CHANGELOG_PENDING.md

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update CHANGELOG_PENDING.md

* add label with validator address

* change address to vali_address

* add metric missed blocks

* add changelog entry for missed blocks metric

* address naming & add docs
2019-12-19 18:04:57 +01:00
Marko
9bd0f9e634 lint: golint issue fixes (#4258)
* lint: golint issue fixes

- on my local machine golint is a lot stricter than the bot so slowly going through and fixing things.

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* more fixes from golint

* remove isPeerPersistentFn

* add changelog entry
2019-12-17 13:02:45 +01:00
Prince Sinha
f0777277ad rpc: added proposer in consensus_state (#4250)
* added proposer in consensus_state

* added proposer in consensus_state

* proposer changes

* added proposer in consensus_state

* proposer added
2019-12-16 15:47:36 +01:00
Alexander Bezobchuk
4bfec797e8 cli: debug sub-command (#4227)
## Issue

Implement a new subcommand: tendermint debug. This subcommand itself has two subcommands:

    $ tendermint debug kill <pid> </path/to/out.zip> --home=</path/to/app.d>

Writes debug info into a compressed archive. The archive will contain the following:

├── config.toml
├── consensus_state.json
├── net_info.json
├── stacktrace.out
├── status.json
└── wal

The Tendermint process will be killed.

    $ tendermint debug dump </path/to/out> --home=</path/to/app.d>

This command will perform similar to kill except it only polls the node and dumps debugging data every frequency seconds to a compressed archive under a given destination directory. Each archive will contain:

├── consensus_state.json
├── goroutine.out
├── heap.out
├── net_info.json
├── status.json
└── wal

Note:

    goroutine.out and heap.out will only be written if a profile address is provided and is operational.
    This command is blocking and will log any error.

replaces: #3327
closes: #3249

## Commits:

* Implement debug tool command stubs

* Implement net getters and zip logic

* Update zip dir API and add home flag

* Add simple godocs for kill aux functions

* Move IO util to new file and implement copy WAL func

* Implement copy config function

* Implement killProc

* Remove debug fmt

* Validate output file input

* Direct STDERR to file

* Godoc updates

* Sleep prior to killing tail proc

* Minor cleanup of godocs

* Move debug command and add make target

* Rename command handler function

* Add example to command long descr

* Move current implementation to cmd/tendermint/commands/debug

* Update kill cmd long description

* Implement dump command

* Add pending log entry

* Add gosec nolint

* Add error check for Mkdir

* Add os.IsNotExist(err)

* Add to debugging section in running-in-prod doc
2019-12-13 12:46:23 +04:00
Cuong Manh Le
c9708105f0 consensus/types: fix BenchmarkRoundStateDeepCopy panics (#4244)
Fixes #4243
2019-12-12 17:32:33 +01:00
Marko
7b52f51700 libs/common: Refactor libs/common 5 (#4240)
* libs/common: Refactor libs/common 5

- move mathematical functions and types out of `libs/common` to math pkg
- move net functions out of `libs/common` to net pkg
- move string functions out of `libs/common` to strings pkg
- move async functions out of `libs/common` to async pkg
- move bit functions out of `libs/common` to bits pkg
- move cmap functions out of `libs/common` to cmap pkg
- move os functions out of `libs/common` to os pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing issues

* fix tests

closes #41417

woooooooooooooooooo kill the cmn pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix goimport issues

* run gofmt
2019-12-12 09:33:27 +01:00
Marko
89f0bbbd76 libs/common: Refactor libs/common 4 (#4237)
* libs/common: Refactor libs/common 4

- move byte function out of cmn to its own pkg
- move tempfile out of cmn to its own pkg
- move throttletimer to its own pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix linting issues
2019-12-11 23:16:35 +01:00
Anton Kaliaev
15e80d2448 config: add rocksdb as a db backend option (#4239)
Closes #3831

The support for rocksdb was added a while back in
https://github.com/tendermint/tm-db/pull/12. This commit merely updates
the documentation.
2019-12-11 21:15:27 +04:00
Anton Kaliaev
4da816a849 cli: add optional --genesis_hash flag to check genesis hash upon startup (#4238)
Closes #4113
2019-12-11 15:48:46 +04:00
Prince Sinha
2b30f02879 cli: add --db_backend and --db_dir flags to tendermint node cmd (#4235)
Fixes #4234
2019-12-11 12:59:57 +04:00
Marko
27b00cf8d1 libs/common: refactor libs common 3 (#4232)
* libs/common: refactor libs common 3

- move nil.go into types folder and make private
- move service & baseservice out of common into service pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry
2019-12-11 09:31:25 +01:00