Commit Graph

146 Commits

Author SHA1 Message Date
Callum Waters
ebff8a96a5 docs: use hyphens instead of snake case (#5802) 2020-12-17 08:59:58 +01:00
Erik Grinaker
e198edf20e p2p: remove NodeInfo interface and rename DefaultNodeInfo struct (#5799)
The `NodeInfo` interface does not appear to serve any purpose at all, so I removed it and renamed the `DefaultNodeInfo` struct to `NodeInfo` (including the Protobuf representations). Let me know if this is actually needed for anything.

Only the Protobuf rename is listed in the changelog, since we do not officially support API stability of the `p2p` package (according to `README.md`). The on-wire protocol remains compatible.
2020-12-15 18:54:25 +00:00
Erik Grinaker
bcfc889f25 p2p: implement new Transport interface (#5791)
This implements a new `Transport` interface and related types for the P2P refactor in #5670. Previously, `conn.MConnection` was very tightly coupled to the `Peer` implementation -- in order to allow alternative non-multiplexed transports (e.g. QUIC), MConnection has now been moved below the `Transport` interface, as `MConnTransport`, and decoupled from the peer. Since the `p2p` package is not covered by our Go API stability, this is not considered a breaking change, and not listed in the changelog.

The initial approach was to implement the new interface in its final form (which also involved possible protocol changes, see https://github.com/tendermint/spec/pull/227). However, it turned out that this would require a large amount of changes to existing P2P code because of the previous tight coupling between `Peer` and `MConnection` and the reliance on subtleties in the MConnection behavior. Instead, I have broadened the `Transport` interface to expose much of the existing MConnection interface, preserved much of the existing MConnection logic and behavior in the transport implementation, and tried to make as few changes to the rest of the P2P stack as possible. We will instead reduce this interface gradually as we refactor other parts of the P2P stack.

The low-level transport code and protocol (e.g. MConnection, SecretConnection and so on) has not been significantly changed, and refactoring this is not a priority until we come up with a plan for QUIC adoption, as we may end up discarding the MConnection code entirely.

There are no tests of the new `MConnTransport`, as this code is likely to evolve as we proceed with the P2P refactor, but tests should be added before a final release. The E2E tests are sufficient for basic validation in the meanwhile.
2020-12-15 15:08:16 +00:00
Callum Waters
3283a84ab2 cmd: hyphen case cli and config (#5777) 2020-12-11 13:14:04 +01:00
Anton Kaliaev
28e79a4d02 cmd: modify gen_node_key to print key to STDOUT (#5772)
closes: #5770
closes: #5769

also, include node ID in the output (#5769) and modify NodeKey to use
value semantics (it makes perfect sense for NodeKey to not be a
pointer).
2020-12-10 11:02:35 +04:00
Aleksandr Bezobchuk
a879eb444d p2p: state sync reactor refactor (#5671) 2020-12-09 09:31:06 -05:00
Tess Rinearson
79890d8393 reactors: omit incoming message bytes from reactor logs (#5743)
After a reactor has failed to parse an incoming message, it shouldn't output the "bad" data into the logs, as that data is unfiltered and could have anything in it. (We also don't think this information is helpful to have in the logs anyways.)
2020-12-03 22:12:08 +00:00
Anton Kaliaev
243ff4b43d blockchain/v1: remove in favor of v2 (#5728) 2020-12-03 09:35:47 +04:00
Anton Kaliaev
b1bbd37519 libs/bits: validate BitArray in FromProto (#5720)
Closes #5705
2020-12-01 12:44:56 +00:00
Anton Kaliaev
e13b4386ff abci: modify Client interface and socket client (#5673)
`abci.Client`:

- Sync and Async methods now accept a context for cancellation
    * grpc client uses context to cancel both Sync and Async requests
    * local client ignores context parameter
    * socket client uses context to cancel Sync requests and to drop Async requests before sending them if context was cancelled prior to that

- Async methods return an error
    * socket client returns an error immediately if queue is full for Async requests
    * local client always returns nil error
    * grpc client returns an error if context was cancelled before we got response or the receiving queue had a space for response (do not confuse with the sending queue from the socket client)

- specify clients semantics in [doc.go](https://raw.githubusercontent.com/tendermint/tendermint/27112fffa62276bc016d56741f686f0f77931748/abci/client/doc.go)

`mempool.TxInfo`

- add optional `Context` to `TxInfo`, which can be used to cancel `CheckTx` request

Closes #5190
2020-11-30 16:46:16 +04:00
Anton Kaliaev
170cb70e19 test/e2e: enable v1 and v2 blockchains (#5702)
* test/e2e: enable v1 and v2 blockchains

* modify networks/ci.toml
2020-11-23 13:32:22 +00:00
Marko
b2d72dce7e e2e: use ed25519 for secretConn (remote signer) (#5678)
## Description

Hardcode ed25519 to dialTCPFn in e2e tests. 

I will backport `DefaultRequestHandler` fixes

This will be replaced when grpc is implemented.
2020-11-17 13:55:23 +00:00
Marko
e0950515ff test/e2e: fix secp failures (#5649) 2020-11-16 12:31:32 +01:00
Marko
95cff1efb4 proto: buf for everything (#5650)
## Description

- remove installation of protoc
- use buf protoc to generate proto stubs

prior to approving could someone test locally. I restarted my docker instance and its been stuck for 20+ minutes

Closes: #XXX
2020-11-12 11:15:18 +00:00
Anton Kaliaev
7041cae8e1 bump go version to 1.15 (#5639) 2020-11-10 15:19:52 +04:00
Marko
bf35cc6443 cmd: add support for --key (#5612)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-11-09 15:22:36 +01:00
Callum Waters
9fe7b4fe77 remove misbehaviors from e2e generator (#5629) 2020-11-06 14:27:48 +01:00
Callum Waters
3922dde05d evidence: structs can independently form abci evidence (#5610) 2020-11-04 17:14:48 +01:00
Erik Grinaker
53022220f6 test: fix various E2E test issues (#5576)
* Don't use state sync for nodes starting at initial height.
* Also remove stopped containers when cleaning up.
* Start nodes in order of startAt, mode, name to avoid full nodes starting before their seeds.
* Tweak network waiting to avoid halts caused by validator changes and perturbations.
* Disable most tests for seed nodes, which aren't always able to join consensus.
* Disable `blockchain/v2` due to known bugs.
2020-10-27 16:22:00 +00:00
Erik Grinaker
7daf6a1a03 test: disable E2E misbehaviors due to bugs (#5569)
Disables misbehaviors in E2E testnets due to failures caused by #5554 and #5560. Should be re-enabled once these are fixed.
2020-10-26 10:28:11 +00:00
Erik Grinaker
10dda219a1 test: fix handling of start height in generated E2E testnets (#5563)
In #5488 the E2E testnet generator changed to setting explicit `StartAt` heights for initial nodes. This broke the runner, which expected all initial nodes to have `StartAt: 0`, as well as validator set scheduling in the generator. Testnet loading now normalizes initial nodes to have `StartAt: 0`.

This also tweaks waiting for misbehavior heights to only use an additional wait if there actually is any misbehavior in the testnet, and to output information when waiting.
2020-10-26 10:09:46 +00:00
Callum Waters
50b91867c3 test: add evidence e2e tests (#5488) 2020-10-23 12:33:08 +02:00
Erik Grinaker
d11e5993b1 test: tag E2E Docker resources and autoremove them (#5558)
Fixes #5555.
2020-10-23 08:17:15 +00:00
Erik Grinaker
e0e006d10f test: run remaining E2E testnets on run-multiple.sh failure (#5557)
Fixes #5542.
2020-10-22 20:53:27 +00:00
Erik Grinaker
8c5fe166a6 test: enable restart/kill perturbations in E2E tests (#5537)
When #5536 lands we can re-enable restart/kill perturbations in E2E tests.
2020-10-20 19:30:07 +00:00
Erik Grinaker
9e6248c0d7 test: enable blockchain v2 in E2E testnet generator (#5533)
When #5499 and #5530 land, we can re-enable v2 in the E2E testnet generator (and thus the nightly E2E tests).
2020-10-20 11:44:22 +00:00
Erik Grinaker
3dabfbeae0 test: enable ABCI gRPC client in E2E testnets (#5521)
Once #5520 lands, we can re-enable gRPC ABCI protocol in the E2E testnets.
2020-10-20 08:27:58 +00:00
Erik Grinaker
f0d4ddcf3c test: tweak E2E tests for nightly runs (#5512) 2020-10-16 12:10:29 +02:00
Erik Grinaker
b6979e7fbd test: clean up E2E test volumes using a container (#5509) 2020-10-15 20:59:16 +02:00
Erik Grinaker
8a67968416 github: add nightly E2E testnet action (#5480) 2020-10-14 16:55:02 +02:00
Erik Grinaker
260cc5dd69 test/e2e: add random testnet generator (#5479)
Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
2020-10-09 12:33:48 +00:00
Erik Grinaker
8d28e7467c test: add E2E test for node peering (#5465)
This was a missing test case from the old P2P tests removed in #5453, which makes sure that all nodes are able to peer with each other regardless of how they discover peers.

Fixes #2795, since the default CI testnet uses a combination of (partially meshed) persistent peers and PEX-based seed nodes.
2020-10-06 06:00:20 +00:00
Erik Grinaker
b894f07380 test: remove P2P tests (#5453) 2020-10-05 17:52:01 +02:00
Erik Grinaker
7e27e9b852 test: add GitHub action for end-to-end tests (#5452)
Partial fix for #5291.
2020-10-05 12:44:35 +02:00
Erik Grinaker
090afe30f9 test: add basic end-to-end test cases (#5450)
Partial fix for #5291.

This adds a basic set of test cases for core network invariants. Although small, it is sufficient to replace and extend the current set of P2P tests. Further test cases can be added later.
2020-10-05 10:23:12 +00:00
Erik Grinaker
250c3aa92e test: add end-to-end testing framework (#5435)
Partial fix for #5291. For details, see [README.md](https://github.com/tendermint/tendermint/blob/erik/e2e-tests/test/e2e/README.md) and [RFC-001](https://github.com/tendermint/tendermint/blob/master/docs/rfc/rfc-001-end-to-end-testing.md).

This only includes a single test case under `test/e2e/tests/`, as a proof of concept - additional test cases will be submitted separately. A randomized testnet generator will also be submitted separately, there a currently just a handful of static testnets under `test/e2e/networks/`. This will eventually replace the current P2P tests and run in CI.
2020-10-05 09:35:01 +00:00
dongsam
e30b125725 consensus: double-sign risk reduction (ADR-51) (#5147)
Implementation spec of Double Signing Risk Reduction [ADR-51](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-051-double-signing-risk-reduction.md) by B-Harvest
- Add `DoubleSignCheckHeight` config variable to ConsensusConfig for "How many blocks looks back to check existence of the node's consensus votes when before joining consensus"
- Add `consensus.double_sign_check_height` to `config.toml` and `tendermint node` as flag for set `DoubleSignCheckHeight`
- Set default `consensus.double_sign_check_height` to `0`  ( it could be adjustable in this PR, disable when 0  )

Refs

- [ADR-51](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-051-double-signing-risk-reduction.md)
- [https://github.com/tendermint/tendermint/issues/4059](https://github.com/tendermint/tendermint/issues/4059)
- [https://github.com/tendermint/tendermint/pull/4262](https://github.com/tendermint/tendermint/pull/4262)
2020-08-27 08:57:36 +04:00
Marko
42e4e8b58e lint: add markdown linter (#5254) 2020-08-17 16:40:50 +02:00
Marko
66003d7bd1 bump go version (#5070)
## Description

since 0.34 is breaking we should bump the go version to 1.14

Closes: #XXX
2020-07-01 08:36:16 +00:00
Erik Grinaker
db8f1b3df3 migrate all JSON to new JSON encoder (#4975)
Uses new JSON encoder in #4955 for all JSON. Branched off of #4968.
2020-06-08 12:22:59 +00:00
Erik Grinaker
d28b47ced7 test: revert Go 1.13→1.14 bump
Since we support 1.13 and above, let's use that as a baseline for tests.
2020-05-11 14:10:31 +00:00
Erik Grinaker
960c7402fe test: fix p2p test build breakage caused by Debian testing
Debian testing caused Docker image build failures:

```
The following packages have unmet dependencies:
 
 libc6-dev : Breaks: libgcc-8-dev (< 8.4.0-2~) but 8.3.0-6 is to be installed
 
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
```

It does not appear that we actually need testing, so removing it.
2020-05-11 12:42:11 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
Erik Grinaker
c63fd32fc2 circleci: run P2P IPv4 and IPv6 tests in parallel (#4459) 2020-02-21 15:23:48 +01:00
Erik Grinaker
48be9bcb09 Add IPv6 support for P2P integration tests (#4340) 2020-01-27 10:40:54 +01:00
Marko
3e2751d274 lint: Enable Golint (#4212)
* Fix many golint errors

* Fix golint errors in the 'lite' package

* Don't export Pool.store

* Fix typo

* Revert unwanted changes

* Fix errors in counter package

* Fix linter errors in kvstore package

* Fix linter error in example package

* Fix error in tests package

* Fix linter errors in v2 package

* Fix linter errors in consensus package

* Fix linter errors in evidence package

* Fix linter error in fail package

* Fix linter errors in query package

* Fix linter errors in core package

* Fix linter errors in node package

* Fix linter errors in mempool package

* Fix linter error in conn package

* Fix linter errors in pex package

* Rename PEXReactor export to Reactor

* Fix linter errors in trust package

* Fix linter errors in upnp package

* Fix linter errors in p2p package

* Fix linter errors in proxy package

* Fix linter errors in mock_test package

* Fix linter error in client_test package

* Fix linter errors in coretypes package

* Fix linter errors in coregrpc package

* Fix linter errors in rpcserver package

* Fix linter errors in rpctypes package

* Fix linter errors in rpctest package

* Fix linter error in json2wal script

* Fix linter error in wal2json script

* Fix linter errors in kv package

* Fix linter error in state package

* Fix linter error in grpc_client

* Fix linter errors in types package

* Fix linter error in version package

* Fix remaining errors

* Address review comments

* Fix broken tests

* Reconcile package coregrpc

* Fix golangci bot error

* Fix new golint errors

* Fix broken reference

* Enable golint linter

* minor changes to bring golint into line

* fix failing test

* fix pex reactor naming

* address PR comments
2019-12-05 10:12:08 +01:00
Marko
21d46dea93 Update Makefile (#3949)
* Update makefile

- these two files were taken from the sdk with adjustments for usage in TM

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* .phony

* remove slate

* more tools

* circleci update 1/2

* fixed typo

* fixed bash error

* '<<' must be escaped in 2.1

* fixed invalid config

* removed golangci-lint from tools. runs as github app now

* one more issue in Circle config

* some more work on the Circle config

* minor changes

* fiddling with restore cache config

* fiddling with restore cache config

* added commands in circle config. makefile changes

* bash shenannigans

* bash shenannigans

* fighting the config demons

* fighting the config demons, v2

* fighting the config demons, v3

* updating p2p dockerfile

* updating p2p dockerfile

* updating p2p test

* updating p2p test

* testing circle config

* testing circle config

* remove dontcover command

* its the weekend, custom docker image
2019-09-13 18:52:35 +02:00
Thane Thomson
d88a639838 Make RPC bind to localhost by default (#3746)
* Make RPC bind to localhost by default

* Add CHANGELOG_PENDING entry

* Allow testnet command to override RPC listen address

* Update localnet test to bind RPC to 0.0.0.0

* Update p2p test to bind RPC to 0.0.0.0

* Remove rpc-laddr parameter

* Update localnet to use config template with RPC listen address override

* Use config template override method for RPC listen address

* Build config template into localnode image

* Build localnode image locally before starting localnet

* Move testnet config overrides into templates

* Revert deletion of config overrides

* Remove extraneous config parameter overrides
2019-06-24 10:32:12 -04:00
Jack Zampolin
8b7ca8fd99 switch to go mod (#3613)
* Update to using go mod from dep

* Remove references to make get_vendor_deps

* Specify go version

* Set GO111MODULE=on and add -mod=readonly

* Fix exported env

* switch to using go1.12 everywhere

* Fix test scripts

* Typo:

* Prepend GO111MODULE=on

* remove dep cache

* Revert "remove dep cache"

This reverts commit 45117bda

Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* bring back the dependency cache and change it to cache modules instead
of vendored deps; also:

 - bump version for dependency cache
 - bump version on pkg-cache (includes modules directory)

Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* remove some more traces of dep:
 - remove Gopkg.(toml | lock)
 - update contributing guidlines
 - set global default in circleci (GO111MODULE=on)

Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* global var failed for `test_cover` with
`go: unknown environment setting GO111MODULE=true`
although the var was `GO111MODULE: on`

Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* Changelog pending entry

Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* Add bbolt dependency to go.mod

Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* move -mod=readonly to build flags
2019-06-09 16:27:48 +04:00
Anton Kaliaev
fb91ef7462 validate reactor messages (#2711)
* validate reactor messages

Refs #2683

* validate blockchain messages

Refs #2683

* validate evidence messages

Refs #2683

* todo

* check ProposalPOL and signature sizes

* add a changelog entry

* check addr is valid when we add it to the addrbook

* validate incoming netAddr (not just nil check!)

* fixes after Bucky's review

* check timestamps

* beef up block#ValidateBasic

* move some checks into bcBlockResponseMessage

* update Gopkg.lock

Fix

```
grouped write of manifest, lock and vendor: failed to export github.com/tendermint/go-amino: fatal: failed to unpack tree object 6dcc6ddc143e116455c94b25c1004c99e0d0ca12
```

by running `dep ensure -update`

* bump year since now we check it

* generate test/p2p/data on the fly using tendermint testnet

* allow sync chains older than 1 year

* use full path when creating a testnet

* move testnet gen to test/docker/Dockerfile

* relax LastCommitRound check

Refs #2737

* fix conflicts after merge

* add small comment

* some ValidateBasic updates

* fixes

* AppHash length is not fixed
2018-11-01 02:07:18 -04:00