Commit Graph

55 Commits

Author SHA1 Message Date
William Banfield
15f2436c28 p2p: pqueue proposal 2022-06-29 18:19:12 -04:00
William Banfield
978f754ad3 p2p: set empty timeouts to configed values. (manual backport of #8847) (#8869)
* regenerate mocks using newer style

* p2p: set empty timeouts to small values. (#8847)

These timeouts default to 'do not time out' if they are not set. This times up resources, potentially indefinitely. If node on the other side of the the handshake is up but unresponsive, the[ handshake call](edec79448a/internal/p2p/router.go (L720)) will _never_ return.

* fix light client select statement
2022-06-28 16:07:15 -04:00
mergify[bot]
c4ef566071 p2p: remove dial sleep and provide disconnect cooldown (backport #8839) (#8875)
(cherry picked from commit 52b6dc19ba)
2022-06-27 10:49:51 -04:00
mergify[bot]
826f224c2d p2p: add eviction metrics and cleanup dialing error handling (backport #8819) (#8820) 2022-06-24 10:42:58 -04:00
mergify[bot]
6f4ef72964 p2p: track peers by address (#8841) (#8855)
(cherry picked from commit 436a38f876)

Co-authored-by: Sam Kleinman <garen@tychoish.com>
2022-06-23 13:21:46 -04:00
mergify[bot]
24701cd587 p2p: more dial routines (#8827) (#8828) 2022-06-21 21:27:28 -04:00
William Banfield
e9c87a3c49 remove dial wake change (#8824) 2022-06-21 20:20:04 -04:00
Sam Kleinman
83526cacbc p2p: peer store and dialing changes (0.35.x backport) (#8740)
* p2p: peer store and dialing changes

(cherry picked from commit 9dbb135152)

* reduce persistent peer max

(cherry picked from commit b213a2766f)

* don't gossip inactive peers

(cherry picked from commit cc28ce298f)

* fix small case

(cherry picked from commit 56a91642dc)

* fix error message

(cherry picked from commit 86db59f53b)

* remove seed flag

(cherry picked from commit 000aa05485)

* reduce logging level

(cherry picked from commit 4e2bc8f51e)

* make const

(cherry picked from commit e3068b50b2)

* update comment

(cherry picked from commit 31bd396c88)

* cleanup

(cherry picked from commit eddb23b5af)

* oops

* overflows

(cherry picked from commit 4c8651026a)

* Update internal/p2p/peermanager.go

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
(cherry picked from commit f23f6e1089)

* Update internal/p2p/peermanager.go

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
(cherry picked from commit 1c02758eaf)

* comment

(cherry picked from commit 9f604fd2ef)

* test: new scoring

(cherry picked from commit 930fd7f2be)

* fix scoring test

(cherry picked from commit 9abc55f3a0)

* cleanup peer manager

* fix panic

* add metrics

* fix compile

* fix test

* default metrics to noop

* noop metrics

* update metrics

(cherry picked from commit 720600ef62)

* rename metrics

* actually shuffle peers more

* fix up advertise

(cherry picked from commit 8195c97590)

* add max dialing attempts

* connection tracking

* comments mostly

(cherry picked from commit 053ecd9b8c)

* Apply suggestions from code review

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>

* comments

* fix lint

* cr feedback

* fixup cherrypick

* make wb happy

* more comments

* fixup

* fix lint

* iota fix

* add skip

* cleanup

* remove comment

* fix rand

* fix rand

* use numaddresses correctly

* advertise fixes

* remove some things

* cleanup comment

* more fixes

* toml

* fix comment

* fix spell

* dec limit

* fixes

* up the attmept max

* cr feedback

* probablistic test

* fix spell

* add metrics for peers stored on startup

* p2p: peer score should not wrap around (#8790)

(cherry picked from commit 4d820ff4f5)

# Conflicts:
#	internal/p2p/peermanager.go

* fix

* wake more

* wake if we need to

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2022-06-20 13:13:21 -04:00
mergify[bot]
74c6d8100d p2p: fix typo (#8793) (#8794) 2022-06-19 11:52:43 -07:00
mergify[bot]
ce8284c027 p2p: accept should not abort on first error (backport #8759) (#8760) 2022-06-15 07:56:15 -04:00
mergify[bot]
0e3a3fe58b p2p: shed peers from store from other networks (backport #8678) (#8681) 2022-06-02 12:15:55 -04:00
Callum Waters
e8ac37223f pex: align max address thresholds (#8657) 2022-05-31 14:07:25 -04:00
mergify[bot]
4ee91663da p2p: reduce ability of SendError to disconnect peers (backport #8597) (#8603) 2022-05-25 04:12:43 -04:00
mergify[bot]
2f8483aa85 p2p: remove unused get height methods (backport #8569) (#8571) 2022-05-17 11:32:13 -04:00
mergify[bot]
14f0d60f24 p2p: fix setting in con-tracker (#8370) (#8371)
(cherry picked from commit 889341152a)
2022-04-19 23:32:54 -07:00
Callum Waters
308d283241 cli: fix reset command for v0.35 (#8259) 2022-04-07 11:24:46 -07:00
M. J. Fromberger
a6fd04f1be p2p: update polling interval calculation for PEX requests (backport #8106) (#8118)
A manual cherry-pick of 89b4321af2.
2022-03-14 13:01:00 -07:00
Sam Kleinman
cc88a31a60 p2p: update shim to transfer information about peers (#8047) 2022-03-02 08:43:47 -05:00
mergify[bot]
0e08c66926 p2p: plumb rudamentary service discovery to rectors and update statesync (backport #8030) (#8036) 2022-02-28 20:51:47 -08:00
mergify[bot]
a281c0bbf1 p2p: retry failed connections slightly more aggressively (backport #8010) (#8012) 2022-02-25 13:53:46 -05:00
Sam Kleinman
a0321633b0 p2p: backport changes in ping/pong tolerances (#8009) 2022-02-25 12:37:11 -05:00
Gui
4b565fe58a p2p: always advertise self, to enable mutual address discovery (#7620)
Fixes #7593
2022-01-19 13:46:30 -05:00
mergify[bot]
b2c4128ac0 build: Make sure to test packages with external tests (backport #7608) (#7635)
* build: Make sure to test packages with external tests (#7608)

The test filter was looking for "TestGoFiles", which does not include tests in
a separate package (e.g., "package foo_test" for "package foo").
This caused several packages not to be tested in CI, including:

  github.com/tendermint/tendermint/abci/client
  github.com/tendermint/tendermint/crypto
  github.com/tendermint/tendermint/crypto/tmhash
  github.com/tendermint/tendermint/internal/eventbus
  github.com/tendermint/tendermint/internal/evidence
  github.com/tendermint/tendermint/internal/inspect
  github.com/tendermint/tendermint/internal/jsontypes
  github.com/tendermint/tendermint/internal/libs/protoio
  github.com/tendermint/tendermint/internal/libs/sync
  github.com/tendermint/tendermint/internal/p2p/pex
  github.com/tendermint/tendermint/internal/pubsub
  github.com/tendermint/tendermint/internal/pubsub/query
  github.com/tendermint/tendermint/internal/pubsub/query/syntax
  github.com/tendermint/tendermint/internal/state/indexer
  github.com/tendermint/tendermint/internal/state/indexer/block/kv
  github.com/tendermint/tendermint/libs/json
  github.com/tendermint/tendermint/libs/log
  github.com/tendermint/tendermint/libs/os
  github.com/tendermint/tendermint/light
  github.com/tendermint/tendermint/light/provider/http
  github.com/tendermint/tendermint/privval/grpc
  github.com/tendermint/tendermint/proto/tendermint/blocksync
  github.com/tendermint/tendermint/proto/tendermint/consensus
  github.com/tendermint/tendermint/proto/tendermint/statesync
  github.com/tendermint/tendermint/rpc/client
  github.com/tendermint/tendermint/rpc/client/mock
  github.com/tendermint/tendermint/test/e2e/tests
  github.com/tendermint/tendermint/test/fuzz/mempool
  github.com/tendermint/tendermint/test/fuzz/p2p/secretconnection
  github.com/tendermint/tendermint/test/fuzz/rpc/jsonrpc/server

Updates #7626 and #7634.

(cherry picked from commit aea428d322)

* Fix build tags to satisfy Go 1.16.

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-01-18 21:36:33 -08:00
mergify[bot]
657afc7322 Remove now-unused nolint:lll directives. (backport #7356) (#7358)
(cherry picked from commit 76dea94a01)

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2021-11-30 17:01:51 -05:00
mergify[bot]
37e0779d6d p2p: reduce peer score for dial failures (backport #7265) (#7271)
* p2p: reduce peer score for dial failures (#7265)

When dialing fails to succeed we should reduce the score of the peer,
which puts the peer at (potentially) greater chances of being removed
from the peer manager, and reduces the chance of the peer being
gossiped by the PEX reactor.

(cherry picked from commit 27560cf7a4)

Co-authored-by: Sam Kleinman <garen@tychoish.com>
2021-11-10 11:10:06 -05:00
Sam Kleinman
003d15fa4b lint: cleanup branch lint errors (#7238) 2021-11-04 07:44:13 -04:00
Sam Kleinman
e40a8468a4 config: backport file writing changes (#7182) 2021-10-29 06:38:52 -04:00
M. J. Fromberger
85086d7452 Fix metric cardinality left over from backport (#7180)
One of the patched uses in #7161 missed the message type field,
triggering panic failures from Prometheus.
2021-10-28 15:29:53 -07:00
mergify[bot]
dd1471da91 p2p: add message type into the send/recv bytes metrics (backport #7155) (#7161)
* p2p: add message type into the send/recv bytes metrics (#7155)

This pull request adds a new "mesage_type" label to the send/recv bytes metrics calculated in the p2p code.

Below is a snippet of the updated metrics that includes the updated label:
```
tendermint_p2p_peer_receive_bytes_total{chID="32",chain_id="ci",message_type="consensus_HasVote",peer_id="2551a13ed720101b271a5df4816d1e4b3d3bd133"} 652
tendermint_p2p_peer_receive_bytes_total{chID="32",chain_id="ci",message_type="consensus_HasVote",peer_id="4b1068420ef739db63377250553562b9a978708a"} 631
tendermint_p2p_peer_receive_bytes_total{chID="32",chain_id="ci",message_type="consensus_HasVote",peer_id="927c50a5e508c747830ce3ba64a3f70fdda58ef2"} 631
tendermint_p2p_peer_receive_bytes_total{chID="32",chain_id="ci",message_type="consensus_NewRoundStep",peer_id="2551a13ed720101b271a5df4816d1e4b3d3bd133"} 393
tendermint_p2p_peer_receive_bytes_total{chID="32",chain_id="ci",message_type="consensus_NewRoundStep",peer_id="4b1068420ef739db63377250553562b9a978708a"} 357
tendermint_p2p_peer_receive_bytes_total{chID="32",chain_id="ci",message_type="consensus_NewRoundStep",peer_id="927c50a5e508c747830ce3ba64a3f70fdda58ef2"} 386
```

(cherry picked from commit b4bc6bb4e8)
2021-10-27 07:34:24 -04:00
M. J. Fromberger
bdd815ebc9 Align atomic struct field for compatibility in 32-bit ABIs. (#7037)
The layout of struct fields means that interior fields may not be properly
aligned for 64-bit access.

Fixes #7000.
2021-09-30 10:53:05 -07:00
lklimek
1bd1593f20 fix: race condition in p2p_switch and pex_reactor (#7015)
Closes https://github.com/tendermint/tendermint/issues/7014
2021-09-28 09:32:14 -04:00
M. J. Fromberger
cf7537ea5f cleanup: Reduce and normalize import path aliasing. (#6975)
The code in the Tendermint repository makes heavy use of import aliasing.
This is made necessary by our extensive reuse of common base package names, and
by repetition of similar names across different subdirectories.

Unfortunately we have not been very consistent about which packages we alias in
various circumstances, and the aliases we use vary. In the spirit of the advice
in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports,
his change makes an effort to clean up and normalize import aliasing.

This change makes no API or behavioral changes. It is a pure cleanup intended
o help make the code more readable to developers (including myself) trying to
understand what is being imported where.

Only unexported names have been modified, and the changes were generated and
applied mechanically with gofmt -r and comby, respecting the lexical and
syntactic rules of Go.  Even so, I did not fix every inconsistency. Where the
changes would be too disruptive, I left it alone.

The principles I followed in this cleanup are:

- Remove aliases that restate the package name.
- Remove aliases where the base package name is unambiguous.
- Move overly-terse abbreviations from the import to the usage site.
- Fix lexical issues (remove underscores, remove capitalization).
- Fix import groupings to more closely match the style guide.
- Group blank (side-effecting) imports and ensure they are commented.
- Add aliases to multiple imports with the same base package name.
2021-09-23 07:52:07 -07:00
William Banfield
63aeb50665 upgrading: add information into the UPGRADING.md for users of the codebase wishing to upgrade (#6898)
* add information on upgrading to the new p2p library

* clarify p2p backwards compatibility

* reorder p2p queue list

* add demo for p2p selection

* fix spacing in upgrading
2021-09-08 09:41:12 -04:00
William Banfield
4e96c6b234 tools: add mockery to tools.go and remove mockery version strings (#6787)
This change aims to keep versions of mockery consistent across developer laptops.

This change adds mockery to the `tools.go` file so that its version can be managed consistently in the `go.mod` file.

Additionally, this change temporarily disables adding mockery's version number to generated files. There is an outstanding issue against the mockery project related to the version string behavior when running from `go get`. I have created a pull request to fix this issue in the mockery project.
see: https://github.com/vektra/mockery/issues/397
2021-07-30 20:47:15 +00:00
Callum Waters
02f8e4c0bd blockstore: fix problem with seen commit (#6782) 2021-07-30 17:37:04 +02:00
M. J. Fromberger
6dd8984fef Fix and clarify breaks from select cases. (#6781)
Update those break statements inside case clauses that are intended to reach an
enclosing for loop, so that they correctly exit the loop.

The candidate files for this change were located using:

    % staticcheck -checks SA4011 ./... | cut -d: -f-2

This change is intended to preserve the intended semantics of the code, but
since the code as-written did not have its intended effect, some behaviour may
change. Specifically: Some loops may have run longer than they were supposed
to, prior to this change.

In one case I was not able to clearly determine the intended outcome. That case
has been commented but otherwise left as-written.

Fixes #6780.
2021-07-29 22:28:32 -04:00
William Banfield
a751eee7f2 p2p: add test for pqueue dequeue full error (#6760)
This adds a test for closing the `pqueue` while the `pqueue` contains data that has not yet been dequeued.

This issue was found while debugging #6705 

This test will fail until @cmwaters fix for this condition is merged.
2021-07-26 19:22:32 +00:00
Callum Waters
a341a626e0 p2p: avoid blocking on the dequeCh (#6765) 2021-07-26 09:09:07 -04:00
William Banfield
c3ae6f5b58 p2p: add coverage for mConnConnection.TrySendMessage (#6754)
This change adds additional coverage to the `mConnConnection.TrySendMessage` code path. Adds test to ensure it returns `io.EOF` when closed.

Addresses: #6570
2021-07-23 17:29:19 +00:00
William Banfield
f70396c6fd add and run make target for generating existing mocks (#6732)
There are many `//go:generate mockery` lines in the source code.
This change adds a make target to invoke these mock generations.
This change also invokes the mock invocations and adds the resulting mocks to the repo.

Related to #5274
2021-07-18 00:46:04 +00:00
Callum Waters
6dd0cf92c8 router/statesync: add helpful log messages (#6724) 2021-07-15 19:26:35 +02:00
Callum Waters
aa4854ff8f docs: add docs file for the peer exchange (#6665) 2021-07-12 14:11:29 +02:00
Callum Waters
decac693ab p2p: remove annoying error log (#6688)
I put this error log in here because I thought it might be a helpful indicator to see when a reactor sends a message to a peer that doesn't have that channel open but it turns out this is happening all the time and it's kind of annoying
2021-07-09 12:48:33 +00:00
Aleksandr Bezobchuk
1dec3e139a add stacktrace to panic logs (#6662) 2021-07-06 14:26:18 -04:00
Callum Waters
c769e3e09b p2p: track peer channels to avoid sending across a channel a peer doesn't have (#6601) 2021-07-05 13:00:19 +02:00
William Banfield
0cb0dd7ce9 p2p: do not redial peers with different chain id (#6630)
Updates the p2p logic to reject peers if the peer is not on the same chain.

Fixes #6245
2021-07-02 16:38:30 -04:00
Marko
0e9bec1b53 pkg: expose p2p functions (#6627)
## Description

Expose p2p functions for use in the sdk. 

These functions could also be copied over to the sdk. I dont have a preference of which is better.
2021-07-02 15:04:42 +00:00
Yawning Angel
c5cc3c8d3f crypto: Use a different library for ed25519/sr25519 (#6526)
At Oasis we have spend some time writing a new Ed25519/X25519/sr25519 implementation called curve25519-voi.  This PR switches the import from ed25519consensus/go-schnorrkel, which should lead to performance gains on most systems.

Summary of changes:
 * curve25519-voi is now used for Ed25519 operations, following the existing ZIP-215 semantics.
 * curve25519-voi's public key cache is enabled (hardcoded size of 4096 entries, should be tuned, see the code comment) to accelerate repeated Ed25519 verification with the same public key(s).
 * (BREAKING) curve25519-voi is now used for sr25519 operations.  This is a breaking change as the current sr25519 support does something decidedly non-standard when going from a MiniSecretKey to a SecretKey and or PublicKey (The expansion routine is called twice).  While I believe the new behavior (that expands once and only once) to be more "correct", this changes the semantics as implemented.
 * curve25519-voi is now used for merlin since the included STROBE implementation produces much less garbage on the heap.

Side issues fixed:
 * The version of go-schnorrkel that is currently imported by tendermint has a badly broken batch verification implementation.  Upstream has fixed the issue after I reported it, so the version should be bumped in the interim.

Open design questions/issues:
 * As noted, the public key cache size should be tuned.  It is currently backed by a trivial thread-safe LRU cache, which is not scan-resistant, but replacing it with something better is a matter of implementing an interface.
 * As far as I can tell, the only reason why serial verification on batch failure is necessary is to provide more detailed error messages (that are only used in some unit tests).  If you trust the batch verification to be consistent with serial verification then the fallback can be eliminated entirely (the BatchVerifier provided by the new library supports an option that omits the fallback if this is chosen as the way forward).
 * curve25519-voi's sr25519 support could use more optimization and more eyes on the code.  The algorithm unfortunately is woefully under-specified, and the implementation was done primarily because I got really sad when I actually looked at go-schnorrkel, and we do not use the algorithm at this time.
2021-06-26 16:53:30 +00:00
Sam Kleinman
917180dfd2 p2p: reduce buffering on channels (#6609)
Having smaller buffers in each reactor/channel will mean that there will be fewer stale messages.
2021-06-24 20:38:35 +00:00
Sam Kleinman
9ffa7e8a2b types: move NodeInfo from p2p (#6618) 2021-06-24 12:18:19 -04:00