M. J. Fromberger and GitHub
b94470a6a4
mempool: ensure evicted transactions are removed from the cache ( #9000 )
...
In the original implementation transactions evicted for priority were also
removed from the cache. In addition, remove expired transactions from
the cache.
Related:
- Add Has method to cache implementations.
- Update tests to exercise this condition.
2022-07-14 06:51:54 -07:00
M. J. Fromberger and GitHub
3790968156
mempool: release lock during app connection flush ( #8984 )
...
This case is symmetric to what we did for CheckTx calls, where we release the
mempool mutex to ensure callbacks can fire during call setup. We also need
this behaviour for application flush, for the same reason: The caller holds the
lock by contract from the Mempool interface.
2022-07-12 10:28:51 -07:00
M. J. Fromberger and GitHub
9e64c95e56
mempool: reduce lock contention during CheckTx (cleanup) ( #8983 )
...
The way this was originally structured, we reacquired the lock after issuing
the initial ABCI CheckTx call, only to immediately release it. Restructure the
code so that this redundant acquire is no longer necessary.
2022-07-12 08:00:29 -07:00
M. J. Fromberger and GitHub
cb93d3b587
mempool: don't log message type mismatch in the default callback ( #8969 )
2022-07-11 18:06:49 -07:00
M. J. Fromberger and GitHub
f98de20f7e
p2p: ensure closed channels stop receiving service ( #8979 )
...
Once these channels are closed, we should not continue to service them, as they
will never again deliver nonzero values.
2022-07-11 16:34:05 -07:00
M. J. Fromberger and GitHub
451e697331
Update generated mocks after upgrade of Mockery v2. ( #8973 )
2022-07-11 09:18:36 -04:00
mergify[bot] and GitHub
e3292a48e3
p2p: simpler priority queue (backport #8929 ) ( #8956 )
2022-07-08 13:29:42 -04:00
1daf7b939d
p2p: make peer gossiping coinflip safer ( #8949 ) ( #8963 )
...
Closes #8948
(cherry picked from commit 61ce384d75garen@tychoish.com >
2022-07-08 12:32:12 -04:00
156c305b08
p2p: delete cruft ( #8958 ) ( #8959 )
...
I think the decision in #8806 is that we shouldn't do this yet, so I think it's best to just drop this.
(cherry picked from commit 636320f901garen@tychoish.com >
2022-07-08 09:59:57 -04:00
M. J. Fromberger and GitHub
bc49f66c35
Add more unit tests for the priority mempool. ( #8961 )
...
- Add a test for time-based (TTL) expiration.
- Add tests for eviction based on size and priority.
2022-07-07 14:56:34 -07:00
9b02094827
Fix unbounded heap growth in the priority mempool. ( #8944 )
...
The primary effect of this change is to simplify the implementation of the
priority mempool to eliminate an unbounded heap growth observed by Vega team
when it was enabled in their testnet. It updates and fixes #8775 .
The main body of this change is to remove the auxiliary indexing structures,
and use only the concurrent list structure (the same as the legacy mempool) to
maintain both gossip order and priority.
This means that operations that require priority information, such as block
updates and insert-time evictions, require a linear scan over the mempool.
This tradeoff greatly simplifies the code and eliminates the long-term heap
load, at the cost of some extra CPU and short-lived working memory during
CheckTx and Update calls.
Rough benchmark results:
- This PR:
BenchmarkTxMempool_CheckTx-10 486373 2271 ns/op
- Original priority mempool implementation:
BenchmarkTxMempool_CheckTx-10 500302 2113 ns/op
- Legacy (v0) mempool:
BenchmarkCheckTx-10 364591 3571 ns/op
These benchmarks are not a good proxy for production load, but at least suggest
that the overhead of the implementation changes are not cause for concern.
In addition:
- Rework synchronization so that access to shared data structures is safe.
Previously shared locks were used to exclude block updates during calls that
update mempool state. Now access is properly exclusive where necessary.
- Fix a bug in the recheck flow, where priority updates from the application
were not correctly reflected in the index structures.
- Eliminate the need for separate recheck cursors during block update. This
avoids the need to explicitly invalidate elements of the concurrent list,
which averts the dependency cycle that led to objects being pinned.
- Clean up, clarify, and fix inaccuracies in documentation comments throughout
the package.
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com >
2022-07-07 07:15:08 -07:00
da83edc588
p2p: return from conn send on stopped mconn ( #8904 )
...
Co-authored-by: Sam Kleinman <garen@tychoish.com >
2022-07-06 10:41:55 -04:00
047d7c927b
p2p: fix flakey test due to disconnect cooldown ( #8917 ) ( #8918 )
...
This test was made flakey by #8839 . The cooldown period means that the node in the test will not try to reconnect as quickly as the test expects. This change makes the cooldown shorter in the test so that the node quickly reconnects.
(cherry picked from commit 5274f80de44561443+williambanfield@users.noreply.github.com >
Co-authored-by: Sam Kleinman <garen@tychoish.com >
2022-07-05 19:11:38 -04:00
49788adde5
p2p: use correct context error ( #8916 ) ( #8920 )
...
handshakeCtx is the internal context carrying the timeout. Its error should be used for the error return.
(cherry picked from commit 921530c3524561443+williambanfield@users.noreply.github.com >
Co-authored-by: Sam Kleinman <garen@tychoish.com >
Co-authored-by: Callum Waters <cmwaters19@gmail.com >
2022-07-05 13:36:26 -04:00
William Banfield and GitHub
978f754ad3
p2p: set empty timeouts to configed values. (manual backport of #8847 ) ( #8869 )
...
* regenerate mocks using newer style
* p2p: set empty timeouts to small values. (#8847 )
These timeouts default to 'do not time out' if they are not set. This times up resources, potentially indefinitely. If node on the other side of the the handshake is up but unresponsive, the[ handshake call](https://github.com/tendermint/tendermint/blob/edec79448aa1d62b84683b1b22e12e145dbdda7c/internal/p2p/router.go#L720 ) will _never_ return.
* fix light client select statement
2022-06-28 16:07:15 -04:00
mergify[bot] and GitHub
c4ef566071
p2p: remove dial sleep and provide disconnect cooldown (backport #8839 ) ( #8875 )
...
(cherry picked from commit 52b6dc19ba
2022-06-27 10:49:51 -04:00
mergify[bot] and GitHub
826f224c2d
p2p: add eviction metrics and cleanup dialing error handling (backport #8819 ) ( #8820 )
2022-06-24 10:42:58 -04:00
6f4ef72964
p2p: track peers by address ( #8841 ) ( #8855 )
...
(cherry picked from commit 436a38f876garen@tychoish.com >
2022-06-23 13:21:46 -04:00
mergify[bot] and GitHub
24701cd587
p2p: more dial routines ( #8827 ) ( #8828 )
2022-06-21 21:27:28 -04:00
William Banfield and GitHub
e9c87a3c49
remove dial wake change ( #8824 )
2022-06-21 20:20:04 -04:00
Callum Waters and GitHub
4322f7d0b9
mempool: make error throwing for CheckTx consistent ( #8817 )
2022-06-21 18:51:50 +02:00
83526cacbc
p2p: peer store and dialing changes (0.35.x backport) ( #8740 )
...
* p2p: peer store and dialing changes
(cherry picked from commit 9dbb135152b213a2766fcc28ce298f56a91642dc86db59f53b000aa054854e2bc8f51ee3068b50b231bd396c88eddb23b5af4c8651026amichael.j.fromberger@gmail.com >
(cherry picked from commit f23f6e1089michael.j.fromberger@gmail.com >
(cherry picked from commit 1c02758eaf9f604fd2ef930fd7f2be9abc55f3a0720600ef628195c97590053ecd9b8cmichael.j.fromberger@gmail.com >
* comments
* fix lint
* cr feedback
* fixup cherrypick
* make wb happy
* more comments
* fixup
* fix lint
* iota fix
* add skip
* cleanup
* remove comment
* fix rand
* fix rand
* use numaddresses correctly
* advertise fixes
* remove some things
* cleanup comment
* more fixes
* toml
* fix comment
* fix spell
* dec limit
* fixes
* up the attmept max
* cr feedback
* probablistic test
* fix spell
* add metrics for peers stored on startup
* p2p: peer score should not wrap around (#8790 )
(cherry picked from commit 4d820ff4f5michael.j.fromberger@gmail.com >
2022-06-20 13:13:21 -04:00
mergify[bot] and GitHub
74c6d8100d
p2p: fix typo ( #8793 ) ( #8794 )
2022-06-19 11:52:43 -07:00
mergify[bot] and GitHub
ce8284c027
p2p: accept should not abort on first error (backport #8759 ) ( #8760 )
2022-06-15 07:56:15 -04:00
Callum Waters and GitHub
28c38522e0
do not log an error for duplicate txs ( #8732 )
2022-06-10 11:56:00 +02:00
af0590a819
consensus: switch timeout message to be debug and clarify meaning ( #8694 ) ( #8696 )
...
(cherry picked from commit 75a12ea0c64561443+williambanfield@users.noreply.github.com >
Co-authored-by: Sam Kleinman <garen@tychoish.com >
Co-authored-by: Callum Waters <cmwaters19@gmail.com >
2022-06-09 09:45:58 -04:00
mergify[bot] and GitHub
0e3a3fe58b
p2p: shed peers from store from other networks (backport #8678 ) ( #8681 )
2022-06-02 12:15:55 -04:00
Callum Waters and GitHub
e8ac37223f
pex: align max address thresholds ( #8657 )
2022-05-31 14:07:25 -04:00
Sam Kleinman and GitHub
a889f17e51
consensus: restructure peer catchup sleep ( #8651 )
2022-05-31 11:31:51 -04:00
mergify[bot] and GitHub
4ee91663da
p2p: reduce ability of SendError to disconnect peers (backport #8597 ) ( #8603 )
2022-05-25 04:12:43 -04:00
mergify[bot] and GitHub
2f8483aa85
p2p: remove unused get height methods (backport #8569 ) ( #8571 )
2022-05-17 11:32:13 -04:00
mergify[bot] and GitHub
12fed0ed53
blocksync: validate block before persisting it (backport #8493 ) ( #8496 )
2022-05-12 10:36:48 +02:00
Sam Kleinman and GitHub
bdd59c892c
statesync: avoid potential race ( #8494 )
2022-05-11 15:09:41 -04:00
mergify[bot] and GitHub
14f0d60f24
p2p: fix setting in con-tracker ( #8370 ) ( #8371 )
...
(cherry picked from commit 889341152a
2022-04-19 23:32:54 -07:00
mergify[bot] and GitHub
04c1f76569
rpc: avoid leaking threads during checktx (backport #8328 ) ( #8333 )
2022-04-17 09:17:03 -04:00
Ethan Reesor and GitHub
226bc94c5f
node: always close database engine ( #7113 ) ( #8330 )
2022-04-15 14:37:34 -07:00
William Banfield and GitHub
8579cc382e
invoke callbacks when set late in socket client (Forward-Port #8331 ) ( #8336 )
2022-04-14 18:36:09 -04:00
Callum Waters and GitHub
308d283241
cli: fix reset command for v0.35 ( #8259 )
2022-04-07 11:24:46 -07:00
William Banfield and GitHub
114a41f6cc
consensus: change lock handling in reactor and handleMsg for RoundState (forward-port #7994 #7992 ) ( #8138 )
...
This ports the changes from #7992 and #7994 from the v0.34.x branch to v0.35.x.
2022-03-18 12:27:21 -04:00
M. J. Fromberger and GitHub
a6fd04f1be
p2p: update polling interval calculation for PEX requests (backport #8106 ) ( #8118 )
...
A manual cherry-pick of 89b4321af2
2022-03-14 13:01:00 -07:00
M. J. Fromberger and GitHub
9b87606dee
consensus: avert a data race in round state access ( #8112 )
2022-03-11 11:16:19 -08:00
M. J. Fromberger and GitHub
5fb791e020
Fix data race. ( #8105 )
...
Original repro:
go test -run TestStateFullRound1 -race -count=500 ./internal/consensus
2022-03-10 14:29:13 -08:00
William Banfield and GitHub
7f85fc250a
consensus: start the timeout ticker before replay ( #7844 ) ( #8082 )
2022-03-08 20:53:42 -05:00
Sam Kleinman and GitHub
cc88a31a60
p2p: update shim to transfer information about peers ( #8047 )
2022-03-02 08:43:47 -05:00
mergify[bot] and GitHub
ca69da7533
statesync: avoid compounding retry logic for fetching consensus parameters (backport #8032 ) ( #8041 )
...
(cherry picked from commit a965f03c15
2022-03-01 18:59:29 -05:00
mergify[bot] and GitHub
0e08c66926
p2p: plumb rudamentary service discovery to rectors and update statesync (backport #8030 ) ( #8036 )
2022-02-28 20:51:47 -08:00
William Banfield and GitHub
ea497301a7
proxy: fix endblock metric ( #7989 )
2022-02-25 16:03:15 -08:00
mergify[bot] and GitHub
a281c0bbf1
p2p: retry failed connections slightly more aggressively (backport #8010 ) ( #8012 )
2022-02-25 13:53:46 -05:00
Sam Kleinman and GitHub
a0321633b0
p2p: backport changes in ping/pong tolerances ( #8009 )
2022-02-25 12:37:11 -05:00
mergify[bot] and GitHub
cf7e440e68
statesync: assert app version matches (backport #7856 ) ( #7886 )
2022-02-21 11:19:17 +01:00
M. J. FrombergerandGitHub
mergify[bot]andGitHub
Ethan ReesorandGitHub