Commit Graph

362 Commits

Author SHA1 Message Date
William Banfield
7b3feb1778 update uses 2022-04-11 12:04:17 -04:00
William Banfield
32098b4353 Merge branch 'master' into wb/issue-7950 2022-04-06 22:20:44 -04:00
William Banfield
143fc09ad7 add next_validators_hash to abci++ 2022-04-06 22:08:15 -04:00
William Banfield
8824867501 add proposer field to abci 2022-04-06 21:56:30 -04:00
Sam Kleinman
7678ab8850 statesync: tweak test performance (#8267) 2022-04-06 21:50:14 +00:00
Sam Kleinman
d153388446 p2p: inject nodeinfo into router (#8261) 2022-04-06 14:02:07 -04:00
Sam Kleinman
2304ea70f7 consensus: remove string indented function (#8257) 2022-04-06 08:07:21 -07:00
dependabot[bot]
0a23b1e51d build(deps): Bump github.com/vektra/mockery/v2 from 2.10.2 to 2.10.4 (#8250)
Bumps [github.com/vektra/mockery/v2](https://github.com/vektra/mockery) from 2.10.2 to 2.10.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/vektra/mockery/releases">github.com/vektra/mockery/v2's releases</a>.</em></p>
<blockquote>
<h2>v2.10.4</h2>
<h2>Changelog</h2>
<ul>
<li>c943e69 Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/441">#441</a> from cfstras/fix/support-more-env-keys</li>
<li>ed87cf6 fix: allow configuring flags with &quot;-&quot; as Env var</li>
<li>17abd96 fix: unused config field <code>Tags</code></li>
<li>53114cf test: add test for env var configurations</li>
</ul>
<h2>v2.10.3</h2>
<h2>Changelog</h2>
<ul>
<li>ee25bcf Add/update mocks</li>
<li>4703d1a Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/444">#444</a> from vektra/remove_need_deps</li>
<li>ba1f213 Remove packages.NeedDeps</li>
<li>ed38b20 Update go.sum</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="c943e696a0"><code>c943e69</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/441">#441</a> from cfstras/fix/support-more-env-keys</li>
<li><a href="4703d1a42f"><code>4703d1a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/444">#444</a> from vektra/remove_need_deps</li>
<li><a href="ed38b20a49"><code>ed38b20</code></a> Update go.sum</li>
<li><a href="ee25bcfcb5"><code>ee25bcf</code></a> Add/update mocks</li>
<li><a href="ba1f213e17"><code>ba1f213</code></a> Remove packages.NeedDeps</li>
<li><a href="17abd96c57"><code>17abd96</code></a> fix: unused config field <code>Tags</code></li>
<li><a href="53114cfce4"><code>53114cf</code></a> test: add test for env var configurations</li>
<li><a href="ed87cf671a"><code>ed87cf6</code></a> fix: allow configuring flags with &quot;-&quot; as Env var</li>
<li>See full diff in <a href="https://github.com/vektra/mockery/compare/v2.10.2...v2.10.4">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/vektra/mockery/v2&package-manager=go_modules&previous-version=2.10.2&new-version=2.10.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-04-05 15:47:24 +00:00
Sam Kleinman
9d1e8eaad4 node: remove channel and peer update initialization from construction (#8238) 2022-04-05 13:26:53 +00:00
Sam Kleinman
97f7021712 statesync: merge channel processing (#8240) 2022-04-04 12:31:15 -04:00
Sergio Mena
8df38db82e Remove ModifiedTxStatus from the spec and the code (#8210)
* Outstanding abci-gen changes to 'pb.go' files

* Removed modified_tx_status from spec and protobufs

* Fix sed for OSX

* Regenerated abci protobufs with 'abci-proto-gen'

* Code changes. UTs e2e tests passing

* Recovered UT: TestPrepareProposalModifiedTxStatusFalse

* Adapted UT

* Fixed UT

* Revert "Fix sed for OSX"

This reverts commit e576708c61.

* Update internal/state/execution_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update internal/state/execution_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed some comments

* Added one test that tests error at the ABCI client + Fixed some mock calls

* Addressed remaining comments

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed William's latest comments

* Adressed Michael's comment

* Fixed UT

* Some md fixes

* More md fixes

* gofmt

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-04-04 12:43:01 +02:00
Sam Kleinman
6af23ff757 state: avoid premature genericism (#8224) 2022-03-31 13:10:09 +00:00
William Banfield
686b7d36ac abci++: only include meaningful fields in app passed-through data 2022-03-29 16:58:03 -04:00
William Banfield
41a1bf539b timeout parameters take the default if not set (#8189) 2022-03-28 18:25:19 -04:00
William Banfield
462c475abc consensus: timeout params in toml used as overrides (#8186)
Replaces the set of timeout parameters in the config.toml file with unsafe-*override versions of the corresponding ConsensusParams.Timeout field. These fields can be used for the duration of v0.36 to override the consensus param in case of emergency.

Adds a set to the ./internal/consensus/State type for correctly calculating the value of each timeout based on the set of overrides specified.
2022-03-28 17:33:23 -04:00
William Banfield
e4ae922c33 consensus: update state machine to use the new consensus params (#8181) 2022-03-23 11:26:42 -04:00
Sam Kleinman
cbce877480 consensus: add leaktest check to replay tests (#8185) 2022-03-23 00:00:08 +00:00
Sam Kleinman
b29cc95920 consensus: cleanup tempfile explictly (#8184) 2022-03-22 17:41:00 -04:00
Sam Kleinman
0c9558a742 consensus: avoid panic during shutdown (#8170) 2022-03-22 15:28:33 -04:00
William Banfield
2361e0c65c abci++: remove CheckTx call from PrepareProposal flow (#8176) 2022-03-22 10:03:35 -04:00
William Banfield
cc838a5a19 ABCI++: Update new protos to use enum instead of bool (#8158)
closes: #8039 

This pull request updates the new ABCI++ protos to use `enum`s in place of `bool`s. `enums` may be preferred over `bool` because an `enum` can be udpated to include new statuses in the future, whereas a `bool` cannot and is fixed as just `true` or `false` over the whole lifecycle of the API.
2022-03-21 16:57:34 +00:00
Sam Kleinman
c33be0a410 state: propogate error from state store (#8171)
* state: propogate error from state store

* fix lint
2022-03-21 15:28:42 +00:00
Sam Kleinman
c680cca96e consensus: reduce size of test fixtures and logging rate (#8172)
We can reduce the size of test fixtures (which will improve test
reliability) without impacting these tests' primary role (which is
correctness.)

Also reducing these test logging will make the tests easier to read,
which whill be a good quality of life improvement for devs.
2022-03-21 15:07:52 +00:00
William Banfield
485c96b0d3 consensus: change lock handling in reactor and handleMsg for RoundState (forward-port #7994 #7992) (#8139)
Related to #8157
2022-03-18 18:59:44 +00:00
Sam Kleinman
9a833a8495 consensus: skip channel close during shutdown (#8155)
I see this panic in tests occasionally, and I don't think there's any
need to close this channel:

- it's only sent to in one place which has a select case with a
  default clause, so there's no chance of deadlocks.

- the only place we recieve from it thas a timeout.
2022-03-18 18:35:42 +00:00
Sam Kleinman
0bded371c5 testing: logger cleanup (#8153)
This contains two major changes:

- Remove the legacy test logging method, and just explicitly call the
  noop logger. This is just to make the test logging behavior more
  coherent and clear. 
  
- Move the logging in the light package from the testing.T logger to
  the noop logger. It's really the case that we very rarely need/want
  to consider test logs unless we're doing reproductions and running a
  narrow set of tests.
  
In most cases, I (for one) prefer to run in verbose mode so I can
watch progress of tests, but I basically never need to consider
logs. If I do want to see logs, then I can edit in the testing.T
logger locally (which is what you have to do today, anyway.)
2022-03-18 17:39:38 +00:00
Sam Kleinman
12d13cd31d mempool: reduce size of test (#8152)
This is failing intermittently, but it's a really simple test, and I
suspect that we're just running into thread scheduling issues on CI
nodes. I don't think making the test smaller reduces the utility of
this test.
2022-03-18 16:55:10 +00:00
William Banfield
bba8367aac state: panic on ResponsePrepareProposal validation error (#8145)
* state: panic on ResponsePrepareProposal validation error

* lint++

Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-18 16:42:29 +00:00
Sam Kleinman
f1a8f47d4d types: minor cleanup of un or minimally used types (#8154) 2022-03-18 16:27:10 +00:00
Sam Kleinman
f61e6e4201 autofile: remove vestigal close mechanism (#8150) 2022-03-18 12:05:53 -04:00
Sam Kleinman
1db41663c7 consensus: avoid race in accessing channel (#8149) 2022-03-18 09:54:40 -04:00
Sam Kleinman
5e0e05f938 consensus: avoid persistent kvstore in tests (#8148) 2022-03-18 09:39:06 -04:00
Sam Kleinman
5bb51aab03 events: remove service aspects of event switch (#8146) 2022-03-18 12:31:08 +00:00
Sam Kleinman
13f7501950 blocksync: remove intermediate channel (#8140)
Based on local testing, I'm now convinced that this is ok, and also I think the fact that the new p2p layer has more caching and queue.
2022-03-17 22:20:59 +00:00
JayT106
4400b0f6d3 p2p: adjust max non-persistent peer score (#8137)
Guarantee persistent peers have the highest connecting priority. 
The peerStore.Ranked returns an arbitrary order of peers with the same scores.
2022-03-17 14:30:45 -07:00
Sam Kleinman
a68e356596 consensus: avoid extra close channel (#8144)
Saw this in a test panic, doesn't seem neccessary.
2022-03-17 20:27:20 +00:00
William Banfield
7c91b53999 docs: PBTS synchrony issues runbook (#8129)
closes: #7756 

# What does this pull request change?
This pull request adds a new runbook for operators enountering errors related to the new Proposer-Based Timestamps algorithm. The goal of this runbook is to give operators a set of clear steps that they can follow if they are having issues producing blocks because of clock synchronization problems. 
This pull request also renames the `*PrevoteDelay` metrics to drop the term `MessageDelay`. These metrics provide a combined view of `message_delay` + `synchrony` so the name may be confusing.
# Questions to reviewers
* Are there ways to make the set of steps clearer or are there any pieces that seem confusing?
2022-03-17 19:20:46 +00:00
Sam Kleinman
1dd8807cc3 mempool: test harness should expose application (#8143)
This is minor, but I was trying to write a test and realized that the
application reference in the harness isn't actually used, which is
quite confusing.
2022-03-17 17:45:27 +00:00
Sam Kleinman
07b46d5a05 blocksync: drop redundant shutdown mechanisms (#8136) 2022-03-17 13:30:13 -04:00
Sam Kleinman
7a0b05f22d libs/events: remove unneccessary unsubscription code (#8135)
The events switch code is largely vestigal and is responsible for
wiring between the consensus state machine and the consensus
reactor. While there might have been a need, historicallly to managed
these subscriptions at runtime, it's nolonger used: subscriptions are
registered during startup, and then the switch shuts down at at the
end. 

Eventually the EventSwitch should be replaced by a much smaller
implementation of an eventloop in the consensus state machine, but
cutting down on the scope of the event switch will help clarify the
requirements from the consensus side.
2022-03-17 13:02:02 +00:00
Sam Kleinman
bedb68078c libs/clist: remove unused surface area (#8134) 2022-03-16 11:57:41 -04:00
Sam Kleinman
48b1952f18 state: avoid panics for marshaling errors (#8125) 2022-03-15 22:27:23 +00:00
William Banfield
68c624f5de abci++: synchronize PrepareProposal with the newest version of the spec (#8094)
This change implements the logic for the PrepareProposal ABCI++ method call. The main logic for creating and issuing the PrepareProposal request lives in execution.go and is tested in a set of new tests in execution_test.go. This change also updates the mempool mock to use a mockery generated version and removes much of the plumbing for the no longer used ABCIResponses.
2022-03-15 15:37:30 -04:00
Sam Kleinman
faf123bda2 autofile: reduce minor panic and docs changes (#8122)
* autofile: reduce minor panic and docs changes

* fix lint
2022-03-15 14:25:25 +00:00
Sam Kleinman
da5c09cf6f cleanup: remove commented code (#8123) 2022-03-15 10:07:06 -04:00
Sam Kleinman
b08dd93d88 libs/log: remove Must constructor (#8120)
* libs/log: remove Must constructor

* Update test/e2e/node/main.go

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>

* use stdlog

Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2022-03-15 13:02:31 +00:00
M. J. Fromberger
e9bc33d807 consensus: ensure the node terminates on consensus failure (#8111)
Updates #8077. The panic handler for consensus currently attempts to effect a
clean shutdown, but this can leave a failed node running in an unknown state
for an arbitrary amount of time after the failure.

Since a panic at this point means consensus is already irrecoverably broken, we
should not allow the node to continue executing. After making a best effort to
shut down the writeahead log, re-panic to ensure the node will terminate before
any further state transitions are processed.

Even with this change, it is possible some transitions may occur while the
cleanup is happening. It might be preferable to abort unconditionally without
any attempt at cleanup.

Related changes:

- Clean up the creation of WAL directories.
- Filter WAL close errors at rethrow.
2022-03-11 13:30:15 -08:00
M. J. Fromberger
658a7661c5 p2p: remove unnecessary panic handling in PEX reactor (#8110)
The message handling in this reactor is all under control of the reactor
itself, and does not call out to callbacks or other externally-supplied code.
It doesn't need to check for panics.

- Remove an irrelevant channel ID check.
- Remove an unnecessary panic recovery wrapper.
2022-03-11 08:23:03 -08:00
M. J. Fromberger
89b4321af2 p2p: update polling interval calculation for PEX requests (#8106)
The PEX reactor has a simple feedback control mechanism to decide how often to
poll peers for peer address updates. The idea is to poll more frequently when
knowledge of the network is less, and decrease frequency as knowledge grows.

This change solves two problems:

1. It is possible in some cases we may poll a peer "too often" and get dropped
   by that peer for spamming.

2. The first successful peer update with any content resets the polling timer
   to a very long time (10m), meaning if we are unlucky in getting an
   incomplete reply while the network is small, we may not try again for a very
   long time. This may contribute to difficulties bootstrapping sync.

The main change here is to only update the interval when new information is
added to the system, and not (as before) whenever a request is sent out to a
peer. The rate computation is essentially the same as before, although the code
has been a bit simplified, and I consolidated some of the error handling so
that we don't have to check in multiple places for the same conditions.

Related changes:

- Improve error diagnostics for too-soon and overflow conditions.
- Clean up state handling in the poll interval computation.
- Pin the minimum interval avert a chance of PEX spamming a peer.
2022-03-11 07:49:33 -08:00
JayT106
d9c9675e2a p2p+flowrate: rate control refactor (#7828)
Adding `CurrentTransferRate ` in the flowrate package because only the status of the transfer rate has been used.
2022-03-10 13:48:23 +00:00