Commit Graph

274 Commits

Author SHA1 Message Date
Sam Kleinman
4e5c2b5e8f consensus: use delivertxsync (#7616) 2022-01-19 16:58:12 -05:00
Gui
ebbc3f02f5 p2p: always advertise self, to enable mutual address discovery (#7594)
Fixes #7593
2022-01-19 21:39:59 +00:00
M. J. Fromberger
c8e8a62084 abci/client: simplify client interface (#7607)
This change has two main effects:

1. Remove most of the Async methods from the abci.Client interface.
   Remaining are FlushAsync, CommitTxAsync, and DeliverTxAsync.

2. Rename the synchronous methods to remove the "Sync" suffix.

The rest of the change is updating the implementations, subsets, and mocks of
the interface, along with the call sites that point to them.

* Fix stringly-typed mock stubs.
* Rename helper method.
2022-01-19 10:58:56 -08:00
M. J. Fromberger
68d4fed236 consensus/state: avert a data race with state update and tests (#7643) 2022-01-19 10:47:06 -08:00
M. J. Fromberger
aea428d322 build: Make sure to test packages with external tests (#7608)
The test filter was looking for "TestGoFiles", which does not include tests in
a separate package (e.g., "package foo_test" for "package foo").
This caused several packages not to be tested in CI, including:

  github.com/tendermint/tendermint/abci/client
  github.com/tendermint/tendermint/crypto
  github.com/tendermint/tendermint/crypto/tmhash
  github.com/tendermint/tendermint/internal/eventbus
  github.com/tendermint/tendermint/internal/evidence
  github.com/tendermint/tendermint/internal/inspect
  github.com/tendermint/tendermint/internal/jsontypes
  github.com/tendermint/tendermint/internal/libs/protoio
  github.com/tendermint/tendermint/internal/libs/sync
  github.com/tendermint/tendermint/internal/p2p/pex
  github.com/tendermint/tendermint/internal/pubsub
  github.com/tendermint/tendermint/internal/pubsub/query
  github.com/tendermint/tendermint/internal/pubsub/query/syntax
  github.com/tendermint/tendermint/internal/state/indexer
  github.com/tendermint/tendermint/internal/state/indexer/block/kv
  github.com/tendermint/tendermint/libs/json
  github.com/tendermint/tendermint/libs/log
  github.com/tendermint/tendermint/libs/os
  github.com/tendermint/tendermint/light
  github.com/tendermint/tendermint/light/provider/http
  github.com/tendermint/tendermint/privval/grpc
  github.com/tendermint/tendermint/proto/tendermint/blocksync
  github.com/tendermint/tendermint/proto/tendermint/consensus
  github.com/tendermint/tendermint/proto/tendermint/statesync
  github.com/tendermint/tendermint/rpc/client
  github.com/tendermint/tendermint/rpc/client/mock
  github.com/tendermint/tendermint/test/e2e/tests
  github.com/tendermint/tendermint/test/fuzz/mempool
  github.com/tendermint/tendermint/test/fuzz/p2p/secretconnection
  github.com/tendermint/tendermint/test/fuzz/rpc/jsonrpc/server

Updates #7626 and #7634.
2022-01-18 18:36:46 -08:00
William Banfield
b6307c42e0 consensus: check proposal non-nil in prevote message delay metric (#7625) 2022-01-18 19:57:00 -05:00
M. J. Fromberger
a7eb95065d autofile: ensure files are not reopened after closing (#7628)
During file rotation and WAL shutdown, there was a race condition between users
of an autofile and its termination. To fix this, ensure operations on an
autofile are properly synchronized, and report errors when attempting to use an
autofile after it was closed.

Notably:

- Simplify the cancellation protocol between signal and Close.
- Exclude writers to an autofile during rotation.
- Add documentation about what is going on.

There is a lot more that could be improved here, but this addresses the more
obvious races that have been panicking unit tests.
2022-01-18 14:57:20 -08:00
M. J. Fromberger
5cca45bb45 pex: improve handling of closed channels (#7623)
Reverts and improves on #7622. The problem turns out not to be on the PEX
channel side, but on the pass-through (Go) channel.
2022-01-18 14:32:22 -08:00
M. J. Fromberger
417166704a pex: do not send nil envelopes to the reactor (#7622) 2022-01-18 12:01:04 -08:00
M. J. Fromberger
7fd97bf44b pex: avert a data race on map access in the reactor (#7614)
There was a path on which computing the next delivery time did not hold the
lock, defying the admonition on its comment.
2022-01-18 07:22:50 -08:00
William Banfield
0c82ceaa5f consensus: calculate prevote message delay metric (#7551)
## What does this pull request do?
This pull requests adds two metrics intended for use in calculating an experimental value for `MessageDelay`.

The metrics are as follows:
```
# HELP tendermint_consensus_complete_prevote_message_delay Difference in seconds between the proposal timestamp and the timestamp of the prevote that achieved 100% of the voting power in the prevote step.
# TYPE tendermint_consensus_complete_prevote_message_delay gauge
tendermint_consensus_complete_prevote_message_delay{chain_id="test-chain-aZbwF1"} 0.013025505

# HELP tendermint_consensus_quorum_prevote_message_delay Difference in seconds between the proposal timestamp and the timestamp of the prevote that achieved a quorum in the prevote step.
# TYPE tendermint_consensus_quorum_prevote_message_delay gauge
tendermint_consensus_quorum_prevote_message_delay{chain_id="test-chain-aZbwF1"} 0.013025505
```

## Why this change?

 For more information on what these metrics are calculating, see #7202. The aim is to merge to backport these metrics to v0.34 and run nodes on a few popular chains with these metrics to determine the experimental values for `MessageDelay` on these popular chains and use these to select our default `SynchronyParams.MessageDelay` value.

## Why Gauges for the metrics?
Gauges allow us to overwrite the metric on each successive observation. We can then capture these metrics over time to track the highest and lowest observed value.
2022-01-18 14:55:18 +00:00
Sam Kleinman
c0b56e207a consensus: test shutdown to avoid hangs (#7603) 2022-01-18 08:55:13 -05:00
Kene
49153b753c rpc: paginate mempool /unconfirmed_txs endpoint (#7612)
This commit changes the behaviour of the /unconfirmed_txs endpoint by replacing limit with a page and perPage parameter for pagination.
The test case for unconfirmed_txs have been accommodated to properly test this change and the documentation for the API as well.
2022-01-18 10:58:32 +01:00
M. J. Fromberger
679b6a65b8 light: fix provider error plumbing (#7610)
The custom error types in the provider package did not propagate their wrapped
underlying reasons, making it difficult for the test to check that the correct
error was observed.

- Fix the custom errors to have a true underlying error (not just a string).
- Add Unwrap methods to support inspection by errors.Is.
- Update usage in a few places.
- Fix the test to check for acceptable variation.

Fixes #7609.
2022-01-16 13:48:21 -08:00
M. J. Fromberger
c24f003b55 protoio: fix incorrect test assertion (#7606)
After writing and then reading a bunch of random messages, the test was
checking that it did not read the same number of messages that it wrote.
The sense of this check was inverted; they should match.

Introduced by accident in #7522. I'm not sure why this did not show up in CI.

Edit: I now know why it didn't show up in ci: #7608.
2022-01-16 20:39:37 +00:00
M. J. Fromberger
dbe2146d0a rpc: simplify the encoding of interface-typed arguments in JSON (#7600)
Add package jsontypes that implements a subset of the custom libs/json 
package. Specifically it handles encoding and decoding of interface types
wrapped in "tagged" JSON objects. It omits the deep reflection on arbitrary
types, preserving only the handling of type tags wrapper encoding.

- Register interface types (Evidence, PubKey, PrivKey) for tagged encoding.
- Update the existing implementations to satisfy the type.
- Register those types with the jsontypes registry.
- Add string tags to 64-bit integer fields where needed.
- Add marshalers to structs that export interface-typed fields.
2022-01-14 18:14:09 -08:00
Sam Kleinman
7ed57ef5f9 statesync: more orderly dispatcher shutdown (#7601) 2022-01-14 16:34:12 -05:00
Sam Kleinman
82b65868ce node+autofile: avoid leaks detected during WAL shutdown (#7599) 2022-01-14 13:04:01 -05:00
Sam Kleinman
7e8fa4ed85 consensus: explicit test timeout (#7585) 2022-01-13 16:11:51 -05:00
M. J. Fromberger
b7c19a5cd4 rpc: clean up the RPCFunc constructor signature (#7586)
Instead of taking a comma-separated string of parameter names, take each
parameter name as a separate argument. Now that we no longer have an extra flag
for caching, this fits nicely into a variadic trailer.

* Update all usage of NewRPCFunc and NewWSRPCFunc.
2022-01-13 12:13:28 -08:00
M. J. Fromberger
81ee41228a rpc: consolidate RPC route map construction (#7582)
Define interfaces for the various methods a service may implement.  This is
basically just the set of things on Environment that are exported as RPCs, but
these are also implemented by the light proxy.

* internal/rpc: use NewRoutesMap to construct routes on service start
* light/proxy: use NewRoutesMap to construct RPC routes
2022-01-13 10:45:36 -08:00
Sam Kleinman
cef17e1c02 node+rpc: rpc environment should own it's creation (#7573) 2022-01-13 12:39:48 -05:00
Sam Kleinman
fd2eccbae1 consensus: use noop logger for WAL test (#7580) 2022-01-13 12:05:12 -05:00
Sam Kleinman
2a348cc1e9 logging: remove reamining instances of SetLogger interface (#7572) 2022-01-12 16:56:49 -05:00
Sam Kleinman
25e665df17 internal/libs: delete unused functionality (#7569) 2022-01-12 15:55:42 -05:00
Sam Kleinman
e07c4cdcf2 node: collapse initialization internals (#7567) 2022-01-12 15:32:22 -05:00
Sam Kleinman
6efdba8aa9 statesync: SyncAny test buffering (#7570) 2022-01-12 13:38:23 -05:00
M. J. Fromberger
1f5e64e5b6 rpc: remove cache control settings from the HTTP server (#7568)
We should not set cache-control headers on RPC responses. HTTP caching
interacts poorly with resources that are expected to change frequently, or
whose rate of change is unpredictable.

More subtly, all calls to the POST endpoint use the same URL, which means a
cacheable response from one call may actually "hide" an uncacheable response
from a subsequent one. This is less of a problem for the GET endpoints, but
that means the behaviour of RPCs varies depending on which HTTP method your
client happens to use. Websocket requests were already marked statically
uncacheable, adding yet a third combination.

To address this:

- Stop setting cache-control headers.
- Update the tests that were checking for those headers.
- Remove the flags to request cache-control.

Apart from affecting the HTTP response headers, this change does not modify the
behaviour of any of the RPC methods.
2022-01-12 18:20:59 +00:00
Sam Kleinman
fb10d1c705 statesync: clarify test cleanup (#7565) 2022-01-12 12:57:23 -05:00
Sam Kleinman
cc51bf7587 tests: remove in-test logging (#7558) 2022-01-11 16:39:31 -05:00
M. J. Fromberger
50ac52e28d rpc: replace custom context-like argument with context.Context (#7559)
* Rename rpctypes.Context to CallInfo.

Add methods to attach and recover this value from a context.Context.

* Rework RPC method handlers to accept "real" contexts.

- Replace *rpctypes.Context arguments with context.Context.
- Update usage of RPC context fields to use CallInfo.
2022-01-11 11:47:56 -08:00
Sam Kleinman
5bf1bdcfb4 reactors: skip log on some routine cancels (#7556) 2022-01-11 12:56:52 -05:00
Sam Kleinman
d331a08607 statesync: use specific testing.T logger for tests (#7543) 2022-01-10 15:38:20 -05:00
M. J. Fromberger
366ab1947a Replace uses of libs/json with encoding/json. (#7534)
Where possible, replace uses of the custom JSON library with the standard
library. The custom library treats interface and unnamed lteral types
differently, so this change avoids those even where it would probably be safe
to switch them.
2022-01-08 08:47:26 -08:00
Sam Kleinman
d5c39f907d test/factory: pass testing.T around rather than errors for test fixtures (#7518) 2022-01-07 15:51:39 -05:00
Sam Kleinman
90cf742065 pex: regularize reactor constructor (#7532) 2022-01-07 13:52:11 -05:00
Sam Kleinman
aa76a367e0 blocksync: standardize construction process (#7531) 2022-01-07 13:40:08 -05:00
Sam Kleinman
ae7a76a175 evidence: reactor constructor (#7533) 2022-01-07 13:28:23 -05:00
Sam Kleinman
059b38afe4 mempool: refactor mempool constructor (#7530) 2022-01-07 12:49:22 -05:00
Sam Kleinman
fc36c7782f statesync: reactor and channel construction (#7529) 2022-01-07 12:04:17 -05:00
Sam Kleinman
10402b728f consensus+p2p: change how consensus reactor is constructed (#7525) 2022-01-07 10:59:10 -05:00
Sam Kleinman
74a8941854 consensus: remove reactor options (#7526) 2022-01-07 09:15:35 -05:00
Sam Kleinman
569629486b tests: remove panics from test fixtures (#7522) 2022-01-06 17:34:32 -05:00
Sam Kleinman
332163ede6 testing: remove background contexts (#7509) 2022-01-05 12:42:57 -05:00
Sam Kleinman
f2cc496f09 testing: pass testing.T to assert and require always, assertion cleanup (#7508) 2022-01-05 09:25:08 -05:00
Sam Kleinman
3c8955e4b8 errors: formating cleanup (#7507) 2022-01-04 16:11:28 -05:00
Sam Kleinman
430817d9e9 types: remove panic from block methods (#7501) 2022-01-03 16:14:26 -05:00
Sam Kleinman
1630d1cf3e privval: remove panics in privval implementation (#7475) 2021-12-17 16:28:32 -05:00
Sam Kleinman
bef120dadf contexts: remove all TODO instances (#7466) 2021-12-16 15:15:26 -05:00
Callum Waters
184b105509 statesync: assert app version matches (#7463) 2021-12-16 18:25:26 +01:00