Commit Graph
221 Commits
Author SHA1 Message Date
M. J. FrombergerandGitHub 68d4fed236 consensus/state: avert a data race with state update and tests (#7643) 2022-01-19 10:47:06 -08:00
M. J. FrombergerandGitHub aea428d322 build: Make sure to test packages with external tests (#7608)
The test filter was looking for "TestGoFiles", which does not include tests in
a separate package (e.g., "package foo_test" for "package foo").
This caused several packages not to be tested in CI, including:

  github.com/tendermint/tendermint/abci/client
  github.com/tendermint/tendermint/crypto
  github.com/tendermint/tendermint/crypto/tmhash
  github.com/tendermint/tendermint/internal/eventbus
  github.com/tendermint/tendermint/internal/evidence
  github.com/tendermint/tendermint/internal/inspect
  github.com/tendermint/tendermint/internal/jsontypes
  github.com/tendermint/tendermint/internal/libs/protoio
  github.com/tendermint/tendermint/internal/libs/sync
  github.com/tendermint/tendermint/internal/p2p/pex
  github.com/tendermint/tendermint/internal/pubsub
  github.com/tendermint/tendermint/internal/pubsub/query
  github.com/tendermint/tendermint/internal/pubsub/query/syntax
  github.com/tendermint/tendermint/internal/state/indexer
  github.com/tendermint/tendermint/internal/state/indexer/block/kv
  github.com/tendermint/tendermint/libs/json
  github.com/tendermint/tendermint/libs/log
  github.com/tendermint/tendermint/libs/os
  github.com/tendermint/tendermint/light
  github.com/tendermint/tendermint/light/provider/http
  github.com/tendermint/tendermint/privval/grpc
  github.com/tendermint/tendermint/proto/tendermint/blocksync
  github.com/tendermint/tendermint/proto/tendermint/consensus
  github.com/tendermint/tendermint/proto/tendermint/statesync
  github.com/tendermint/tendermint/rpc/client
  github.com/tendermint/tendermint/rpc/client/mock
  github.com/tendermint/tendermint/test/e2e/tests
  github.com/tendermint/tendermint/test/fuzz/mempool
  github.com/tendermint/tendermint/test/fuzz/p2p/secretconnection
  github.com/tendermint/tendermint/test/fuzz/rpc/jsonrpc/server

Updates #7626 and #7634.
2022-01-18 18:36:46 -08:00
William BanfieldandGitHub b6307c42e0 consensus: check proposal non-nil in prevote message delay metric (#7625) 2022-01-18 19:57:00 -05:00
M. J. FrombergerandGitHub a7eb95065d autofile: ensure files are not reopened after closing (#7628)
During file rotation and WAL shutdown, there was a race condition between users
of an autofile and its termination. To fix this, ensure operations on an
autofile are properly synchronized, and report errors when attempting to use an
autofile after it was closed.

Notably:

- Simplify the cancellation protocol between signal and Close.
- Exclude writers to an autofile during rotation.
- Add documentation about what is going on.

There is a lot more that could be improved here, but this addresses the more
obvious races that have been panicking unit tests.
2022-01-18 14:57:20 -08:00
M. J. FrombergerandGitHub 5cca45bb45 pex: improve handling of closed channels (#7623)
Reverts and improves on #7622. The problem turns out not to be on the PEX
channel side, but on the pass-through (Go) channel.
2022-01-18 14:32:22 -08:00
M. J. FrombergerandGitHub 417166704a pex: do not send nil envelopes to the reactor (#7622) 2022-01-18 12:01:04 -08:00
M. J. FrombergerandGitHub 7fd97bf44b pex: avert a data race on map access in the reactor (#7614)
There was a path on which computing the next delivery time did not hold the
lock, defying the admonition on its comment.
2022-01-18 07:22:50 -08:00
William BanfieldandGitHub 0c82ceaa5f consensus: calculate prevote message delay metric (#7551)
## What does this pull request do?
This pull requests adds two metrics intended for use in calculating an experimental value for `MessageDelay`.

The metrics are as follows:
```
# HELP tendermint_consensus_complete_prevote_message_delay Difference in seconds between the proposal timestamp and the timestamp of the prevote that achieved 100% of the voting power in the prevote step.
# TYPE tendermint_consensus_complete_prevote_message_delay gauge
tendermint_consensus_complete_prevote_message_delay{chain_id="test-chain-aZbwF1"} 0.013025505

# HELP tendermint_consensus_quorum_prevote_message_delay Difference in seconds between the proposal timestamp and the timestamp of the prevote that achieved a quorum in the prevote step.
# TYPE tendermint_consensus_quorum_prevote_message_delay gauge
tendermint_consensus_quorum_prevote_message_delay{chain_id="test-chain-aZbwF1"} 0.013025505
```

## Why this change?

 For more information on what these metrics are calculating, see #7202. The aim is to merge to backport these metrics to v0.34 and run nodes on a few popular chains with these metrics to determine the experimental values for `MessageDelay` on these popular chains and use these to select our default `SynchronyParams.MessageDelay` value.

## Why Gauges for the metrics?
Gauges allow us to overwrite the metric on each successive observation. We can then capture these metrics over time to track the highest and lowest observed value.
2022-01-18 14:55:18 +00:00
Sam KleinmanandGitHub c0b56e207a consensus: test shutdown to avoid hangs (#7603) 2022-01-18 08:55:13 -05:00
KeneandGitHub 49153b753c rpc: paginate mempool /unconfirmed_txs endpoint (#7612)
This commit changes the behaviour of the /unconfirmed_txs endpoint by replacing limit with a page and perPage parameter for pagination.
The test case for unconfirmed_txs have been accommodated to properly test this change and the documentation for the API as well.
2022-01-18 10:58:32 +01:00
M. J. FrombergerandGitHub 679b6a65b8 light: fix provider error plumbing (#7610)
The custom error types in the provider package did not propagate their wrapped
underlying reasons, making it difficult for the test to check that the correct
error was observed.

- Fix the custom errors to have a true underlying error (not just a string).
- Add Unwrap methods to support inspection by errors.Is.
- Update usage in a few places.
- Fix the test to check for acceptable variation.

Fixes #7609.
2022-01-16 13:48:21 -08:00
M. J. FrombergerandGitHub c24f003b55 protoio: fix incorrect test assertion (#7606)
After writing and then reading a bunch of random messages, the test was
checking that it did not read the same number of messages that it wrote.
The sense of this check was inverted; they should match.

Introduced by accident in #7522. I'm not sure why this did not show up in CI.

Edit: I now know why it didn't show up in ci: #7608.
2022-01-16 20:39:37 +00:00
M. J. FrombergerandGitHub dbe2146d0a rpc: simplify the encoding of interface-typed arguments in JSON (#7600)
Add package jsontypes that implements a subset of the custom libs/json 
package. Specifically it handles encoding and decoding of interface types
wrapped in "tagged" JSON objects. It omits the deep reflection on arbitrary
types, preserving only the handling of type tags wrapper encoding.

- Register interface types (Evidence, PubKey, PrivKey) for tagged encoding.
- Update the existing implementations to satisfy the type.
- Register those types with the jsontypes registry.
- Add string tags to 64-bit integer fields where needed.
- Add marshalers to structs that export interface-typed fields.
2022-01-14 18:14:09 -08:00
Sam KleinmanandGitHub 7ed57ef5f9 statesync: more orderly dispatcher shutdown (#7601) 2022-01-14 16:34:12 -05:00
Sam KleinmanandGitHub 82b65868ce node+autofile: avoid leaks detected during WAL shutdown (#7599) 2022-01-14 13:04:01 -05:00
Sam KleinmanandGitHub 7e8fa4ed85 consensus: explicit test timeout (#7585) 2022-01-13 16:11:51 -05:00
M. J. FrombergerandGitHub b7c19a5cd4 rpc: clean up the RPCFunc constructor signature (#7586)
Instead of taking a comma-separated string of parameter names, take each
parameter name as a separate argument. Now that we no longer have an extra flag
for caching, this fits nicely into a variadic trailer.

* Update all usage of NewRPCFunc and NewWSRPCFunc.
2022-01-13 12:13:28 -08:00
M. J. FrombergerandGitHub 81ee41228a rpc: consolidate RPC route map construction (#7582)
Define interfaces for the various methods a service may implement.  This is
basically just the set of things on Environment that are exported as RPCs, but
these are also implemented by the light proxy.

* internal/rpc: use NewRoutesMap to construct routes on service start
* light/proxy: use NewRoutesMap to construct RPC routes
2022-01-13 10:45:36 -08:00
Sam KleinmanandGitHub cef17e1c02 node+rpc: rpc environment should own it's creation (#7573) 2022-01-13 12:39:48 -05:00
Sam KleinmanandGitHub fd2eccbae1 consensus: use noop logger for WAL test (#7580) 2022-01-13 12:05:12 -05:00
Sam KleinmanandGitHub 2a348cc1e9 logging: remove reamining instances of SetLogger interface (#7572) 2022-01-12 16:56:49 -05:00
Sam KleinmanandGitHub 25e665df17 internal/libs: delete unused functionality (#7569) 2022-01-12 15:55:42 -05:00
Sam KleinmanandGitHub e07c4cdcf2 node: collapse initialization internals (#7567) 2022-01-12 15:32:22 -05:00
Sam KleinmanandGitHub 6efdba8aa9 statesync: SyncAny test buffering (#7570) 2022-01-12 13:38:23 -05:00
M. J. FrombergerandGitHub 1f5e64e5b6 rpc: remove cache control settings from the HTTP server (#7568)
We should not set cache-control headers on RPC responses. HTTP caching
interacts poorly with resources that are expected to change frequently, or
whose rate of change is unpredictable.

More subtly, all calls to the POST endpoint use the same URL, which means a
cacheable response from one call may actually "hide" an uncacheable response
from a subsequent one. This is less of a problem for the GET endpoints, but
that means the behaviour of RPCs varies depending on which HTTP method your
client happens to use. Websocket requests were already marked statically
uncacheable, adding yet a third combination.

To address this:

- Stop setting cache-control headers.
- Update the tests that were checking for those headers.
- Remove the flags to request cache-control.

Apart from affecting the HTTP response headers, this change does not modify the
behaviour of any of the RPC methods.
2022-01-12 18:20:59 +00:00
Sam KleinmanandGitHub fb10d1c705 statesync: clarify test cleanup (#7565) 2022-01-12 12:57:23 -05:00
Sam KleinmanandGitHub cc51bf7587 tests: remove in-test logging (#7558) 2022-01-11 16:39:31 -05:00
M. J. FrombergerandGitHub 50ac52e28d rpc: replace custom context-like argument with context.Context (#7559)
* Rename rpctypes.Context to CallInfo.

Add methods to attach and recover this value from a context.Context.

* Rework RPC method handlers to accept "real" contexts.

- Replace *rpctypes.Context arguments with context.Context.
- Update usage of RPC context fields to use CallInfo.
2022-01-11 11:47:56 -08:00
Sam KleinmanandGitHub 5bf1bdcfb4 reactors: skip log on some routine cancels (#7556) 2022-01-11 12:56:52 -05:00
Sam KleinmanandGitHub d331a08607 statesync: use specific testing.T logger for tests (#7543) 2022-01-10 15:38:20 -05:00
M. J. FrombergerandGitHub 366ab1947a Replace uses of libs/json with encoding/json. (#7534)
Where possible, replace uses of the custom JSON library with the standard
library. The custom library treats interface and unnamed lteral types
differently, so this change avoids those even where it would probably be safe
to switch them.
2022-01-08 08:47:26 -08:00
Sam KleinmanandGitHub d5c39f907d test/factory: pass testing.T around rather than errors for test fixtures (#7518) 2022-01-07 15:51:39 -05:00
Sam KleinmanandGitHub 90cf742065 pex: regularize reactor constructor (#7532) 2022-01-07 13:52:11 -05:00
Sam KleinmanandGitHub aa76a367e0 blocksync: standardize construction process (#7531) 2022-01-07 13:40:08 -05:00
Sam KleinmanandGitHub ae7a76a175 evidence: reactor constructor (#7533) 2022-01-07 13:28:23 -05:00
Sam KleinmanandGitHub 059b38afe4 mempool: refactor mempool constructor (#7530) 2022-01-07 12:49:22 -05:00
Sam KleinmanandGitHub fc36c7782f statesync: reactor and channel construction (#7529) 2022-01-07 12:04:17 -05:00
Sam KleinmanandGitHub 10402b728f consensus+p2p: change how consensus reactor is constructed (#7525) 2022-01-07 10:59:10 -05:00
Sam KleinmanandGitHub 74a8941854 consensus: remove reactor options (#7526) 2022-01-07 09:15:35 -05:00
Sam KleinmanandGitHub 569629486b tests: remove panics from test fixtures (#7522) 2022-01-06 17:34:32 -05:00
Sam KleinmanandGitHub 332163ede6 testing: remove background contexts (#7509) 2022-01-05 12:42:57 -05:00
Sam KleinmanandGitHub f2cc496f09 testing: pass testing.T to assert and require always, assertion cleanup (#7508) 2022-01-05 09:25:08 -05:00
Sam KleinmanandGitHub 3c8955e4b8 errors: formating cleanup (#7507) 2022-01-04 16:11:28 -05:00
Sam KleinmanandGitHub 430817d9e9 types: remove panic from block methods (#7501) 2022-01-03 16:14:26 -05:00
Sam KleinmanandGitHub 1630d1cf3e privval: remove panics in privval implementation (#7475) 2021-12-17 16:28:32 -05:00
Sam KleinmanandGitHub bef120dadf contexts: remove all TODO instances (#7466) 2021-12-16 15:15:26 -05:00
Callum WatersandGitHub 184b105509 statesync: assert app version matches (#7463) 2021-12-16 18:25:26 +01:00
M. J. FrombergerandGitHub 82738eb016 Move the libs/pubsub package to internal scope (#7451)
No API changes, merely changes the import path.
2021-12-15 07:09:32 -08:00
M. J. FrombergerandGitHub ab7da86b06 Internalize libs/sync. (#7450)
Inline the one usage of this library, and remove the lib.
2021-12-14 15:14:30 -08:00
M. J. FrombergerandGitHub da697089d0 Move libs/async to internal/libs/async. (#7449) 2021-12-14 14:05:42 -08:00