Commit Graph

38 Commits

Author SHA1 Message Date
Marko Baricevic
42343ddef2 fix e2e tests 2021-02-10 10:39:25 +01:00
Marko Baricevic
798ac145f9 e2e & maverick 2021-02-04 15:16:31 +01:00
Erik Grinaker
5f88d6aa44 test/e2e: increase validator tolerances (#6037) 2021-02-02 16:41:27 +00:00
Erik Grinaker
3d01d98f67 test/e2e: increase sign/propose tolerances (#6033)
E2E tests often fail because validators miss signing or proposing blocks. Often this is because e.g. there's a lot of disruption in the network or it takes a long time to start up all the nodes.

This changes the test criteria to only check for 3 signed/proposed blocks, rather than a fraction of the expected blocks. This should be enough to catch most issues, apart from performance problems causing nodes to miss signing/proposing, but we may want separate tests for those sorts of things.
2021-02-02 12:43:08 +00:00
Anton Kaliaev
b1646e51e2 test/e2e: enable pprof server to help debugging failures (#6003) 2021-01-28 15:21:07 +00:00
Anton Kaliaev
a54f1544f8 .github: rename crashers output (fuzz-nightly-test) (#5993) 2021-01-28 19:12:48 +04:00
Aleksandr Bezobchuk
bd8a9372d2 consensus: Groom Logs (#5917)
Executed a local network using simapp and looked for logs that seemed superfluous. This isn't by any means an exhaustive grooming, but should drastically help legibility of logs.


ref: #5912
2021-01-27 20:53:24 +00:00
Callum Waters
aecfb0ecf0 e2e: add control over the log level of nodes (#5958) 2021-01-25 17:20:39 +01:00
Erik Grinaker
66ba12d9bc test/e2e: tolerate up to 2/3 missed signatures for a validator (#5878)
E2E tests often fail due to fast sync stalls causing the validator to miss signing blocks. This increases the tolerance for missed signatures to 2/3 to allow validators to spend more time starting up.
2021-01-08 11:06:51 +00:00
Marko
09cf0bcb01 privval: add grpc (#5725)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2021-01-06 10:49:30 -08:00
Erik Grinaker
1570d26f84 test/e2e: add conceptual overview (#5857)
This should be useful to understand the overall purpose and structure of the end-to-end tests.
2021-01-04 17:51:49 +00:00
Erik Grinaker
17ca6c6c98 test/e2e: disable abci/grpc and blockchain/v2 due to flake (#5854)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-04 15:19:07 +00:00
Callum Waters
3283a84ab2 cmd: hyphen case cli and config (#5777) 2020-12-11 13:14:04 +01:00
Anton Kaliaev
28e79a4d02 cmd: modify gen_node_key to print key to STDOUT (#5772)
closes: #5770
closes: #5769

also, include node ID in the output (#5769) and modify NodeKey to use
value semantics (it makes perfect sense for NodeKey to not be a
pointer).
2020-12-10 11:02:35 +04:00
Anton Kaliaev
243ff4b43d blockchain/v1: remove in favor of v2 (#5728) 2020-12-03 09:35:47 +04:00
Anton Kaliaev
170cb70e19 test/e2e: enable v1 and v2 blockchains (#5702)
* test/e2e: enable v1 and v2 blockchains

* modify networks/ci.toml
2020-11-23 13:32:22 +00:00
Marko
b2d72dce7e e2e: use ed25519 for secretConn (remote signer) (#5678)
## Description

Hardcode ed25519 to dialTCPFn in e2e tests. 

I will backport `DefaultRequestHandler` fixes

This will be replaced when grpc is implemented.
2020-11-17 13:55:23 +00:00
Marko
e0950515ff test/e2e: fix secp failures (#5649) 2020-11-16 12:31:32 +01:00
Marko
bf35cc6443 cmd: add support for --key (#5612)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-11-09 15:22:36 +01:00
Callum Waters
9fe7b4fe77 remove misbehaviors from e2e generator (#5629) 2020-11-06 14:27:48 +01:00
Callum Waters
3922dde05d evidence: structs can independently form abci evidence (#5610) 2020-11-04 17:14:48 +01:00
Erik Grinaker
53022220f6 test: fix various E2E test issues (#5576)
* Don't use state sync for nodes starting at initial height.
* Also remove stopped containers when cleaning up.
* Start nodes in order of startAt, mode, name to avoid full nodes starting before their seeds.
* Tweak network waiting to avoid halts caused by validator changes and perturbations.
* Disable most tests for seed nodes, which aren't always able to join consensus.
* Disable `blockchain/v2` due to known bugs.
2020-10-27 16:22:00 +00:00
Erik Grinaker
7daf6a1a03 test: disable E2E misbehaviors due to bugs (#5569)
Disables misbehaviors in E2E testnets due to failures caused by #5554 and #5560. Should be re-enabled once these are fixed.
2020-10-26 10:28:11 +00:00
Erik Grinaker
10dda219a1 test: fix handling of start height in generated E2E testnets (#5563)
In #5488 the E2E testnet generator changed to setting explicit `StartAt` heights for initial nodes. This broke the runner, which expected all initial nodes to have `StartAt: 0`, as well as validator set scheduling in the generator. Testnet loading now normalizes initial nodes to have `StartAt: 0`.

This also tweaks waiting for misbehavior heights to only use an additional wait if there actually is any misbehavior in the testnet, and to output information when waiting.
2020-10-26 10:09:46 +00:00
Callum Waters
50b91867c3 test: add evidence e2e tests (#5488) 2020-10-23 12:33:08 +02:00
Erik Grinaker
d11e5993b1 test: tag E2E Docker resources and autoremove them (#5558)
Fixes #5555.
2020-10-23 08:17:15 +00:00
Erik Grinaker
e0e006d10f test: run remaining E2E testnets on run-multiple.sh failure (#5557)
Fixes #5542.
2020-10-22 20:53:27 +00:00
Erik Grinaker
8c5fe166a6 test: enable restart/kill perturbations in E2E tests (#5537)
When #5536 lands we can re-enable restart/kill perturbations in E2E tests.
2020-10-20 19:30:07 +00:00
Erik Grinaker
9e6248c0d7 test: enable blockchain v2 in E2E testnet generator (#5533)
When #5499 and #5530 land, we can re-enable v2 in the E2E testnet generator (and thus the nightly E2E tests).
2020-10-20 11:44:22 +00:00
Erik Grinaker
3dabfbeae0 test: enable ABCI gRPC client in E2E testnets (#5521)
Once #5520 lands, we can re-enable gRPC ABCI protocol in the E2E testnets.
2020-10-20 08:27:58 +00:00
Erik Grinaker
f0d4ddcf3c test: tweak E2E tests for nightly runs (#5512) 2020-10-16 12:10:29 +02:00
Erik Grinaker
b6979e7fbd test: clean up E2E test volumes using a container (#5509) 2020-10-15 20:59:16 +02:00
Erik Grinaker
8a67968416 github: add nightly E2E testnet action (#5480) 2020-10-14 16:55:02 +02:00
Erik Grinaker
260cc5dd69 test/e2e: add random testnet generator (#5479)
Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
2020-10-09 12:33:48 +00:00
Erik Grinaker
8d28e7467c test: add E2E test for node peering (#5465)
This was a missing test case from the old P2P tests removed in #5453, which makes sure that all nodes are able to peer with each other regardless of how they discover peers.

Fixes #2795, since the default CI testnet uses a combination of (partially meshed) persistent peers and PEX-based seed nodes.
2020-10-06 06:00:20 +00:00
Erik Grinaker
7e27e9b852 test: add GitHub action for end-to-end tests (#5452)
Partial fix for #5291.
2020-10-05 12:44:35 +02:00
Erik Grinaker
090afe30f9 test: add basic end-to-end test cases (#5450)
Partial fix for #5291.

This adds a basic set of test cases for core network invariants. Although small, it is sufficient to replace and extend the current set of P2P tests. Further test cases can be added later.
2020-10-05 10:23:12 +00:00
Erik Grinaker
250c3aa92e test: add end-to-end testing framework (#5435)
Partial fix for #5291. For details, see [README.md](https://github.com/tendermint/tendermint/blob/erik/e2e-tests/test/e2e/README.md) and [RFC-001](https://github.com/tendermint/tendermint/blob/master/docs/rfc/rfc-001-end-to-end-testing.md).

This only includes a single test case under `test/e2e/tests/`, as a proof of concept - additional test cases will be submitted separately. A randomized testnet generator will also be submitted separately, there a currently just a handful of static testnets under `test/e2e/networks/`. This will eventually replace the current P2P tests and run in CI.
2020-10-05 09:35:01 +00:00