* e2e: setup testing for multi-version (#9819)
This pull requests sets up the e2e tests to be able to support multiple versions within the same test network. This is achieved through a few simple changes:
* Each node takes a `version` parameter in the testnet manifest. This dictates which version of the testapp to use. If not set, the locally available version is used.
* Adds a `testapp-docker.yml` workflow that publishes the testapp to docker hub so that tagged versions may be available for use in a multi-version test network.
This change does not actually add a testnetwork that does multi-version testing. Since no previous versions of the testapp have been published to dockerhub, there are not old versions available to test against. We'll either need to configure this after the next minor release which will trigger a testapp to be pushed to dockerhub, or push an image from the previous version of Tendermint so that the multiversion test has an old version to pull.
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
(cherry picked from commit 5ba0d131c4)
# Conflicts:
# test/e2e/pkg/manifest.go
* fix conflict
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
Fixes the issue observed in an [e2e test run](https://github.com/tendermint/tendermint/actions/runs/3588927225). The issue arises when the e2e runner process cannot connect to the remote node. In the previous version of this code, the runner would simply skip the transaction if the client couldn't connect. This pull request resurrects that behavior.
---
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
(cherry picked from commit d09f4f503d)
# Conflicts:
# test/e2e/runner/load.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
The "toml" annotation of "send no load" had a typo (`send_no_laod`).
This is suspected to have caused failures in e2e, however I couldn't reproduce the error locally, so not sure this will fix it. Still, the typo needs to be fixed in any case.
---
#### PR checklist
- [x] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
(cherry picked from commit 739b92bf01)
Co-authored-by: Sergio Mena <sergio@informal.systems>
* p2p: add a per-message type send and receive metric (#9622)
* p2p: ressurrect the p2p envelope and use to calculate message metric
Add new SendEnvelope, TrySendEnvelope, BroadcastEnvelope, and ReceiveEnvelope methods in the p2p package to work with the new envelope type.
Care was taken to ensure this was performed in a non-breaking manner.
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
* Added print
* Fix unmarshall
* Fix unmarshalling
* Simplified steps to unmarshall
* minor
* Use 'encoding/hex'
* Forget about C, this is Go!
* gosec warning
* Set maximum payload size
* nosec annotation
(cherry picked from commit b42c439776)
Co-authored-by: Sergio Mena <sergio@informal.systems>
* security/p2p: prevent peers who errored being added to the peer_set (#9500)
* Mark failed removal of peer to address security bug
Co-authored-by: Callum Waters <cmwaters19@gmail.com>
(cherry picked from commit c0bdb2423a)
* Changelong entry and added missing functions for implementations of Peer
Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com>
* Extend the load report tool to include transactions' hashes (#9509)
* Add transaction hash to raw data
* Add hash in formatted output
* Cosmetic
(cherry picked from commit cdd3479f20)
# Conflicts:
# test/loadtime/cmd/report/main.go
* Resolve conflict
* Appease linter
Co-authored-by: Sergio Mena <sergio@informal.systems>
* loadtime: add block time to the data point (#9484)
This pull request adds the block time as the unix time since the epoch to the `report` tool's csv output.
```csv
...
a7a8b903-1136-4da1-97aa-d25da7b4094f,1614226790,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1614196724,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1613097336,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1609365168,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1617199169,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1615197134,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1610399447,1663707084905417366,4,200,1024
...
```
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
(cherry picked from commit 5fe1a72416)
* lint fix
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
the `NewClient` method is called by the load test framework for each connection. This means that if multiple connections are instantiated, each connection will erroneously have its own UUID. This PR changes the UUID generation to happen at the _beginning_ of the script instead of on client creation so that each experimental run shares a UUID.
Caught while preparing the script for production readiness.
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
(cherry picked from commit 59a711eabe)
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* add separated runs by UUID (#9367)
This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.
The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.
Sample output is as follows
```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30
Connections: 1
Rate: 1000
Size: 1024
Total Valid Tx: 9000
Total Negative Latencies: 0
Minimum Latency: 86.632837ms
Maximum Latency: 1.151089602s
Average Latency: 813.759361ms
Standard Deviation: 225.189977ms
Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0
Connections: 1
Rate: 1000
Size: 1024
Total Valid Tx: 9000
Total Negative Latencies: 0
Minimum Latency: 79.312992ms
Maximum Latency: 1.162446243s
Average Latency: 422.755139ms
Standard Deviation: 241.832475ms
Total Invalid Tx: 0
```
closes: #9352
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
(cherry picked from commit 1067ba1571)
# Conflicts:
# go.mod
* fix merge conflict
* fix lint
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
* test: add the loadtime report tool (#9351)
This pull request adds the report tool and modifies the loadtime libraries to better support its use.
(cherry picked from commit 8655080a0f)
* add nolint
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
* test: add the loadtime tool (#9342)
This pull request adds the loadtime tool. This tool leverages the tm-load-test framework. Using the framework means that the only real logic that needs to be written is the logic for Tx generation. The framework does the rest.
The tool writes a set of metadata into the transaction, including the current transaction rate, number of connections, specified size of the transaction, and the current time.
* lint
* Ignore generated/copied RPC docs
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync vuepress config with main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync docs package-lock.json with main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync docs redirects with main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync docs versions with main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update OpenAPI version to v0.34
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync DOCS_README with main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update all v0.34.x docs references from master to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update v0.34 OpenAPI references from master to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update repo doc links from master to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update code comment references from master to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update repo root doc links from master to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update repo root doc links for docs.tendermint.com from master to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Build v0.34.x as "latest"
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Explicitly mark v0.34 docs as latest in version selector
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add nav link to main and clearly mark as unstable
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Direct all docs.tendermint.com links to v0.34 on v0.34.x
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update all relevant links on v0.34.x branch to be v0.34-specific
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update changelog refs to docs.tendermint.com
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update remaining GH master link to main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync docs build and nav config with main
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Migrate spec links to GitHub repo from docs site
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: abci protocol should be consistent across networks (#7078)
It seems weird in retrospect that we allow networks to contain
applications that use different ABCI protocols.
(cherry picked from commit f2a8f5e054)
I realized after my last commit that my change made a following line of code a bit redundant.
(alternatively my last change was redunadnt to the existing code.)
I took this oppertunity to make some minor cleanups and logging changes to the node changes which I hope will make tests a bit more clear.
(cherry picked from commit a374f74f7c)
Co-authored-by: Sam Kleinman <garen@tychoish.com>
Executed a local network using simapp and looked for logs that seemed superfluous. This isn't by any means an exhaustive grooming, but should drastically help legibility of logs.
ref: #5912
