1067ba1571
This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.
The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.
Sample output is as follows
```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30
Connections: 1
Rate: 1000
Size: 1024
Total Valid Tx: 9000
Total Negative Latencies: 0
Minimum Latency: 86.632837ms
Maximum Latency: 1.151089602s
Average Latency: 813.759361ms
Standard Deviation: 225.189977ms
Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0
Connections: 1
Rate: 1000
Size: 1024
Total Valid Tx: 9000
Total Negative Latencies: 0
Minimum Latency: 79.312992ms
Maximum Latency: 1.162446243s
Average Latency: 422.755139ms
Standard Deviation: 241.832475ms
Total Invalid Tx: 0
```
closes: #9352
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
2.4 KiB
loadtime
This directory contains the loadtime tools, a set of tools for generating
transaction load against Tendermint and measuring their resulting latency.
loadtime generates transactions that contain the timestamp of when they were
generated as well as additional metadata to track the variables used when
generating the load.
Building the tool set
The Makefile contains a target for building the loadtime tools.
The following command will build the tool and place the resulting binaries in ./build/.
make build
load
The load binary is built when make build is invoked. The load tool generates
transactions and broadcasts them to Tendermint.
load leverages the tm-load-test
framework. As a result, all flags and options specified on the tm-load-test apply to
load.
Below is a basic invocation for generating load against a Tendermint websocket running
on localhost:25567
./build/load \
-c 1 -T 10 -r 1000 -s 1024 \
--broadcast-tx-method sync \
--endpoints ws://localhost:26657/websocket
report
The report binary is built when make build is invoked. The report tool
reads all of the blocks from the specified blockstore database and calculates
transaction latency metrics. report reads transactions generated by load
and uses the difference between the timestamp contained in the transaction and
the timestamp of the block the transaction was executed in to determine transaction latency.
report outputs a set of metrics calculated on the list of latencies, including
minimum, maximum, and average latency as well as the standard deviation.
Below is a basic invocation of the report tool with a data directory under /home/test/.tendermint/data/
where the data was saved in a goleveldb database.
./build/report --database-type goleveldb --data-dir ~/.tendermint/data
The report tool also supports outputting the raw data as csv. This can be
useful if you want to use a more powerful tool to aggregate and analyze the data.
Below is an invocation of the report tool that outputs the data to a csv file
in out.csv
./build/report --database-type goleveldb --data-dir ~/.tendermint/data --csv out.csv
The report tool outputs the data for each experiment separately, identified
by the UUID generated by the load tool at the start of the experiment. It also
outputs the experimental values used for the run.