Commit Graph

202 Commits

Author SHA1 Message Date
Thane Thomson
45071d1f23 abci: Add unsynchronized local client (#9660)
* Remove extra interface cast

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove irrelevant comment

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* abci: Add unsynchronized local client

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* proxy: Add unsync local client creator

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* e2e: Add sync app for use with unsync local client

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* abci: Elaborate on mutex param in unsync local client

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* proxy: Remove unnecessary comment

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* abcicli: Remove unnecessary mutex param from unsync client

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* ci/e2e: Explicitly use sync app for validator04

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* e2e: Ensure app is definitely the E2E app

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Signed-off-by: Thane Thomson <connect@thanethomson.com>
2022-11-07 06:46:55 -05:00
Thane Thomson
816c6bac00 rpc: Add caching support (#9650)
* Set cache control in the HTTP-RPC response header

* Add a simply cache policy to the RPC routes

* add a condition to check the RPC request has default height settings

* fix cherry pick error

* update pending log

* use options struct intead of single parameter

* refacor FuncOptions to functional options

* add functional options in WebSocket RPC function

* revert doc

* replace deprecated function call

* revise functional options

* remove unuse comment

* fix revised error

* adjust cache-control settings

* Update rpc/jsonrpc/server/http_json_handler.go

Co-authored-by: Thane Thomson <connect@thanethomson.com>

* linter: Fix false positive

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* rpc: Separate cacheable and non-cacheable HTTP response writers

Allows us to roll this change out in a non-API-breaking way, since this
is an additive change.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* rpc: Ensure consistent caching strategy

Ensure a consistent caching strategy across both JSONRPC- and URI-based
requests.

This requires a bit of a refactor of the previous caching logic, which
is complicated a little by the complex reflection-based approach taken
in the Tendermint RPC.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* rpc: Add more tests for caching

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update CHANGELOG_PENDING

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* light: Sync routes config with RPC core

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* rpc: Update OpenAPI docs

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: jayt106 <jaytseng106@gmail.com>
Co-authored-by: jay tseng <jay.tseng@crypto.com>
Co-authored-by: JayT106 <JayT106@users.noreply.github.com>
2022-11-03 13:19:44 -04:00
William Banfield
f6709208b0 e2e: configurable IP addresses for e2e testnet generator (#9592)
* add the infrastructure types

* add infra data to testnetload

* extract infrastructure generation from manifest creation

* add infrastructure type and data flags

* rename docker ifd constructor

* implement read ifd from file

* add 'provider' field to the infrastructure data file to disable ip range check

* return error from infrastructure from data file function

* remove ifd from Setup

* implement a basic infra provider with a simple setup command

* remove misbehavior remnants

* use manifest instead of file in all places

* include cidr block range in the infrastructure data

* nolint gosec

* gosec

* lint
2022-10-25 10:19:10 -04:00
Sergio Mena
b42c439776 Fix TX payload for DO testnets (#9540)
* Added print

* Fix unmarshall

* Fix unmarshalling

* Simplified steps to unmarshall

* minor

* Use 'encoding/hex'

* Forget about C, this is Go!

* gosec warning

* Set maximum payload size

* nosec annotation
2022-10-12 19:58:50 +02:00
Sergio Mena
cdd3479f20 Extend the load report tool to include transactions' hashes (#9509)
* Add transaction hash to raw data

* Add hash in formatted output

* Cosmetic
2022-10-05 21:16:45 +02:00
William Banfield
5fe1a72416 loadtime: add block time to the data point (#9484)
This pull request adds the block time as the unix time since the epoch to the `report` tool's csv output.

```csv
...
a7a8b903-1136-4da1-97aa-d25da7b4094f,1614226790,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1614196724,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1613097336,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1609365168,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1617199169,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1615197134,1663707084905417366,4,200,1024
a7a8b903-1136-4da1-97aa-d25da7b4094f,1610399447,1663707084905417366,4,200,1024
...
```

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed
2022-09-23 13:55:55 +00:00
Elias Naur
ac866e38d4 test/fuzz: bring changes from v0.34.x into main (#9299) 2022-09-11 16:27:42 +02:00
William Banfield
59a711eabe test: generate uuid on startup for load tool (#9383)
the `NewClient` method is called by the load test framework for each connection. This means that if multiple connections are instantiated, each connection will erroneously have its own UUID. This PR changes the UUID generation to happen at the _beginning_ of the script instead of on client creation so that each experimental run shares a UUID.

Caught while preparing the script for production readiness.

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed
2022-09-08 01:00:07 +00:00
William Banfield
1067ba1571 add separated runs by UUID (#9367)
This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.

Sample output is as follows

```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0
```

closes: #9352 

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed
2022-09-06 14:38:08 +00:00
William Banfield
8655080a0f test: add the loadtime report tool (#9351)
This pull request adds the report tool and modifies the loadtime libraries to better support its use.
2022-09-02 13:32:56 -04:00
William Banfield
70fd943bdf test: add the loadtime tool (#9342)
This pull request adds the loadtime tool. This tool leverages the tm-load-test framework. Using the framework means that the only real logic that needs to be written is the logic for Tx generation. The framework does the rest.

The tool writes a set of metadata into the transaction, including the current transaction rate, number of connections, specified size of the transaction, and the current time.
2022-08-31 16:41:15 -04:00
Thane Thomson
cceea4de22 chore: Format and fix lints (#9336)
* make format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix linting directives

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* make mockery

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Appease CI linter

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Appease CI linter

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Signed-off-by: Thane Thomson <connect@thanethomson.com>
2022-08-30 12:28:46 -04:00
Thane Thomson
7b40167f58 abci: Port EventAttribute field type change to main (#9335)
* types: Refactor EventAttribute (#6408)

Cherry-pick of 09a6ad7b1e

* Ensure context is honored

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Replace with tagged switch

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Ensure contexts are honored

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add UPGRADING note about type change

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove unnecessary conversion

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
2022-08-30 10:13:55 -04:00
Callum Waters
5f932a53f0 e23: fix generation by considering evidence age in retain height (#9306) 2022-08-23 11:28:32 +02:00
Sergio Mena
50b5c23d88 Merge branch 'feature/abci++ppp' 2022-08-22 17:16:17 +02:00
Callum Waters
b37f062619 e2e: add evidence tests (#9292) 2022-08-22 13:33:47 +02:00
Callum Waters
2d8df1bd4e proto: deduplicate consensus params (#9287) 2022-08-22 10:50:21 +02:00
Callum Waters
0ca3a89c90 e2e: add abci delays (#9254) 2022-08-19 12:00:58 +02:00
William Banfield
1a4d9397e9 abci++: update PrepareProposal API to accept just a list of bytes (#9283)
This changes the ResponsePrepareProposal type, substituting the []TxRecord for just []bytes. This change is made in the .proto file and in all necessary additional places in the code.
2022-08-18 16:33:19 -04:00
William Banfield
1069ffc6aa config: backport the rename of fastsync to blocksync (#9259)
This is largely a cherry pick of #6755 with some additional fixups added where detected. 
This change moves the blockchain package to a package called blocksync. Additionally, it renames the relevant uses of the term `fastsync` to `blocksync`.

closes: #9227 

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [x] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed
2022-08-17 15:19:20 +00:00
AdamKorcz
498657f128 test/fuzz: fix OSS-Fuzz build (#9183) 2022-08-16 15:59:24 +02:00
Sergio Mena
7dc4f934b0 Merge branch main into feature/abci++ppp 2022-08-12 13:59:19 +02:00
Jacob Gadikian
a6dde14ec4 do not use ioutil (#9206) 2022-08-10 14:35:14 -04:00
Callum Waters
f861062ee2 abci: implement process proposal to spec (#9122) 2022-08-09 15:15:18 +02:00
Sergio Mena
b2409b3345 Follow-up fixes to main PrepareProposal implementation (#9162)
* -----start------

* [cherrypicked] state: panic on ResponsePrepareProposal validation error (#8145)

* state: panic on ResponsePrepareProposal validation error

* lint++

Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* [cherrypicked] abci++: remove CheckTx call from PrepareProposal flow (#8176)

* [cherrypicked] abci++: correct max-size check to only operate on added and unmodified (#8242)

* [cherrypicked] Remove `ModifiedTxStatus` from the spec and the code (#8210)

* Outstanding abci-gen changes to 'pb.go' files

* Removed modified_tx_status from spec and protobufs

* Fix sed for OSX

* Regenerated abci protobufs with 'abci-proto-gen'

* Code changes. UTs e2e tests passing

* Recovered UT: TestPrepareProposalModifiedTxStatusFalse

* Adapted UT

* Fixed UT

* Revert "Fix sed for OSX"

This reverts commit e576708c61.

* Update internal/state/execution_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update internal/state/execution_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed some comments

* Added one test that tests error at the ABCI client + Fixed some mock calls

* Addressed remaining comments

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update abci/example/kvstore/kvstore.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed William's latest comments

* Adressed Michael's comment

* Fixed UT

* Some md fixes

* More md fixes

* gofmt

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* make proto-gen

* Fixed testcase on PrepareProposal error

* mockery

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-08-03 19:39:17 +02:00
Sergio Mena
d268e56383 Sync PrepareProposal with Spec. Main part (#9158)
* ----start----

* [PARTIAL cherry-pick] ABCI Vote Extension 2 (#6885)

* Cherry-picked #6567: state/types: refactor makeBlock, makeBlocks and makeTxs (#6567)

* [Cherrypicked] types: remove panic from block methods (#7501)

* [cherrypicked] abci++: synchronize PrepareProposal with the newest version of the spec (#8094)

This change implements the logic for the PrepareProposal ABCI++ method call. The main logic for creating and issuing the PrepareProposal request lives in execution.go and is tested in a set of new tests in execution_test.go. This change also updates the mempool mock to use a mockery generated version and removes much of the plumbing for the no longer used ABCIResponses.

* make proto-gen

* Backported EvidenceList's method ToABCI from #7961

* make build

* Fix mockery for Mempool

* mockery

* Backported abci Application mocks from #7961

* mockery2

* Fixed new PrepareProposal test cases in state/execution_test.go

* Fixed returned errors in consensus/state.go

* lint

* Addressed @cmwaters' comment

Co-authored-by: mconcat <monoidconcat@gmail.com>
Co-authored-by: JayT106 <JayT106@users.noreply.github.com>
Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
2022-08-03 17:24:24 +02:00
Callum Waters
ea271c534a abci: remove counter app (port #6684) (#9143) 2022-08-03 13:44:34 +02:00
Sergio Mena
2164883501 Old PrepareProposal and vote extension integration on feature/abci++ppp (#9117)
* abci: PrepareProposal-VoteExtension integration [2nd try] (#7821)

* PrepareProposal-VoteExtension integration (#6915)

* make proto-gen

* Fix protobuf crash in e2e nightly tests

* Update types/vote.go

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Addressed @creachadair's comments

Co-authored-by: mconcat <monoidconcat@gmail.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Proto changes

* make proto-gen

* Fixed UTs

* bump

* lint

* lint2

* lint3

* lint4

* lint5

* lint6

* no_lint

Co-authored-by: mconcat <monoidconcat@gmail.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-08-03 13:29:15 +02:00
Callum Waters
4206a0e9b7 remove v1 and v2 blocksync protocol impementations (#9146) 2022-08-02 11:30:28 +02:00
Callum Waters
07d242c461 e2e: remove maverick (#9148) 2022-08-01 17:15:03 +02:00
Callum Waters
1e9d81fb8a chore: bump go to 1.18 (#9147) 2022-08-01 16:45:25 +02:00
Callum Waters
49ec3b9780 chore: lint repo (use american english) (#9144) 2022-08-01 14:24:49 +02:00
Sergio Mena
58effdd8b3 Old PrepareProposal on feature/abci++ppp (#9106)
* [Rebased to v0.34.x] abci: PrepareProposal (#6544)

* fixed cherry-pick

* proto changes

* make proto-gen

* UT fixes

* generate Client directive

* mockery

* App fixes

* Disable 'modified tx' hack

* mockery

* Make format

* Fix lint

Co-authored-by: Marko <marbar3778@yahoo.com>
2022-07-27 15:09:02 +02:00
Aleksandr Bezobchuk
6b7d30cf37 feat: v0.34.x Prioritized Mempool (#8695)
* Updated mocks

* add reactor tests

* add v1 reactor tests

* Fix fuzz test for priority mempool

* e2e adapted to mempool v1; prio pool is default now

* Reverted default mempool to be fifo

* Changed buf version

* Added priority mempool to ci testnet

* Fixed linter

* Updated makefile

* Aligned makefile changes to v0.34.x

* Added go install for proto

* Add log message to warn about prioritized mempool bug

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Changelog message

Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com>
Co-authored-by: Callum Waters <cmwaters19@gmail.com>
Co-authored-by: Sam Kleinman <garen@tychoish.com>
Co-authored-by: Thane Thomson <connect@thanethomson.com>
2022-06-27 11:34:28 +02:00
Joe Abbey
4a1df4911d fix: "Lazy" Stringers to defer Sprintf and Hash until logs print (#8845) 2022-06-23 14:56:34 -04:00
mergify[bot]
db60bbad54 statesync: assert app version matches (backport #7856) (#7885) 2022-02-23 12:17:12 +01:00
mergify[bot]
05340ca069 cmd: add integration test for rollback functionality (backport #7315) (#7368)
* cmd: add integration test and fix bug in rollback command (#7315)

(cherry picked from commit bca2080c01)

Co-authored-by: Callum Waters <cmwaters19@gmail.com>
2021-12-02 08:30:57 -08:00
Sam Kleinman
4ef140f6ca lint: cleanup pending lint errors (#7237) 2021-11-04 08:08:55 -04:00
mergify[bot]
1dfb3451ea e2e: light nodes should use builtin abci app (#7095) (#7096)
(cherry picked from commit befd669794)

Co-authored-by: Sam Kleinman <garen@tychoish.com>
2021-10-09 00:32:41 -04:00
mergify[bot]
9f13b9b083 e2e: abci protocol should be consistent across networks (backport #7078) (#7085)
* e2e: abci protocol should be consistent across networks (#7078)

It seems weird in retrospect that we allow networks to contain
applications that use different ABCI protocols.

(cherry picked from commit f2a8f5e054)
2021-10-08 10:37:12 -04:00
Callum Waters
2d8287d0f7 e2e: allow running of single node using the e2e app (backport) (#7024) 2021-09-29 16:17:32 +02:00
Sam Kleinman
294a9695b4 e2e: backport minor reliability improvements (#6967) 2021-09-21 17:29:56 -04:00
mergify[bot]
4023580a25 e2e: cleanup node start function (#6842) (#6848)
I realized after my last commit that my change made a following line of code a bit redundant.

(alternatively my last change was redunadnt to the existing code.)

I took this oppertunity to make some minor cleanups and logging changes to the node changes which I hope will make tests a bit more clear.

(cherry picked from commit a374f74f7c)

Co-authored-by: Sam Kleinman <garen@tychoish.com>
2021-08-20 16:12:13 -04:00
mergify[bot]
2db1e422d8 e2e: avoid starting nodes from the future (#6835) (#6838)
(cherry picked from commit a4cc8317da)

Co-authored-by: Sam Kleinman <garen@tychoish.com>
2021-08-18 14:42:27 -04:00
Callum Waters
8b84c7c168 e2e: disable app tests for light client (#6672) 2021-07-07 20:06:55 +02:00
Callum Waters
d515bbcf1d statesync: increase chunk priority and robustness (#6582) 2021-06-18 09:59:52 +02:00
mergify[bot]
be8c9833ca state sync: tune request timeout and chunkers (backport #6566) (#6581)
* state sync: tune request timeout and chunkers (#6566)

(cherry picked from commit 7d961b55b2)

# Conflicts:
#	CHANGELOG_PENDING.md
#	config/config.go
#	internal/statesync/reactor.go
#	internal/statesync/reactor_test.go
#	node/node.go
#	statesync/syncer.go

* fix build

* fix config

* fix config

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>
2021-06-15 15:10:16 -04:00
mergify[bot]
353e3a3243 evidence: fix bug with hashes (backport #6375) (#6381) 2021-04-22 15:05:56 +02:00
mergify[bot]
1614e12035 statesync: improve e2e test outcomes (backport #6378) (#6380)
(cherry picked from commit d36a5905a6)

Co-authored-by: Sam Kleinman <garen@tychoish.com>
2021-04-21 12:30:17 -04:00
Callum Waters
b878326396 e2e: relax timeouts (#6356)
* remove duplicate light error

* quieten handling of txs that already exist in the mempool

* notch back e2e timeouts
2021-04-14 19:53:54 +02:00