24 Commits

Author SHA1 Message Date
Sam Kleinman
013b46a6c3 libs/strings: move to internal (#8890)
I think we were leaving this library public because the SDK dependend
upon it, but the function the SDK was using was one that we'd removed
because *we* weren't using it any more, and I made a PR agasint the
SDK to clean that up.

ref: https://github.com/cosmos/cosmos-sdk/pull/12368
2022-06-27 21:44:50 +00:00
Sergio Mena
ce6485fa70 Remove the abci responses type - prune legacy responses (#8673)
Closes #8069 

* Type `ABCIResponses` was just wrapping type `ResponseFinalizeBlock`. This patch removes the former.
* Did some renaming to avoid confusion on the data structure we are working with.
* We also remove any stale ABCIResponses we may have in the state store at the time of pruning

**IMPORTANT**: There is an undesirable side-effect of the unwrapping. An empty `ResponseFinalizeBlock` yields a 0-length proto-buf serialized buffer. This was not the case with `ABCIResponses`. I have added an interim solution, but open for suggestions on more elegant ones.
2022-06-02 19:13:08 +00:00
M. J. Fromberger
c620900fdd rpc: fix plumbing of broadcast_tx_commit timeout (#8573)
In #3435 we allowed this timeout to override the global write timeout.
But after #8570 this meant we were applying a shorter timeout by default.
Don't do the patch if the timeout is already unlimited.

This is a temporary workaround; in light of #8561 I plan to get rid of this
option entirely during the v0.37 cycle, but meanwhile we should keep existing
use more or less coherent.
2022-05-17 10:34:43 -07:00
M. J. Fromberger
da1b871808 Unify RPC method signatures and parameter decoding (#8397)
Pass all parameters from JSON-RPC requests to their corresponding handlers
using struct types instead of positional parameters. This allows us to control
encoding of arguments using only the standard library, and to eliminate the
remaining special-purpose JSON encoding hooks in the server.

To support existing use, the server still allows arguments to be encoded in
JSON as either an array or an object.

Related changes:

- Rework the RPCFunc constructor to reduce reflection during RPC call service.
- Add request parameter wrappers for each RPC service method.
- Update the RPC Environment methods to use these types.
- Update the interfaces and shims derived from Environment to the new
  signatures.
- Update and extend test cases.
2022-04-27 07:53:51 -07:00
Sam Kleinman
0bded371c5 testing: logger cleanup (#8153)
This contains two major changes:

- Remove the legacy test logging method, and just explicitly call the
  noop logger. This is just to make the test logging behavior more
  coherent and clear. 
  
- Move the logging in the light package from the testing.T logger to
  the noop logger. It's really the case that we very rarely need/want
  to consider test logs unless we're doing reproductions and running a
  narrow set of tests.
  
In most cases, I (for one) prefer to run in verbose mode so I can
watch progress of tests, but I basically never need to consider
logs. If I do want to see logs, then I can edit in the testing.T
logger locally (which is what you have to do today, anyway.)
2022-03-18 17:39:38 +00:00
William Banfield
0b8a62c87b abci: Synchronize FinalizeBlock with the updated specification (#7983)
This change set implements the most recent version of `FinalizeBlock`. 

# What does this change actually contain?

* This change set is rather large but fear not! The majority of the files touched and changes are renaming `ResponseDeliverTx` to `ExecTxResult`. This should be a pretty inoffensive change since they're effectively the same type but with a different name.
* The `execBlockOnProxyApp` was totally removed since it served as just a wrapper around the logic that is now mostly encapsulated within `FinalizeBlock`
* The `updateState` helper function has been made a public method on `State`. It was being exposed as a shim through the testing infrastructure, so this seemed innocuous.
* Tests already existed to ensure that the application received the `ByzantineValidators` and the `ValidatorUpdates`, but one was fixed up to ensure that `LastCommitInfo` was being sent across.
* Tests were removed from the `psql` indexer that seemed to search for an event in the indexer that was not being created.

# Questions for reviewers
* We store this [ABCIResponses](5721a13ab1/proto/tendermint/state/types.pb.go (L37)) type in the data base as the block results. This type has changed since v0.35 to contain the `FinalizeBlock` response. I'm wondering if we need to do any shimming to keep the old data retrieveable?
* Similarly, this change is exposed via the RPC through [ResultBlockResults](5721a13ab1/rpc/coretypes/responses.go (L69)) changing. Should we somehow shim or notify for this change? 


closes: #7658
2022-03-04 22:32:37 +00:00
Sergio Mena
d3548eb706 Completed the existing FinalizeBlock PR and rebased to master (#7798)
* Rebased and git-squashed the commits in PR #6546

migrate abci to finalizeBlock

work on abci, proxy and mempool

abciresponse, blok events, indexer, some tests

fix some tests

fix errors

fix errors in abci

fix tests amd errors

* Fixes after rebasing PR#6546

* Restored height to RequestFinalizeBlock & other

* Fixed more UTs

* Fixed kvstore

* More UT fixes

* last TC fixed

* make format

* Update internal/consensus/mempool_test.go

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

* Addressed @williambanfield's comments

* Fixed UTs

* Addressed last comments from @williambanfield

* make format

Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
2022-02-14 23:41:28 +01:00
M. J. Fromberger
7e09c2ef43 Clean up temp files more thoroughly after testing. (#7815)
Our test cases spew a lot of files and directories around $TMPDIR.  Make more
thorough use of the testing package's TempDir methods to ensure these are
cleaned up.

In a few cases, this required plumbing test contexts through existing helper
code. In a couple places an explicit path was required, to work around cases
where we do global setup during a TestMain function. Those cases probably
deserve more thorough cleansing (preferably with fire), but for now I have just
worked around it to keep focused on the cleanup.
2022-02-14 06:32:07 -08:00
M. J. Fromberger
b7c19a5cd4 rpc: clean up the RPCFunc constructor signature (#7586)
Instead of taking a comma-separated string of parameter names, take each
parameter name as a separate argument. Now that we no longer have an extra flag
for caching, this fits nicely into a variadic trailer.

* Update all usage of NewRPCFunc and NewWSRPCFunc.
2022-01-13 12:13:28 -08:00
Sam Kleinman
2a348cc1e9 logging: remove reamining instances of SetLogger interface (#7572) 2022-01-12 16:56:49 -05:00
Sam Kleinman
e07c4cdcf2 node: collapse initialization internals (#7567) 2022-01-12 15:32:22 -05:00
M. J. Fromberger
1f5e64e5b6 rpc: remove cache control settings from the HTTP server (#7568)
We should not set cache-control headers on RPC responses. HTTP caching
interacts poorly with resources that are expected to change frequently, or
whose rate of change is unpredictable.

More subtly, all calls to the POST endpoint use the same URL, which means a
cacheable response from one call may actually "hide" an uncacheable response
from a subsequent one. This is less of a problem for the GET endpoints, but
that means the behaviour of RPCs varies depending on which HTTP method your
client happens to use. Websocket requests were already marked statically
uncacheable, adding yet a third combination.

To address this:

- Stop setting cache-control headers.
- Update the tests that were checking for those headers.
- Remove the flags to request cache-control.

Apart from affecting the HTTP response headers, this change does not modify the
behaviour of any of the RPC methods.
2022-01-12 18:20:59 +00:00
M. J. Fromberger
82738eb016 Move the libs/pubsub package to internal scope (#7451)
No API changes, merely changes the import path.
2021-12-15 07:09:32 -08:00
Sam Kleinman
8a991e288c service: plumb contexts to all (most) threads (#7363)
This continues the push of plumbing contexts through tendermint. I
attempted to find all goroutines in the production code (non-test) and
made sure that these threads would exit when their contexts were
canceled, and I believe this PR does that.
2021-12-02 21:38:38 +00:00
M. J. Fromberger
76dea94a01 Remove now-unused nolint:lll directives. (#7356) 2021-11-30 21:32:21 +00:00
M. J. Fromberger
1dca1a8f97 Performance improvements for the event query API (#7319)
Rework the implementation of event query parsing and execution to
improve performance and reduce memory usage.

Previous memory and CPU profiles of the pubsub service showed query
processing as a significant hotspot. While we don't have evidence that
this is visibly hurting users, fixing it is fairly easy and self-contained.

Updates #6439.

Typical benchmark results comparing the original implementation (PEG) with the reworked implementation (Custom):
```
TEST                        TIME/OP  BYTES/OP  ALLOCS/OP  SPEEDUP   MEM SAVING
BenchmarkParsePEG-12       51716 ns  526832    27
BenchmarkParseCustom-12     2167 ns    4616    17         23.8x     99.1%
BenchmarkMatchPEG-12        3086 ns    1097    22
BenchmarkMatchCustom-12    294.2 ns      64     3         10.5x     94.1%
```

Components:
* Add a basic parsing benchmark.
* Move the original query implementation to a subdirectory.
* Add lexical scanner for Query expressions.
* Add a parser for Query expressions.
* Implement query compiler.
* Add test cases based on OpenAPI examples.
* Add MustCompile to replace the original MustParse, and update usage.
2021-11-29 13:08:48 -08:00
Sam Kleinman
6ab62fe7b6 service: remove stop method and use contexts (#7292) 2021-11-18 17:56:21 -05:00
Sam Kleinman
d7606777cf libs/service: pass logger explicitly (#7288)
This is a very small change, but removes a method from the
`service.Service` interface (a win!) and forces callers to explicitly
pass loggers in to objects during construction rather than (later)
injecting them. There's not a real need for this kind of lazy
construction of loggers, and I think a decent potential for confusion
for mutable loggers.

The main concern I have is that this changes the constructor API for
ABCI clients. I think this is fine, and I suspect that as we plumb
contexts through, and make changes to the RPC services there'll be a
number of similar sorts of changes to various (quasi) public
interfaces, which I think we should welcome.
2021-11-16 16:20:56 +00:00
M. J. Fromberger
d5865af1f4 Add basic metrics to the indexer package. (#7250)
This follows the same model as we did in the p2p package.

Rework the indexer service constructor to take a struct of arguments,
that makes it easier to construct the optional settings.
Deprecate but do not remove the existing constructor.

Clean up node initialization a little bit.
2021-11-05 12:50:53 -07:00
M. J. Fromberger
54d7030510 pubsub: Move indexing out of the primary subscription path (#7231)
This is part of the work described by #7156.

Remove "unbuffered subscriptions" from the pubsub service.
Replace them with a dedicated blocking "observer" mechanism.
Use the observer mechanism for indexing.

Add a SubscribeWithArgs method and deprecate the old Subscribe
method. Remove SubscribeUnbuffered entirely (breaking).

Rework the Subscription interface to eliminate exposed channels.
Subscriptions now use a context to manage lifecycle notifications.

Internalize the eventbus package.
2021-11-05 10:25:25 -07:00
Sam Kleinman
93eb940dcd config: WriteConfigFile should return error (#7169) 2021-10-27 08:46:18 -04:00
Sam Kleinman
bb8ffcb95b store: move pacakge to internal (#6978) 2021-09-23 11:46:42 -04:00
Sam Kleinman
1c4950dbd2 state: move package to internal (#6964) 2021-09-22 13:04:25 -04:00
Sam Kleinman
07d10184a1 inspect: remove duplicated construction path (#6966) 2021-09-22 12:32:49 -04:00