Although we index block.height for blocks in the KV indexer, this reserved
attribute was not previously exposed to the event subscription API. Despite
being advertised in the OpenAPI spec, neither the old (websocket) nor new
(events) query interface could see it. This change exposes block.height to the
/events API.
In addition: Remove a non-public constant from types (finalize_block). This
value is used only as an internal tag by the indexer, and should not be exposed
to users of the public interface. (We could probably drop it entirely, as it
was previously a disambiguator for BeginBlock vs. EndBlock events, but keeping
a tag here simplifies the cleanup).
This change set implements the most recent version of `FinalizeBlock`.
# What does this change actually contain?
* This change set is rather large but fear not! The majority of the files touched and changes are renaming `ResponseDeliverTx` to `ExecTxResult`. This should be a pretty inoffensive change since they're effectively the same type but with a different name.
* The `execBlockOnProxyApp` was totally removed since it served as just a wrapper around the logic that is now mostly encapsulated within `FinalizeBlock`
* The `updateState` helper function has been made a public method on `State`. It was being exposed as a shim through the testing infrastructure, so this seemed innocuous.
* Tests already existed to ensure that the application received the `ByzantineValidators` and the `ValidatorUpdates`, but one was fixed up to ensure that `LastCommitInfo` was being sent across.
* Tests were removed from the `psql` indexer that seemed to search for an event in the indexer that was not being created.
# Questions for reviewers
* We store this [ABCIResponses](5721a13ab1/proto/tendermint/state/types.pb.go (L37)) type in the data base as the block results. This type has changed since v0.35 to contain the `FinalizeBlock` response. I'm wondering if we need to do any shimming to keep the old data retrieveable?
* Similarly, this change is exposed via the RPC through [ResultBlockResults](5721a13ab1/rpc/coretypes/responses.go (L69)) changing. Should we somehow shim or notify for this change?
closes: #7658
This method implements the eventlog extension interface to expose ABCI metadata
to the log for query processing. Only the types that have ABCI events need to
implement this.
- Add an event log to the environment
- Add a sketch of the handler method
- Add an /events RPCFunc to the route map
- Implement query logic
- Subscribe to pubsub if confingured, handle termination
* event: Added Events after evidence validation; evidence: refactored AddEvidence
Added context and Metrics as parameter for the pool constructor
* evidence: pushed event firing into evidence pool and added metrics to represent the size of the evpool
* state: fixed parameters of evpool mock functions
* evidence: added test to confirm events are generated
* Removed obsolete EvidenceEventPublisher interface
* evidence: pool removed error on missing eventbus
* Rebased and git-squashed the commits in PR #6546
migrate abci to finalizeBlock
work on abci, proxy and mempool
abciresponse, blok events, indexer, some tests
fix some tests
fix errors
fix errors in abci
fix tests amd errors
* Fixes after rebasing PR#6546
* Restored height to RequestFinalizeBlock & other
* Fixed more UTs
* Fixed kvstore
* More UT fixes
* last TC fixed
* make format
* Update internal/consensus/mempool_test.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Addressed @williambanfield's comments
* Fixed UTs
* Addressed last comments from @williambanfield
* make format
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
This pull request merges in the changes for implementing Proposer-based timestamps into `master`. The power was primarily being done in the `wb/proposer-based-timestamps` branch, with changes being merged into that branch during development. This pull request represents an amalgamation of the changes made into that development branch. All of the changes that were placed into that branch have been cleanly rebased on top of the latest `master`. The changes compile and the tests pass insofar as our tests in general pass.
### Note To Reviewers
These changes have been extensively reviewed during development. There is not much new here. In the interest of making effective use of time, I would recommend against trying to perform a complete audit of the changes presented and instead examine for mistakes that may have occurred during the process of rebasing the changes. I gave the complete change set a first pass for any issues, but additional eyes would be very appreciated.
In sum, this change set does the following:
closes#6942
merges in #6849
This is the interface shared by types that can be used as event data in, for
example, subscriptions via the RPC.
To be compatible with the RPC service, data need to support JSON encoding.
Require this as part of the interface.
Remove the pubsub.Query interface and instead use the concrete query type.
Nothing uses any other implementation but pubsub/query.
* query: remove the error from the Matches method
* Update all usage.
The main change here is to use encoding/json to encode and decode RPC
parameters, rather than the custom tmjson package. This includes:
- Update the HTTP POST handler parameter handling.
- Add field tags to 64-bit integer types to get string encoding (to match amino/tmjson).
- Add marshalers to struct types that mention interfaces.
- Inject wrappers to decode interface arguments in RPC handlers.
Rework the implementation of event query parsing and execution to
improve performance and reduce memory usage.
Previous memory and CPU profiles of the pubsub service showed query
processing as a significant hotspot. While we don't have evidence that
this is visibly hurting users, fixing it is fairly easy and self-contained.
Updates #6439.
Typical benchmark results comparing the original implementation (PEG) with the reworked implementation (Custom):
```
TEST TIME/OP BYTES/OP ALLOCS/OP SPEEDUP MEM SAVING
BenchmarkParsePEG-12 51716 ns 526832 27
BenchmarkParseCustom-12 2167 ns 4616 17 23.8x 99.1%
BenchmarkMatchPEG-12 3086 ns 1097 22
BenchmarkMatchCustom-12 294.2 ns 64 3 10.5x 94.1%
```
Components:
* Add a basic parsing benchmark.
* Move the original query implementation to a subdirectory.
* Add lexical scanner for Query expressions.
* Add a parser for Query expressions.
* Implement query compiler.
* Add test cases based on OpenAPI examples.
* Add MustCompile to replace the original MustParse, and update usage.
closes#2498
solves part of #3365
Note: difficult to test the event emit in SwitchToFastSync part, might need to change `stateSyncReactor` to an interface in the `nodeImpl` struct
Migrates the `rpc` package to use new JSON encoder in #4955. Branched off of that PR.
Tests pass, but I haven't done any manual testing beyond that. This should be handled as part of broader 0.34 testing.
* format: add format cmd & goimport repo
- replaced format command
- added goimports to format command
- ran goimports
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* fix outliers & undo proto file changes
* Rename Tag(s) to Event(s)
- tag was replaced with event, but in some places it still mentions tag, would be easier to understand if we tried to replace it with event to not confuse people.
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* more changes from tag -> event
* rename events to compositeKeys and keys
* Apply suggestions from code review
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* add minor documentation on how composite keys are constructed
* rename eventkey to compositekey
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* add changelog entry & add info to regenerate confid to changelog entry
* Removal of TotalTx & NumTx
- Removed totalTx and numTx
closes#2521
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* abci proto changes
* proto number fix
* txfilter_test fix
* comments on PR
* further changes
* bring back metrics
* fix indexer
* fix TestBlockMaxDataBytes and TestBlockMaxDataBytesUnknownEvidence
* indexer service back to header
* statistics.go fix
* fix ci
* listen for blocks, not headers
to be able to record txs throughput
* fix TestNetworkNewBlock
* fix tests
* fix tests in types package
* fixes after Anton's review
* fix tests
* bring back `consensus_total_txs` metric
I mistakenly thought it was removed.
* improve changelog
* remove LastBlockTotalTx from state
* docs: remove getNumTxs from BeginBlock Java example
Before we're using it to get a round state in tests. Now it can be done
by calling csX.GetRoundState. We will need to rewrite
TestStateSlashingPrevotes and TestStateSlashingPrecommits, which are
commented right now, to not rely on EventDataRoundState#RoundState
field.
Refs #1527
* add proposer info to EventCompleteProposal
* create separate EventData structure for CompleteProposal
* cant us rs.Proposal to get BlockID because it is not guaranteed to be set yet
* copying RoundState isnt helping us here
* add Step back to make compatible with original RoundState event. update changelog
* add NewRound event
* fix test
* remove unneeded RoundState
* put height round step into a struct
* pull out ValidatorInfo struct. add ensureProposal assert
* remove height-round-state sub-struct refactor
* minor fixes from review
* Introduce EventValidBlock for informing peer about wanted block
* Merge with develop
* Add isCommit flag to NewValidBlock message
- Add test for the case of +2/3 Precommit from the previous round
