Performance improvements for the event query API (#7319)

Rework the implementation of event query parsing and execution to
improve performance and reduce memory usage.

Previous memory and CPU profiles of the pubsub service showed query
processing as a significant hotspot. While we don't have evidence that
this is visibly hurting users, fixing it is fairly easy and self-contained.

Updates #6439.

Typical benchmark results comparing the original implementation (PEG) with the reworked implementation (Custom):
```
TEST                        TIME/OP  BYTES/OP  ALLOCS/OP  SPEEDUP   MEM SAVING
BenchmarkParsePEG-12       51716 ns  526832    27
BenchmarkParseCustom-12     2167 ns    4616    17         23.8x     99.1%
BenchmarkMatchPEG-12        3086 ns    1097    22
BenchmarkMatchCustom-12    294.2 ns      64     3         10.5x     94.1%
```

Components:
* Add a basic parsing benchmark.
* Move the original query implementation to a subdirectory.
* Add lexical scanner for Query expressions.
* Add a parser for Query expressions.
* Implement query compiler.
* Add test cases based on OpenAPI examples.
* Add MustCompile to replace the original MustParse, and update usage.
This commit is contained in:
M. J. Fromberger
2021-11-29 13:08:48 -08:00
committed by GitHub
parent d1f07047ec
commit 1dca1a8f97
34 changed files with 2218 additions and 830 deletions
+10
View File
@@ -0,0 +1,10 @@
gen_query_parser:
go generate .
fuzzy_test:
go get -u -v github.com/dvyukov/go-fuzz/go-fuzz
go get -u -v github.com/dvyukov/go-fuzz/go-fuzz-build
go-fuzz-build github.com/tendermint/tendermint/libs/pubsub/query/fuzz_test
go-fuzz -bin=./fuzz_test-fuzz.zip -workdir=./fuzz_test/output
.PHONY: gen_query_parser fuzzy_test