The `mockery` project recommends against using a binary of `mockery` that has been created using `go install`. https://github.com/vektra/mockery/pull/456. Developers of Tendermint wishing to generate mocks should avoid having a version of `mockery` on their path that does not match the version listed in [mockery_generate.sh](10e1ac8fea/scripts/mockery_generate.sh (L11)). To make this easier for developers, the `mockery_generate.sh` script uses a containerized copy of `mockery` if `mockery` is not present on the developer's `PATH`. This containerized version of `mockery` uses the same version of mockery as our CI pipelines and allows all developers to automatically use the same version without having to manage it themselves.
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
The v0.34.x tests have been failing (or reporting failures, I don't
believe that this is a real failure,) because the CI configuration has
been out of sync with itself, likely due to a mistake during
backporting configs from the `master` branch.
The entire 0.34.x e2e test suite takes 26 minutes to run, plus about 7
minutes to build the docker image. Each split has to build the same
docker image, (and is therefore a cap on the amount of parallelism we
can get at the moment.) Having more groups, just seems like we'll be
burning money building the docker image with no really meaningful
difference in throughput. For a nightly test that people don't really
wait on, the current latency (time-to-completion) of roughly 19
minutes, isn't causing in friction.
remove gas from merklization in headers
Im not sure where to change docs since main points to the spec repo but that repo is archived. Maybe someone can help me?
This pull request completes the change to the `metricsgen` metrics. It adds `go generate` directives to all of the files containing the `Metrics` structs.
Using the outputs of `metricsdiff` between these generated metrics and `main`, we can see that there is a minimal diff between the two sets of metrics when run locally. The diff here stems from removal of the word 'message' which was done in v0.36+ and is ultimately a better phrasing. This metric has not yet been released, so this phrasing is preferred.
```
./metricsdiff old new
Metric changes:
+++ tendermint_consensus_full_prevote_delay
+++ tendermint_consensus_quorum_prevote_delay
--- tendermint_consensus_full_prevote_message_delay
--- tendermint_consensus_quorum_prevote_message_delay
```
This change also adds parsing for a `metrics:` key in a field comment. If a comment line begins with `//metrics:` the rest of the line is interpreted to be the metric help text. Additionally, a bug where lists of labels were not properly quoted in the `metricsgen` rendered output was fixed.
In my view, docs and tests are not needed for this internal only change.
---
#### PR checklist
- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
documentation updates needed
Ports #8488 to main
Update the nightly E2E workflows to fix the notifications for the
v0.34.x branch while also simplifying the messages and making them more
readable.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* internal/proxy: add initial set of abci metrics (#7115)
This PR adds an initial set of metrics for use ABCI. The initial metrics enable the calculation of timing histograms and call counts for each of the ABCI methods. The metrics are also labeled as either 'sync' or 'async' to determine if the method call was performed using ABCI's `*Async` methods.
An example of these metrics is included here for reference:
```
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.0001"} 0
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.0004"} 5
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.002"} 12
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.009"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.02"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.1"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="0.65"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="2"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="6"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="25"} 13
tendermint_abci_connection_method_timing_bucket{chain_id="ci",method="commit",type="sync",le="+Inf"} 13
tendermint_abci_connection_method_timing_sum{chain_id="ci",method="commit",type="sync"} 0.007802058000000001
tendermint_abci_connection_method_timing_count{chain_id="ci",method="commit",type="sync"} 13
```
These metrics can easily be graphed using prometheus's `histogram_quantile(...)` method to pick out a particular quantile to graph or examine. I chose buckets that were somewhat of an estimate of expected range of times for ABCI operations. They start at .0001 seconds and range to 25 seconds. The hope is that this range captures enough possible times to be useful for us and operators.
* fixup
* fixups
* docs: add abci timing metrics to the metrics docs (#7311)
* format table
This is a port of #8470 and #8501 from the former mainline branch, in
support of #2600 and #9076.
No additional changes other than checking out the code have been
implemented in this PR. A subsequent PR will incorporate this code.
* docs/rfc: add testnet RFC
Following several discussions internal to the Tendermint engineering
team, I am posting an RFC discussing the high-level details of the
Tendermint team owning and operating a long-lived testnet in order to
build experience running Tendermint, and to demonstrate that Tendermint
is stable under production workloads.
The outcome of this RFC will be a new track of work to begin building
and maintaining a testnet associated with the main branch of tendermint.
See the "Testnet MVP" section specifically for some of the first
milestones.
Note, I added the RFC where it would live once #9115 is merged to
restore the RFC layout from the v0.36.x branch. docs/rfc/README.md will
need to be updated to include this RFC once #9115 is merged.
This RFC is related to #9078.
* docs/rfc: minor updates to testnet rfc
* docs/rfc: respond to more feedback on testnet RFC
* docs/rfc: add RFC 023 to rfc index
Bumps [github.com/bufbuild/buf](https://github.com/bufbuild/buf) from 1.4.0 to 1.7.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/bufbuild/buf/releases">github.com/bufbuild/buf's releases</a>.</em></p>
<blockquote>
<h2>v1.7.0</h2>
<ul>
<li>Support protocol and encoding client options based on content-type in Studio Agent.</li>
<li>Add <code>--draft</code> flag to <code>buf push</code>.</li>
<li>Add <code>buf beta registry draft {list,delete}</code> commands.</li>
</ul>
<h2>v1.6.0</h2>
<ul>
<li>Fix issue where <code>// buf:lint:ignore</code> comment ignores did not work for the
<code>ENUM_FIRST_VALUE_ZERO</code> rule.</li>
<li>Add <code>buf beta studio-agent</code> command to support the upcoming Buf Studio.</li>
</ul>
<h2>v1.5.0</h2>
<ul>
<li>Upgrade to <code>protoc</code> 3.20.1 support.</li>
<li>Fix an issue where <code>buf</code> would fail if two or more roots contained
a file with the same name, but with different file types (i.e. a
regular file vs. a directory).</li>
<li>Fix check for <code>PACKAGE_SERVICE_NO_DELETE</code> to detect deleted services.</li>
<li>Remove <code>buf beta registry track</code>.</li>
<li>Remove <code>buf beta registry branch</code>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/bufbuild/buf/blob/main/CHANGELOG.md">github.com/bufbuild/buf's changelog</a>.</em></p>
<blockquote>
<h2>[v1.7.0] - 2022-06-27</h2>
<ul>
<li>Support protocol and encoding client options based on content-type in Studio Agent.</li>
<li>Add <code>--draft</code> flag to <code>buf push</code>.</li>
<li>Add <code>buf beta registry draft {list,delete}</code> commands.</li>
</ul>
<h2>[v1.6.0] - 2022-06-21</h2>
<ul>
<li>Fix issue where <code>// buf:lint:ignore</code> comment ignores did not work for the
<code>ENUM_FIRST_VALUE_ZERO</code> rule.</li>
<li>Add <code>buf beta studio-agent</code> command to support the upcoming Buf Studio.</li>
</ul>
<h2>[v1.5.0] - 2022-05-30</h2>
<ul>
<li>Upgrade to <code>protoc</code> 3.20.1 support.</li>
<li>Fix an issue where <code>buf</code> would fail if two or more roots contained
a file with the same name, but with different file types (i.e. a
regular file vs. a directory).</li>
<li>Fix check for <code>PACKAGE_SERVICE_NO_DELETE</code> to detect deleted services.</li>
<li>Remove <code>buf beta registry track</code>.</li>
<li>Remove <code>buf beta registry branch</code>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="028fdd557d"><code>028fdd5</code></a> Update to v1.7.0</li>
<li><a href="c6a545ecc0"><code>c6a545e</code></a> Run <code>make upgrade</code> (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1302">#1302</a>)</li>
<li><a href="65503feaf3"><code>65503fe</code></a> allow Dockerfile.plugin or Dockerfile (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1301">#1301</a>)</li>
<li><a href="075e294d6a"><code>075e294</code></a> add CLI support for repository draft commit (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1269">#1269</a>)</li>
<li><a href="3d2ca26216"><code>3d2ca26</code></a> ModuleReferenceForString support reference with slash (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1298">#1298</a>)</li>
<li><a href="72b51ab871"><code>72b51ab</code></a> rename CuratedPlugin options to default_options (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1300">#1300</a>)</li>
<li><a href="65c921b0b7"><code>65c921b</code></a> rename opts/options to default_opts/default_options (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1299">#1299</a>)</li>
<li><a href="0a8aec22c7"><code>0a8aec2</code></a> BSR-379: document version can be optional (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1297">#1297</a>)</li>
<li><a href="33033b8fc3"><code>33033b8</code></a> BSR-369: update plugin dependencies to a struct (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1296">#1296</a>)</li>
<li><a href="613283806d"><code>6132838</code></a> Add webhook delete and list commands to beta and fixup create (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1293">#1293</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/bufbuild/buf/compare/v1.4.0...v1.7.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6.
<details>
<summary>Commits</summary>
<ul>
<li><a href="7efb22a518"><code>7efb22a</code></a> 1.2.6</li>
<li><a href="ef88b9325f"><code>ef88b93</code></a> security notice for additional prototype pollution issue</li>
<li><a href="c2b981977f"><code>c2b9819</code></a> isConstructorOrProto adapted from PR</li>
<li><a href="bc8ecee438"><code>bc8ecee</code></a> test from prototype pollution PR</li>
<li>See full diff in <a href="https://github.com/substack/minimist/compare/1.2.5...1.2.6">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/tendermint/tendermint/network/alerts).
</details>
Import the readme, contributing guidelines, code of conduct, security
guide and releases guide. Format all of these documents, replacing
references to `master` by references to `main`, and use Markdown link
references instead of embedded links wherever it improves legibility.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update CODEOWNERS to use teams
Update the `CODEOWNERS` file to use the
@tendermint/tendermint-engineering and @tendermint/tendermint-research
teams as opposed to adding people one by one. This makes repository
administration somewhat easier to manage, especially when
onboarding/offboarding people.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add Ethan as superuser
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* libs/pubsub/query: specify peg version in go.mod
The code to generate the pubsub queries was dependent on an unspecified
version of the peg tool. This brings peg into go.mod so it is on a fixed
version. This should also enable dependabot to notify us of future
updates to peg.
The version of query.peg.go generated from the current version of peg
correctly contains the special "Code generated by..." line to indicate
to other tools that the file is automatically generated and should
therefore be excluded from linters, etc.
I removed the make target as there were no git grep results referencing
"gen_query_parser"; directly running "go generate" is a reasonable
expectation in Go projects.
Now that "go run" is module aware, I would typically use "go run" inside
the go:generate directive, but in this case we go build to a gitignore-d
directory in order to work around the nondeterministic output detailed
in pointlander/peg#129.
* libs/pubsub/query: check error from (*QueryParser).Init()
The newly generated peg code returns an error from Init(); the previous
version was niladic.
Co-authored-by: Sam Kleinman <garen@tychoish.com>
This test would fail if run with "go test -count=2" because it uses a
fixed address and was not closing the server, so the subsequent run
could not bind to the address.
While closing the server is correct, it would probably be better if the
API was able to report the bound address so that we could pass
"localhost:0" for an anonymous port. But I am currently focusing on test
cleanup, not ready to change any existing APIs.
