I observed a couple of problems with the generator in some recent tests:
- there were a couple of hybrid test cases which did not have any
legacy nodes (randomness and all.) I change the probability to
produce more reliable results.
- added options to the generation to be able to add a max (to
compliment the earlier min) number of nodes for local testing.
- added an option to support reversing the sort order so "more
complex" networks were first, as well as tweaked some of the point
values.
- this refactored the generators cli parsing to be a bit more clear.
The main effect of this change is to flush the socket client and server message
encoding buffers immediately once the message is fully and correctly encoded.
This allows us to remove the timer and some other special cases, without
changing the observed behaviour of the system.
-- Background
The socket protocol client and server each use a buffered writer to encode
request and response messages onto the underlying connection. This reduces the
possibility of a single message being split across multiple writes, but has the
side-effect that a request may remain buffered for some time.
The implementation worked around this by keeping a ticker that occasionally
triggers a flush, and by flushing the writer in response to an explicit request
baked into the client/server protocol (see also #6994).
These workarounds are both unnecessary: Once a message has been dequeued for
sending and fully encoded in wire format, there is no real use keeping all or
part of it buffered locally. Moreover, using an asynchronous process to flush
the buffer makes the round-trip performance of the request unpredictable.
-- Benchmarks
Code: https://play.golang.org/p/0ChUOxJOiHt
I found no pre-existing performance benchmarks to justify the flush pattern,
but a natural question is whether this will significantly harm client/server
performance. To test this, I implemented a simple benchmark that transfers
randomly-sized byte buffers from a no-op "client" to a no-op "server" over a
Unix-domain socket, using a buffered writer, both with and without explicit
flushes after each write.
As the following data show, flushing every time (FLUSH=true) does reduce raw
throughput, but not by a significant amount except for very small request
sizes, where the transfer time is already trivial (1.9μs). Given that the
client is calibrated for 1MiB transactions, the overhead is not meaningful.
The percentage in each section is the speedup for flushing only when the buffer
is full, relative to flushing every block. The benchmark uses the default
buffer size (4096 bytes), which is the same value used by the socket client and
server implementation:
FLUSH NBLOCKS MAX AVG TOTAL ELAPSED TIME/BLOCK
false 3957471 512 255 1011165416 2.00018873s 505ns
true 1068568 512 255 273064368 2.000217051s 1.871µs
(73%)
false 536096 4096 2048 1098066401 2.000229108s 3.731µs
true 477911 4096 2047 978746731 2.000177825s 4.185µs
(10.8%)
false 124595 16384 8181 1019340160 2.000235086s 16.053µs
true 120995 16384 8179 989703064 2.000329349s 16.532µs
(2.9%)
false 2114 1048576 525693 1111316541 2.000479928s 946.3µs
true 2083 1048576 526379 1096449173 2.001817137s 961.025µs
(1.5%)
Note also that the FLUSH=false baseline is actually faster than the production
code, which flushes more often than is required by the buffer filling up.
Moreover, the timer slows down the overall transaction rate of the client and
server, indepenedent of how fast the socket transfer is, so the loss on a real
workload is probably much less.
I've been noticing that there are a number of situations where the
statesync reactor blocks waiting for peers (or similar,) I've moved
things around to improve outcomes in local tests.
In the last run, there were two problems at the RPC layer returned
from light nodes' RPC end points. I think exercising the light client
proxy RPC system is something that can/should be done via unit
testing, and that likely these errors are (in production) transient
and (in CI) very likely to fail for test environment issues.
The code in the Tendermint repository makes heavy use of import aliasing.
This is made necessary by our extensive reuse of common base package names, and
by repetition of similar names across different subdirectories.
Unfortunately we have not been very consistent about which packages we alias in
various circumstances, and the aliases we use vary. In the spirit of the advice
in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports,
his change makes an effort to clean up and normalize import aliasing.
This change makes no API or behavioral changes. It is a pure cleanup intended
o help make the code more readable to developers (including myself) trying to
understand what is being imported where.
Only unexported names have been modified, and the changes were generated and
applied mechanically with gofmt -r and comby, respecting the lexical and
syntactic rules of Go. Even so, I did not fix every inconsistency. Where the
changes would be too disruptive, I left it alone.
The principles I followed in this cleanup are:
- Remove aliases that restate the package name.
- Remove aliases where the base package name is unambiguous.
- Move overly-terse abbreviations from the import to the usage site.
- Fix lexical issues (remove underscores, remove capitalization).
- Fix import groupings to more closely match the style guide.
- Group blank (side-effecting) imports and ensure they are commented.
- Add aliases to multiple imports with the same base package name.
We moved some files further down in the directory structure in #6964, which
caused the relative paths to the mockery wrapper to stop working.
There does not seem to be an obvious way to get the module root as a default
environment variable, so for now I just added the extra up-slashes.
This package is not used in the tendermint repository since 31e7cdee.
Note that this is not the same package as rpc/client/mock (N.B. singular) which
is still used in some tests.
A search of GitHub turns up only 11 uses, all of which are in clones of the
tendermint repo at old commits..
* rpc: Strip down the base RPC client interface.
Prior to this change, the RPC client interface requires implementing the entire
Service interface, but most of the methods of Service are not needed by the
concrete clients. Dissociate the Client interface from the Service interface.
- Extract only those methods of Service that are necessary to make the existing
clients work.
- Update the clients to combine Start/Onstart and Stop/OnStop. This does not
change what the clients do to start or stop. Only the websocket clients make
use of this functionality anyway.
The websocket implementation uses some plumbing from the BaseService helper.
We should be able to excising that entirely, but the current interface
dependencies among the clients would require a much larger change, and one
that leaks into other (non-RPC) packages.
As a less-invasive intermediate step, preserve the existing client behaviour
(and tests) by extracting the necessary subset of the BaseService
functionality to an analogous RunState helper for clients. I plan to obsolete
that type in a future PR, but for now this makes a useful waypoint.
Related:
- Clean up client implementations.
- Update mocks.
## Description
- Add deprecated to config values in toml
- update config in configuration doc
- explain how to set up a node with the new network
- add sentence about not needing to fork tendermint for built-in tutorial
- closes#6865
- add note to use a released version of tendermint with the tutorials. This is to avoid unknown issues prior to a release.
If the e2e tests error, they leave all of the e2e state around including containers and networks etc.
We should clean this up when the tests shuts down, even if it exits in error.
This should address last night's failure. We've taken the perspective
of "the load generator shouldn't cause tests to fail" in recent
days/weeks, and I think this is just a next step along that line. The
e2e tests shouldn't test performance.
I included some comments indicating the ways that this isn't ideal (it
is perhaps not), and I think that if test networks could make
assertions about the required rate, that might be a cool future
improvement (and good, perhaps, for system benchmarking.)
This document attempts to capture and discuss some of the areas of Tendermint that seem to be cited as causing performance issue. I'm hoping to continue to gather feedback and input on this document to better understand what issues Tendermint performance may cause for our users.
The overall goal of this document is to allow the maintainers and community to get a better sense of these issues and to be more capably able to discuss them and weight trade-offs about any proposed performance-focused changes. This document does not aim to propose any performance improvements. It does suggest useful places for benchmarks and places where additional metrics would be useful for diagnosing and further understanding Tendermint performance.
Please comment with areas where my reasoning seems off or with additional areas that Tendermint performance may be causing user pain.
I think the `Sync` check covers our primary use case, and perhaps we
can turn this back on in the future after some kind of event-system
rewrite, or RPC rewrite that will avoid the serverside timeout.
These are mostly the timeouts that I think we're still hitting in CI.
At this point, the tests (on master) pass on my local machine (which is quite beefy) so I think this is just the first in (perhaps?) a sequence of changes that attempt to change timeouts and load patterns so that the tests pass in CI more reliably.
Communication in Tendermint among consensus nodes, applications, and operator
tools all use different message formats and transport mechanisms. In some
cases there are multiple options. Having all these options complicates both the
code and the developer experience, and hides bugs. To support a more robust,
trustworthy, and usable system, we should document which communication paths
are essential, which could be removed or reduced in scope, and what we can
improve for the most important use cases.
This document proposes a variety of possible improvements of varying size and
scope. Specific design proposals should get their own documentation.
The previous implemention of hybrid set testing, which was entirely my
own creation, was a bit peculiar, and I think this probably clears thins up.
The previous implementation had far fewer legacy nodes in hybrid
networks, *and* also for some reason that I can't quite explain,
caused a test case to fail.
Bumps [github.com/rs/zerolog](https://github.com/rs/zerolog) from 1.24.0 to 1.25.0.
<details>
<summary>Commits</summary>
<ul>
<li><a href="65adfd88ec"><code>65adfd8</code></a> Make Fields method accept both map and slice (<a href="https://github-redirect.dependabot.com/rs/zerolog/issues/352">#352</a>)</li>
<li>See full diff in <a href="https://github.com/rs/zerolog/compare/v1.24.0...v1.25.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
The responses from node RPCs encode hash values as hexadecimal strings. This
behaviour is stipulated in our OpenAPI documentation. In some cases, however,
hashes received as JSON parameters were being decoded as byte buffers, as is
the convention for JSON.
This resulted in the confusing situation that a hash reported by one request
(e.g., broadcast_tx_commit) could not be passed as a parameter to another
(e.g., tx) via JSON, without translating the hex-encoded output hash into the
base64 encoding used by JSON for opaque bytes.
Fixes#6802.
The 0.35 release cycle renamed the 'fastsync' functionality to 'blocksync'. This change brings the configuration parameters in line with that change. Namely, it updates the configuration file `[fastsync]` field to be `[blocksync]` and changes the command line flag and config file parameters `--fast-sync` and `fast-sync` to `--enable-block-sync` and `enable-block-sync` respectively.
Error messages were added to help users encountering these changes be able to quickly make the needed update to their files/scripts.
When using the old command line argument for fast-sync, the following is printed
```
./build/tendermint start --proxy-app=kvstore --consensus.create-empty-blocks=false --fast-sync=false
ERROR: invalid argument "false" for "--fast-sync" flag: --fast-sync has been deprecated, please use --enable-block-sync
```
When using one of the old config file parameters, the following is printed:
```
./build/tendermint start --proxy-app=kvstore --consensus.create-empty-blocks=false
ERROR: error in config file: a configuration parameter named 'fast-sync' was found in the configuration file. The 'fast-sync' parameter has been renamed to 'enable-block-sync', please update the 'fast-sync' field in your configuration file to 'enable-block-sync'
```
* add information on upgrading to the new p2p library
* clarify p2p backwards compatibility
* reorder p2p queue list
* add demo for p2p selection
* fix spacing in upgrading
This is a cosmetic change that restores lexicographic order to the selected
linters in the CI config. No change to which linters we run, only putting them
back in order so it's easier to spot the one you care about.
This changes the focus of the e2e suite, to (roughly) focus on
configurations that are more well used. Most production users of
tendermint run ABCI application in process and the GRPC/socket methods
cover the vast majority of the remaining use cases.
Perhaps we should consider drop support unix domain sockets in a
future release, but I think in the mean time it's useful to have the
tests *mostly* focus on the primary use cases.
Update the schema and implementation of the Postgres event indexer to improve
certain types of queries against the index. These changes address the use cases
raised by #6843, and are partly inspired by the prototype schema in that issue.
In the old schema, events were flattened, making it difficult to find all the events
associated with a particular block or transaction. In addition, events with no key/value
attributes were entirely lost, since entries were generated only for attributes.
To address these issues, this new schema records blocks, transactions, events,
and attributes in separate tables, and provides views that join these tables to
give a more convenient query surface for block and transaction events.
- All events for a given block can be queried from the `block_events` view.
- All events for a given transaction can be queried from the `tx_events` view.
- Multiple events for the same key can be indexed for both blocks and transactions.
The tests have been reworked, but all of the existing test cases for the old schema
still pass with the new implementation. Various other minor cleanups are included,
ADR-065 is also updated to reflect the updated schema.
An explicit exit prevents the deferred cleanup code from running. In this case,
falling off the end of main will achieve the same goal as an explicit exit.
When revwing #6807 I assumed that `probSetChoice` worked this way.
I think that the coverage of various configuration options should
generally track what we expect the actual useage to be to focus the
most test coverage on the configurations that are the most prevelent.
Issues reported in Osmosis, where the message is extremely long. Also, there is absolutely no reason to log the message IMO. If we must, we can make the message log DEBUG.
This is just a configuration change to default to using the new stack
unless explicitly disabled (e.g. `UseLegacy`) this renames the
configuration value and makes the configuration logic more clear.
The legacy option is good to retain as a fallback if the new stack has
issues operationally, but we should make sure that most of the time
we're using the new stack.
In the transaction load generator, the e2e test harness previously distributed load randomly to hosts, which was a source of test non-determinism. This change distributes the load generation to the different nodes in the set in a round robin fashion, to produce more reliable results, but does not otherwise change the behavior of the test harness.
Add documentation comments to the psql event sink package, and simplify the
constructor function so that it does not return the SQL database handle. The
handle is needed for testing, so expose that via a separate method on the
concrete type.
Update the tests and existing usage for the change. This change does not affect
the behaviour of the sink, so there are no functional changes, only syntactic
updates.
EDIT: Updated, see [comment below]( https://github.com/tendermint/tendermint/pull/6785#issuecomment-897793175)
This change adds a sketch of the `Debug` mode.
This change adds a `Debug` struct to the node package. This `Debug` struct is intended to be created and started by a command in the `cmd` directory. The `Debug` struct runs the RPC server on the data directories: both the state store and the block store.
This change required a good deal of refactoring. Namely, a new `rpc.go` file was added to the `node` package. This file encapsulates functions for starting RPC servers used by nodes. A potential additional change is to further factor this code into shared code _in_ the `rpc` package.
Minor API tweaks were also made that seemed appropriate such as the mechanism for fetching routes from the `rpc/core` package.
Additional work is required to register the `Debug` service as a command in the `cmd` directory but I am looking for feedback on if this direction seems appropriate before diving much further.
closes: #5908
This is a very minor change, but I was looking through the code, and
this seems like it shouldn't be exported or used more broadly, so I've
moved it out.
This ADR restores a variation of the old Request for Comments documentation
that we previously used. The proposal differs from the original formulation,
and does not replace ADRs.
I realized after my last commit that my change made a following line of code a bit redundant.
(alternatively my last change was redunadnt to the existing code.)
I took this oppertunity to make some minor cleanups and logging changes to the node changes which I hope will make tests a bit more clear.
* docs: add indexer godoc
* docs++
* docs++
* docs++
* docs++
* docs++
* Update state/indexer/doc.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update state/indexer/doc.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update state/indexer/doc.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update state/indexer/doc.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update state/indexer/doc.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update state/indexer/doc.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* docs++
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
We INFO log every `ABCIQuery`. This can output a tremendous amount of noise in the logs, can cause cosmovisor to completely crash and slows down the node due to I/O. This log is completely unnecessary.
Let's get this backported into v0.43 and get that into v0.43 and v0.42 releases of the SDK
/cc @marbar3778
As written, the encoding step unnecessarily made and moved multiple copies of
the encoded representation. Reduce this to a single allocation and encode the
data in-place so that a shift is no longer required.
Also: Add a test to ensure letter digits are capitalized, which was previously not
verified but was expected downstream.
No functional changes.
Adds a simple property test to the `clist` package. This test uses the [rapid](https://github.com/flyingmutant/rapid) library and works by modeling the internal clist as a simple array.
Follow up from this mornings workshop with the Regen team.
This changes adds a failing test for issue #6660. It achieves this by adding a transaction, starting the `broadcastTxRoutine` in a goroutine and then adding another transaction to the mempool. The `broadcastTxRoutine` can receive the second inserted transaction before `insertTx` returns. In that case, `broadcastTxRoutine` will derefence a nil pointer when referencing the `gossipEl` and panic.
This change aims to keep versions of mockery consistent across developer laptops.
This change adds mockery to the `tools.go` file so that its version can be managed consistently in the `go.mod` file.
Additionally, this change temporarily disables adding mockery's version number to generated files. There is an outstanding issue against the mockery project related to the version string behavior when running from `go get`. I have created a pull request to fix this issue in the mockery project.
see: https://github.com/vektra/mockery/issues/397
Update those break statements inside case clauses that are intended to reach an
enclosing for loop, so that they correctly exit the loop.
The candidate files for this change were located using:
% staticcheck -checks SA4011 ./... | cut -d: -f-2
This change is intended to preserve the intended semantics of the code, but
since the code as-written did not have its intended effect, some behaviour may
change. Specifically: Some loops may have run longer than they were supposed
to, prior to this change.
In one case I was not able to clearly determine the intended outcome. That case
has been commented but otherwise left as-written.
Fixes#6780.
This pull request removes the homegrown mocks in `light/provider/mock` in favor of mockery mocks.
Adds a simple benchmark only mock to avoid the overhead of `reflection` that `mockery` incurs.
part of #5274
This adds a test for closing the `pqueue` while the `pqueue` contains data that has not yet been dequeued.
This issue was found while debugging #6705
This test will fail until @cmwaters fix for this condition is merged.
This change does two things:
1. It fixes the json fuzzer to account for receiving array results. Arrays are returned by the rpc server when the input data is an array.
2. Adds a `fuzz_test.go` file and corresponding `testdata` directory containing the failing test case.
This seems like a reasonable way to add and track previous crash issues in our fuzz test cases. The upcoming stdlib go fuzz tool does effectively this automatically.
This change adds additional coverage to the `mConnConnection.TrySendMessage` code path. Adds test to ensure it returns `io.EOF` when closed.
Addresses: #6570
This changes adds an `MempoolError` field to the `ResponseCheckTx`. This will allow clients to understand that their transaction was rejected from the mempool despite passing the ABCI check.
This change also updates the code to make use of early returns to prevent highly nested code blocks. Namely, it returns when the type assertion fails at the beginning of the method, instead of wrapping the entire method in a large if statement. This has a somewhat large effect on the diff as rendered by github.
addresses: #3546
The Makefile at the root of the repo [includes](cd19ef244e/Makefile (L61)) the Makefile under the `test` package. This fix removes the target defined in the root Makefile in favor of the included one.
Having looked at our network address parsing and connection code, it
really looks like we're not doing anything on top of what the standard
library is doing (both in terms using `net.ParseIP` and also
`net.Dial`,) and I don't think we need to run the tests 2x the number
of times just to run through different areas of the standard
library. I think most of our users are going to be using IPv4, and
would be down to fully remove this dimension as well, if we find it's
making noise, but for now I think it's fine.
closes#2498
solves part of #3365
Note: difficult to test the event emit in SwitchToFastSync part, might need to change `stateSyncReactor` to an interface in the `nodeImpl` struct
There are many `//go:generate mockery` lines in the source code.
This change adds a make target to invoke these mock generations.
This change also invokes the mock invocations and adds the resulting mocks to the repo.
Related to #5274
This tweaks sleeps around pertubations, based on a theory that our
tests with "kill" pertubations restart the nodes fast enough the peers
haven't marked it down when it tries to reconnect. In my local test
runs, this clears out *most* of the test failures that I've seen,
except for one evidence-related test-harness problem (which should be
handled separately.)
This code change amends the dispatcher tests to read from the dispatcher's `requestCh`. This ensures that a request is waiting when the test calls `dispatcher.respond`.
addresses: #6711
I put this error log in here because I thought it might be a helpful indicator to see when a reactor sends a message to a peer that doesn't have that channel open but it turns out this is happening all the time and it's kind of annoying
This commit extends the fix in #6518, so all other goroutine which run
concurrently with processBlockchainCh can safely send data to blockchain
out channel via a bridge channel. This helps eliminating all possible
data race with sending and closing blockchainCh.Out channel at the same
time.
Fixes#6516
Closes: #6661
Note: see another error during the events indexing, guess the raw tx size exceeds the limitation?
```
3:17PM ERR failed to index block txs err="pq: index row size 2768 exceeds btree version 4 maximum 2704 for index \"tx_results_tx_result_key\"" height=5205112 module=txindex
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.2.0 to 1.2.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/spf13/cobra/releases">github.com/spf13/cobra's releases</a>.</em></p>
<blockquote>
<h2>v1.2.1</h2>
<h3>Bug fixes</h3>
<ul>
<li>Quickfix for <a href="https://github-redirect.dependabot.com/spf13/cobra/issues/1437">spf13/cobra#1437</a> after v1.2.0 where parallel use of the <code>cmd.RegisterFlagCompletionFunc()</code> (and subsequent map) now works correctly and flag completions now work again</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="de187e874d"><code>de187e8</code></a> Fix flag completion (<a href="https://github-redirect.dependabot.com/spf13/cobra/issues/1438">#1438</a>)</li>
<li>See full diff in <a href="https://github.com/spf13/cobra/compare/v1.2.0...v1.2.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
Expose p2p functions for use in the sdk.
These functions could also be copied over to the sdk. I dont have a preference of which is better.
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.8.0 to 1.8.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/spf13/viper/releases">github.com/spf13/viper's releases</a>.</em></p>
<blockquote>
<h2>v1.8.1</h2>
<p>This patch releases fixes two minor issues:</p>
<ul>
<li>Replace <code>%s</code> with <code>%w</code> when wrapping errors</li>
<li>Fix <code>pflag.StringArray</code> processing</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="bd03865899"><code>bd03865</code></a> Add a proper processing for pflag.StringArray</li>
<li><a href="3fcad43618"><code>3fcad43</code></a> update %s to %w</li>
<li>See full diff in <a href="https://github.com/spf13/viper/compare/v1.8.0...v1.8.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
It confused many people what they were supposed to add here. For chains with large genesis files you will only see the error after Initgenesis. Best to add a small sentence to provide better UX
At Oasis we have spend some time writing a new Ed25519/X25519/sr25519 implementation called curve25519-voi. This PR switches the import from ed25519consensus/go-schnorrkel, which should lead to performance gains on most systems.
Summary of changes:
* curve25519-voi is now used for Ed25519 operations, following the existing ZIP-215 semantics.
* curve25519-voi's public key cache is enabled (hardcoded size of 4096 entries, should be tuned, see the code comment) to accelerate repeated Ed25519 verification with the same public key(s).
* (BREAKING) curve25519-voi is now used for sr25519 operations. This is a breaking change as the current sr25519 support does something decidedly non-standard when going from a MiniSecretKey to a SecretKey and or PublicKey (The expansion routine is called twice). While I believe the new behavior (that expands once and only once) to be more "correct", this changes the semantics as implemented.
* curve25519-voi is now used for merlin since the included STROBE implementation produces much less garbage on the heap.
Side issues fixed:
* The version of go-schnorrkel that is currently imported by tendermint has a badly broken batch verification implementation. Upstream has fixed the issue after I reported it, so the version should be bumped in the interim.
Open design questions/issues:
* As noted, the public key cache size should be tuned. It is currently backed by a trivial thread-safe LRU cache, which is not scan-resistant, but replacing it with something better is a matter of implementing an interface.
* As far as I can tell, the only reason why serial verification on batch failure is necessary is to provide more detailed error messages (that are only used in some unit tests). If you trust the batch verification to be consistent with serial verification then the fallback can be eliminated entirely (the BatchVerifier provided by the new library supports an option that omits the fallback if this is chosen as the way forward).
* curve25519-voi's sr25519 support could use more optimization and more eyes on the code. The algorithm unfortunately is woefully under-specified, and the implementation was done primarily because I got really sad when I actually looked at go-schnorrkel, and we do not use the algorithm at this time.
Closes#6551
Simple PR to add the total gas used in the block by adding the gas used in all the transactions.
This adds a `TotalGasUsed` field to `coretypes.ResultBlockResults`.
Its my first PR to the repo so let me know if there is anything I am missing!
@fedekunze In case you want to take a look
This PR make some tweaks to backfill after running e2e tests:
- Separates sync and backfill as two distinct processes that the node calls. The reason is because if sync fails then the node should fail but if backfill fails it is still possible to proceed.
- Removes peers who don't have the block at a height from the local peer list. As the process goes backwards if a node doesn't have a block at a height they're likely pruning blocks and thus they won't have any prior ones either.
- Sleep when we've run out of peers, then try again.
There is a possible data race/panic between processBlockchainCh and
processPeerUpdates, since when we send to blockchainCh.Out in one
goroutine and close the channel in the other. The race is seen in some
Github Action runs.
This commit fix the race, by adding a peerUpdatesCh as a bridge between
processPeerUpdates and processBlockchainCh, so the former will send to
this channel, the later will listen and forward the message to
blockchainCh.Out channel.
Updates #6516
## Description
Change block_size gauge to a histogram to observe block size overtime
This will help will see which chains have full blocks vs empty.
closes#5752
## Description
Trying to debug a possible hashing issue, writing test vectors on 0.34 and then porting them to master to double-check it's not a hashing issue.
Per tendermint spec, each Channel has a globally unique byte id, which
is mapped to uint8 in Go. However, the proto PacketMsg.ChannelID field
is declared as int32, and when receive the packet, we cast it to a byte
without checking for possible overflow. That leads to a malform packet
with invalid channel id is sent successfully.
To fix it, we just add a check for possible overflow, and return invalid
channel id error.
Fixed#6521
By pre-creating the hasher, instead of creating new one everytime
addrbook.hash is called.
```
name old time/op new time/op delta
AddrBook_hash-8 181ns ±13% 80ns ± 1% -56.08% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
AddrBook_hash-8 216B ± 0% 8B ± 0% -96.30% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
AddrBook_hash-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10)
```
Fixed#6508
## Description
Add version back to versions, but allow it to be overridden via a ldflag.
Reason:
Many users are not setting the ldflag causing issues with tooling that relies on it (cosmjs)
closes#6488
cc @webmaster128
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.1 to 1.10.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/lib/pq/releases">github.com/lib/pq's releases</a>.</em></p>
<blockquote>
<h2>v1.10.2</h2>
<ul>
<li>fix TimeTZ with second offsets</li>
<li>fix GOOS compilation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="2da6713d67"><code>2da6713</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/lib/pq/issues/1039">#1039</a> from otan-cockroach/timetz_fix</li>
<li><a href="ad47bab1aa"><code>ad47bab</code></a> encode: fix TimeTZ with second offsets</li>
<li><a href="99af95f861"><code>99af95f</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/lib/pq/issues/1041">#1041</a> from otan-cockroach/libpq</li>
<li><a href="62fa4b32ec"><code>62fa4b3</code></a> .travis.yml: fix CI</li>
<li><a href="d2b13db12b"><code>d2b13db</code></a> Delete test.yml</li>
<li><a href="a1b1a43f73"><code>a1b1a43</code></a> Create test.yml</li>
<li><a href="b2cfb1abfd"><code>b2cfb1a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/lib/pq/issues/1036">#1036</a> from bukforks/master</li>
<li><a href="6ed3b8ac03"><code>6ed3b8a</code></a> rm unused imports</li>
<li><a href="feb727accb"><code>feb727a</code></a> userCurrent for unsupported GOOS</li>
<li>See full diff in <a href="https://github.com/lib/pq/compare/v1.10.1...v1.10.2">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
Internalize some libs. This reduces the amount ot public API tendermint is supporting. The moved libraries are mainly ones that are used within Tendermint-core.
To make sure finalizers run, we use channel for synchronization, and a
separate goroutine for trigger runtime.GC every 1 second. In practice,
just two consecutive runtime.GC calls can make all finalizers will run,
but using a separate goroutine make the code more robust and not depend
on garbage collector internal implementation.
Fixes#6452
This change fixes a potential exploitable vulnerability
that can cause the WAL to be consistently truncated by falsely
supplying the WAL path which would be any arbitrary dirrectory.
Fixes#6427
Somehow my previous attempt to fix this test was somewhat
non-deterministic. I think I misjudged how the "median" would impact
the test.
I ran the test in question 10 times without seeing a failure (which
would show up 10-30% of the time previously,) so I'm pretty sure this
is fixed.
Bumps [github.com/confio/ics23/go](https://github.com/confio/ics23) from 0.6.3 to 0.6.6.
<details>
<summary>Commits</summary>
<ul>
<li><a href="53a3a58ab8"><code>53a3a58</code></a> Revert go mod</li>
<li><a href="b66f10fc78"><code>b66f10f</code></a> Bump to 0.6.5</li>
<li><a href="19f273dffb"><code>19f273d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/confio/ics23/issues/40">#40</a> from confio/cleanup</li>
<li><a href="46f21260db"><code>46f2126</code></a> Clippy and cleanup in tests</li>
<li><a href="667ddb335e"><code>667ddb3</code></a> Fix clippy warnings</li>
<li><a href="ea8b91d186"><code>ea8b91d</code></a> cargo fmt</li>
<li><a href="267cfba090"><code>267cfba</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/confio/ics23/issues/39">#39</a> from kostko/kostko/feature/more-ops</li>
<li><a href="346d8d9b19"><code>346d8d9</code></a> Implement FIXED32_LITTLE length operation</li>
<li><a href="61321db422"><code>61321db</code></a> Add SHA-512/256 hash operation</li>
<li><a href="77277ad2f8"><code>77277ad</code></a> Bump Rust to 0.6.4</li>
<li>Additional commits viewable in <a href="https://github.com/confio/ics23/compare/v0.6.3...go/v0.6.6">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
I believe that this, in my testing seems to help the e2e state-sync
tests complete more reliably, by fixing some potential, range-related
slice building, as well as the way the test app hashes snapshots.
Additionally, and I'm not sure if we want to do this, but I added this
hook to the reactor that re-sends the request for snapshots during the
retry. This helps in tests prevent systems from getting stuck, but I
think in reality, it might create more traffic, and operators would
just restart a state-syncing node to get a similar effect.
Per conversations earlier today, we'll consider all proposed implementation changes part of the ADR process rather than the RFC process (which will remain, for now, on the spec; this may get incorporated instead into the burgeoning "CIPS" process).
This change renames RFC 1 to ADR 66, leaving space for the not-yet-merged ADR 65.
* add time warping lunatic attack test
* create too high and connecton refused errors and add to the light client provider
* add height check to provider
* introduce block lag
* add detection logic for processing forward lunatic attack
* add node-side verification logic
* clean up tests and formatting
* update adr's
* update testing
* fix fetching the latest block
* format
* update changelog
* implement suggestions
* modify ADR's
* format
* clean up node evidence verification
Bumps [github.com/minio/highwayhash](https://github.com/minio/highwayhash) from 1.0.1 to 1.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/minio/highwayhash/releases">github.com/minio/highwayhash's releases</a>.</em></p>
<blockquote>
<h2>Version v1.0.2</h2>
<h2>Changelog</h2>
<h3>Fixed</h3>
<p>Issue <a href="https://github-redirect.dependabot.com/minio/highwayhash/issues/17">#17</a> - on arm64 (on Go 1.16) wrong hash values got computed due to incorrectly naming asm constants like regular Go functions. This probably confused the linker and caused the arm64 implementation to compute incorrect hash values. Fixed by 08ce0b4</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="08ce0b4fa7"><code>08ce0b4</code></a> Fix ARM64 assembly (<a href="https://github-redirect.dependabot.com/minio/highwayhash/issues/19">#19</a>)</li>
<li><a href="5311fe963f"><code>5311fe9</code></a> disable arm64 assembler and update CI to Go 1.16 (<a href="https://github-redirect.dependabot.com/minio/highwayhash/issues/18">#18</a>)</li>
<li>See full diff in <a href="https://github.com/minio/highwayhash/compare/v1.0.1...v1.0.2">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [vuepress-theme-cosmos](https://github.com/cosmos/vuepress-theme-cosmos) from 1.0.180 to 1.0.181.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/cosmos/vuepress-theme-cosmos/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
- Add `context.Context` to Privval interface
This pr does not introduce context into our custom privval connection protocol because this will be removed in the next release. When this pr is released.
## Description
Since events are not hashed into the header they can be non deterministic. Changing an event is not consensus breaking. Will update docs in the spec
* test/fuzz: fix rpc, secret_connection and pex tests
- ignore empty data in rpc
- provide correct IP in pex
- spawn a goroutine for Write and do multiple Read(s)
* test/fuzz: fix init in pex test
* test/fuzz: assign NewServeMux to global var
* test/fuzz: only try to Unmarshal if blob is not empty
* run fuzz tests for PRs which modify fuzz tests themselves
* test/fuzz: move MakeSwitch into init
Introduces heuristics that track the amount of no responses or unavailable blocks a provider has for more robust provider handling by the light client. Use concurrent calls to all witnesses when a new primary is needed.
also
- replace `MaxReconnectAttempts`, `ReadWait`, `WriteWait` and `PingPeriod` options with `WSOptions` in `WSClient` (rpc/jsonrpc/client/ws_client.go).
- set default write wait to 10s for `WSClient`(rpc/jsonrpc/client/ws_client.go)
- unexpose `WSEvents`(rpc/client/http.go)
Closes#6162
## Description
Fixes marshaling error in sdk
closes https://github.com/cosmos/cosmos-sdk/issues/8578
the output stays the same, we are avoiding the passing of the callback because sdk uses typed logging.
```
// unbuffered
out, err := httpClient.Subscribe(ctx, "event.type=NewTx and account.name=Jack", 0)
// buffered
out, err := httpClient.Subscribe(ctx, "event.type=NewTx AND account.name=Jack", 20)
```
Before: when the `out` channel is buffered and becomes full, we drop an event (+ log the error)
After: when the `out` channel is buffered and becomes full, we block
**Before it was not apparent to the app when an event was dropped (looking at the logs is manual task). After this PR, if the user does not read from `out` on 1 subscription, all other subscriptions will be stuck too.**
Closes#6161
I'm also going to add the retros for all previous security incidents to this directory - I'd like to have them somewhere more central than the Cosmos Forum, where they currently live.
Missed setting the buffer size on the subscription. Note, this doesn't really "fix" this test (a la ref: https://github.com/tendermint/tendermint/pull/5710).
However, I spent a good chunk of time looking at this test with many logs and I'm pretty sure this is mainly due to the fact that none of the nodes get the conflicting vote in time.
closes: #6127
Fixes the race condition between a callback being set and called during ReCheckTx. Note, I do not see equivalent logic in the gRPC client (anymore) as #5439 suggests, so only the socket client was updated.
closes: #5439
Bumps [watchpack](https://github.com/webpack/watchpack) from 2.1.0 to 2.1.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/webpack/watchpack/releases">watchpack's releases</a>.</em></p>
<blockquote>
<h2>v2.1.1</h2>
<h1>Bugfix</h1>
<ul>
<li>fix warnings with ENOENT when symlinks are resolved by watchpack</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="f1b5e2da2d"><code>f1b5e2d</code></a> 2.1.1</li>
<li><a href="cbfc11a8d7"><code>cbfc11a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/webpack/watchpack/issues/188">#188</a> from Aghassi/fix/enoent-throwing</li>
<li><a href="7684df0846"><code>7684df0</code></a> fix: adds ENOENT for non windows errors</li>
<li>See full diff in <a href="https://github.com/webpack/watchpack/compare/v2.1.0...v2.1.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
I got tired of seeing the literal phrase "Closes #XXX" left in PR bodies.
Also, this template isn't usually viewed as rendered markdown, so I've removed the markdown formatting and the "Description" heading (which usually gets deleted anyways).
This cleans up the `Router` code and adds a bunch of tests. These sorts of systems are a real pain to test, since they have a bunch of asynchronous goroutines living their own lives, so the test coverage is decent but not fantastic. Luckily we've been able to move all of the complex peer management and transport logic outside of the router, as synchronous components that are much easier to test, so the core router logic is fairly small and simple.
This also provides some initial test tooling in `p2p/p2ptest` that automatically sets up in-memory networks and channels for use in integration tests. It also includes channel-oriented test asserters in `p2p/p2ptest/require.go`, but these have primarily been written for router testing and should probably be adapted or extended for reactor testing.
E2E tests often fail because validators miss signing or proposing blocks. Often this is because e.g. there's a lot of disruption in the network or it takes a long time to start up all the nodes.
This changes the test criteria to only check for 3 signed/proposed blocks, rather than a fraction of the expected blocks. This should be enough to catch most issues, apart from performance problems causing nodes to miss signing/proposing, but we may want separate tests for those sorts of things.
This renames `PeerAddress` to `NodeAddress`, moves it and `NodeID` into a separate file `address.go`, adds tests for them, and fixes a bunch of bugs and inconsistencies.
This revises the new P2P `Transport` interface and does some preliminary code cleanups and simplifications.
The major change here is to add `Connection.Handshake()` for performing node handshakes (once the stream transport API is implemented, this can be done entirely independent of the transport). This moves most of the handshaking logic into the `Router`, such as prevention of head-of-line blocking, validation of peer's `NodeInfo`, controlling timeouts, and so on. This significantly simplifies transports, completely removes the need for internal goroutines, and shares common logic across all transports. This also allows varying the handshake `NodeInfo` across peers, e.g. to vary `ListenAddr`. Similarly, connection filtering is also moved into the switch/router so that it can be shared between transports.
Fixes#5981, which was caused by changes in Router behavior after the introduction of the peer manager, leading to a race condition that could halt the test.
This is a temporary measure, I'll start tightening up the new P2P core tomorrow and write "real" tests with better test infrastructure.
This patches over a test data race where the logger would try to read struct internals via `reflect` while these were concurrently modified (specifically `MemoryTransport.closeOnce`).
Executed a local network using simapp and looked for logs that seemed superfluous. This isn't by any means an exhaustive grooming, but should drastically help legibility of logs.
ref: #5912
This test occasionally fails because the peer is already stopped. It is unclear to me exactly what this test is supposed to do, since calling `FlushStop()` will stop the peer, but the test asserts that the peer shouldn't have been stopped by `FlushStop()` since calling `Stop()` afterwards will error in that case.
The current PEX reactor will be removed in the new P2P stack anyway.
Fixes#5998. Sometimes the connection returns "use of closed network connection" instead, so for now we just accept any error. The switch is not long for this world anyway.
This test relied on connecting to the external site `foo-bar.net`, and (predictably) the site went down and broke all of our CI runs. This changes it to use local HTTP servers instead.
This changes the new prototype PEX reactor to resolve peer address URLs into IP/port PEX addresses itself. Branched off of #5974.
I've spent some time thinking about address handling in the P2P stack. We currently use `PeerAddress` URLs everywhere, except for two places: when dialing a peer, and when exchanging addresses via PEX. We had two options:
1. Resolve addresses to endpoints inside `PeerManager`. This would introduce a lot of added complexity: we would have to track connection statistics per endpoint, have goroutines that asynchronously resolve and refresh these endpoints, deal with resolve scheduling before dialing (which is trickier than it sounds since it involves multiple goroutines in the peer manager and router and messes with peer rating order), handle IP address visibility issues, and so on.
2. Resolve addresses to endpoints (IP/port) only where they're used: when dialing, and in PEX. Everywhere else we use URLs.
I went with 2, because this significantly simplifies the handling of hostname resolution, and because I really think the PEX reactor should migrate to exchanging URLs instead of IP/port numbers anyway -- this allows operators to use DNS names for validators (and can easily migrate them to new IPs and/or load balance requests), and also allows different protocols (e.g. QUIC and `MemoryTransport`). Happy to discuss this.
Fixes#5899 by renaming a bunch of P2P Protobuf entities (while maintaining wire compatibility):
* `Message` to `PexMessage` (as it's only used for PEX messages).
* `PexAddrs` to `PexResponse`.
* `PexResponse.Addrs` to `PexResponse.Addresses`.
* `NetAddress` to `PexAddress` (as it's only used by PEX).
## Description
Fixes the data race in usage of `WaitGroup`. Specifically, the case where we invoke `Wait` _before_ the first delta `Add` call when the current waitgroup counter is zero. See https://golang.org/pkg/sync/#WaitGroup.Add.
Still not sure how this manifests itself in a test since the reactor has to be stopped virtually immediately after being started (I think?).
Regardless, this is the appropriate fix.
closes: #5968
Adds a naïve `PeerManager.Advertise()` method that the new PEX reactor can use to fetch addresses to advertise, as well as some other `FIXME`s on address advertisement.
Follow-up from #5947, branched off of #5954.
This simplifies the upgrade logic by adding explicit eviction requests, which can also be useful for other use-cases (e.g. if we need to ban a peer that's misbehaving). Changes:
* Add `evict` map which queues up peers to explicitly evict.
* `upgrading` now only tracks peers that we're upgrading via dialing (`DialNext` → `Dialed`/`DialFailed`).
* `Dialed` will unmark `upgrading`, and queue `evict` if still beyond capacity.
* `Accepted` will pick a random lower-scored peer to upgrade to, if appropriate, and doesn't care about `upgrading` (the dial will fail later, since it's already connected).
* `EvictNext` will return a peer scheduled in `evict` if any, otherwise if beyond capacity just evict the lowest-scored peer.
This limits all of the `upgrading` logic to `DialNext`, `Dialed`, and `DialFailed`, making it much simplier, and it should generally do the right thing in all cases I can think of.
This improves the `peerStore` prototype by e.g.:
* Using a database with Protobuf for persistence, but also keeping full peer set in memory for performance.
* Simplifying the API, by taking/returning struct copies for safety, and removing errors for in-memory operations.
* Caching the ranked peer set, as a temporary solution until a better data structure is implemented.
* Adding `PeerManagerOptions.MaxPeers` and pruning the peer store (based on rank) when it's full.
* Rewriting `PeerAddress` to be independent of `url.URL`, normalizing it and tightening semantics.
## Description
Update the faux router to either drop channel errors or handle them based on an argument. This prevents deadlocks in tests where we try to send an error on the mempool channel but there is no reader.
Closes: #5956
See #5936 and #5938 for background.
The plan was initially to have `DialNext()` and `EvictNext()` return a channel. However, implementing this became unnecessarily complicated and error-prone. As an example, the channel would be both consumed and populated (via method calls) by the same driving method (e.g. `Router.dialPeers()`) which could easily cause deadlocks where a method call blocked while sending on the channel that the caller itself was responsible for consuming (but couldn't since it was busy making the method call). It would also require a set of goroutines in the peer manager that would interact with the goroutines in the router in non-obvious ways, and fully populating the channel on startup could cause deadlocks with other startup tasks. Several issues like these made the solution hard to reason about.
I therefore simply made `DialNext()` and `EvictNext()` block until the next peer was available, using internal triggers to wake these methods up in a non-blocking fashion when any relevant state changes occurred. This proved much simpler to reason about, since there are no goroutines in the peer manager (except for trivial retry timers), nor any blocking channel sends, and it instead relies entirely on the existing goroutine structure of the router for concurrency. This also happens to be the same pattern used by the `Transport.Accept()` API, following Go stdlib conventions, so all router goroutines end up using a consistent pattern as well.
Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
Closes#5907
- add init-corpus to blockchain reactor
- remove validator-set FromBytes test
now that we have proto, we don't need to test it! bye amino
- simplify mempool test
do we want to test remote ABCI app?
- do not recreate mux on every crash in jsonrpc test
- update p2p pex reactor test
- remove p2p/listener test
the API has changed + I did not understand what it's tested anyway
- update secretconnection test
- add readme and makefile
- list inputs in readme
- add nightly workflow
- remove blockchain fuzz test
EncodeMsg / DecodeMsg no longer exist
Fixes#5941.
Not entirely sure that this will fix the problem (couldn't reproduce), but in any case this is an artifact of a hack in the P2P transport refactor to make it work with the legacy P2P stack, and will be removed when the refactor is done anyway.
The `proto-gen-docker` target didn't pull an updated Docker image, and would use a local image if present which could be outdated and produce wrong results.
This improves the prototype peer manager by:
* Exporting `PeerManager`, making it accessible by e.g. reactors.
* Replacing `Router.SubscribePeerUpdates()` with `PeerManager.Subscribe()`.
* Tracking address/peer connection statistics, and retrying dial failures with exponential backoff.
* Prioritizing peers, with persistent peers configuration.
* Limiting simultaneous connections.
* Evicting peers and upgrading to higher-priority peers.
* Tracking peer heights, as a workaround for legacy shared peer state APIs.
This is getting to a point where we need to determine precise semantics and implement tests, so we should figure out whether it's a reasonable abstraction that we want to use. The main questions are around the API model (i.e. synchronous method calls with the router polling the manager, vs. an event-driven model using channels, vs. the peer manager calling methods on the router to connect/disconnect peers), and who should have the responsibility of managing actual connections (currently the router, while the manager only tracks peer state).
Conflicting votes are now sent to the evidence pool to form duplicate vote evidence only once
the height of the evidence is finished and the time of the block finalised.
This adds a prototype peer lifecycle manager, `peerManager`, which stores peer data in an internal `peerStore`. The overall idea here is to have methods for peer lifecycle events which exchange a very narrow subset of peer data, and to keep all of the peer metadata (i.e. the `peerInfo` struct) internal, to decouple this from the router and simplify concurrency control. See `peerManager` GoDoc for more information.
The router is still responsible for actually dialing and accepting peer connections, and routing messages across them, but the peer manager is responsible for determining which peers to dial next, preventing multiple connections being established for the same peer (e.g. both inbound and outbound), and making sure we don't dial the same peer several times in parallel. Later it will also track retries and exponential backoff, as well as peer and address quality. It also assumes responsibility for peer updates subscriptions.
It's a bit unclear to me whether we want the peer manager to take on the responsibility of actually dialing and accepting connections as well, or if it should only be tracking peer state for the router while the router is responsible for all transport concerns. Let's revisit this later.
Bumps [vuepress-theme-cosmos](https://github.com/cosmos/vuepress-theme-cosmos) from 1.0.179 to 1.0.180.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/cosmos/vuepress-theme-cosmos/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.6.1 to 1.7.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/stretchr/testify/releases">github.com/stretchr/testify's releases</a>.</em></p>
<blockquote>
<h2>Minor improvements and bug fixes</h2>
<p>Minor feature improvements and bug fixes</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="acba37e5db"><code>acba37e</code></a> Only use repeatability if no repeatability left</li>
<li><a href="eb8c41ec07"><code>eb8c41e</code></a> Add more tests to mock package</li>
<li><a href="a5830c56d3"><code>a5830c5</code></a> Extract method to evaluate closest match</li>
<li><a href="1962448488"><code>1962448</code></a> Use Repeatability as tie-breaker for closest match</li>
<li><a href="92707c0b2d"><code>92707c0</code></a> Fixed the link to not point to assert only</li>
<li><a href="05dd0b2b35"><code>05dd0b2</code></a> Updated the readme to point to pkg.dev</li>
<li><a href="c26b7f39f8"><code>c26b7f3</code></a> Update assertions.go</li>
<li><a href="8fb4b2442e"><code>8fb4b24</code></a> [Fix] The most recent changes to golang/protobuf breaks the spew Circular dat...</li>
<li><a href="dc8af7208c"><code>dc8af72</code></a> add generated code for positive/negative assertion</li>
<li><a href="1544508911"><code>1544508</code></a> add assert positive/negative</li>
<li>Additional commits viewable in <a href="https://github.com/stretchr/testify/compare/v1.6.1...v1.7.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Early but functional prototype of the new `p2p.Router`, see its GoDoc comment for details on how it works. Expect much of this logic to change and improve as we evolve the new P2P stack.
There is a simple test that sets up an in-memory network of four routers with reactors and passes messages between them, but otherwise no exhaustive tests since this is very much a work-in-progress.
E2E tests often fail due to fast sync stalls causing the validator to miss signing blocks. This increases the tolerance for missed signatures to 2/3 to allow validators to spend more time starting up.
#5852 fixed an issue with error propagation in `os.EnsureDir()`. However, this function is basically identical to `os.MkdirAll()`, and can be replaced entirely with a call to it. We keep the function for backwards compatibility.
blockchain/vX reactor priority was decreased because during the normal operation
(i.e. when the node is not fast syncing) blockchain priority can't be
the same as consensus reactor priority. Otherwise, it's theoretically possible to
slow down consensus by constantly requesting blocks from the node.
NOTE: ideally blockchain/vX reactor priority would be dynamic. e.g. when
the node is fast syncing, the priority is 10 (max), but when it's done
fast syncing - the priority gets decreased to 5 (only to serve blocks
for other nodes). But it's not possible now, therefore I decided to
focus on the normal operation (priority = 5).
evidence and consensus critical messages are more important than
the mempool ones, hence priorities are bumped by 1 (from 5 to 6).
statesync reactor priority was changed from 1 to 5 to be the same as
blockchain/vX priority.
Refs https://github.com/tendermint/tendermint/issues/5816
Bumps [github.com/cosmos/iavl](https://github.com/cosmos/iavl) from 0.15.2 to 0.15.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/cosmos/iavl/releases">github.com/cosmos/iavl's releases</a>.</em></p>
<blockquote>
<h2>v0.15.3</h2>
<p><a href="257e8b9292/CHANGELOG.md (0153-december-21-2020)</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/cosmos/iavl/blob/master/CHANGELOG.md">github.com/cosmos/iavl's changelog</a>.</em></p>
<blockquote>
<h2>0.15.3 (December 21, 2020)</h2>
<p>Special thanks to external contributors on this release: <a href="https://github.com/odeke-em">@odeke-em</a></p>
<h3>Improvements</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/cosmos/iavl/pull/352">#352</a> Reuse buffer to improve performance of <code>GetMembershipProof()</code> and <code>GetNonMembershipProof()</code>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="257e8b9292"><code>257e8b9</code></a> changelog: release 0.15.3 (<a href="https://github-redirect.dependabot.com/cosmos/iavl/issues/353">#353</a>)</li>
<li><a href="b2dffed4b2"><code>b2dffed</code></a> convertVarIntToBytes: use reusable bytes array (<a href="https://github-redirect.dependabot.com/cosmos/iavl/issues/352">#352</a>)</li>
<li><a href="9e510e5a64"><code>9e510e5</code></a> github: run tests with 32-bit arch as well (<a href="https://github-redirect.dependabot.com/cosmos/iavl/issues/350">#350</a>)</li>
<li>See full diff in <a href="https://github.com/cosmos/iavl/compare/v0.15.2...v0.15.3">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
@p4u from vocdoni.io reported that the mempool might behave incorrectly under a
high load. The consequences can range from pauses between blocks to the peers
disconnecting from this node.
My current theory is that the flowrate lib we're using to control flow
(multiplex over a single TCP connection) was not designed w/ large blobs
(1MB batch of txs) in mind.
I've tried decreasing the Mempool reactor priority, but that did not
have any visible effect. What actually worked is adding a time.Sleep
into mempool.Reactor#broadcastTxRoutine after an each successful send ==
manual control flow of sort.
As a temporary remedy (until the mempool package
is refactored), the max-batch-bytes was disabled. Transactions will be sent
one by one without batching
Closes#5796
When set to true, an invalid transaction will be kept in the cache (this may help some applications to protect against spam).
NOTE: this is a temporary config option. The more correct solution would be to add a TTL to each transaction (i.e. CheckTx may return a TTL in ResponseCheckTx).
Closes: #5751
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.8.0 to 1.9.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/prometheus/client_golang/releases">github.com/prometheus/client_golang's releases</a>.</em></p>
<blockquote>
<h2>1.9.0 / 2020-12-17</h2>
<ul>
<li>[FEATURE] <code>NewPidFileFn</code> helper to create process collectors for processes whose PID is read from a file. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/804">#804</a></li>
<li>[BUGFIX] promhttp: Prevent endless loop in <code>InstrumentHandler...</code> middlewares with invalid metric or label names. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/823">#823</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md">github.com/prometheus/client_golang's changelog</a>.</em></p>
<blockquote>
<h2>1.9.0 / 2020-12-17</h2>
<ul>
<li>[FEATURE] <code>NewPidFileFn</code> helper to create process collectors for processes whose PID is read from a file. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/804">#804</a></li>
<li>[BUGFIX] promhttp: Prevent endless loop in <code>InstrumentHandler...</code> middlewares with invalid metric or label names. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/823">#823</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d89cf5af88"><code>d89cf5a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/826">#826</a> from prometheus/beorn7/release</li>
<li><a href="80ca9cdc4e"><code>80ca9cd</code></a> Cut release 1.9.0</li>
<li><a href="8d16199dea"><code>8d16199</code></a> Update dependencies</li>
<li><a href="8b73bd904c"><code>8b73bd9</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/823">#823</a> from prometheus/beorn7/promhttp</li>
<li><a href="98eb6cbf7c"><code>98eb6cb</code></a> promhttp: Correctly detect invalid metric and label names</li>
<li><a href="37c26edd5b"><code>37c26ed</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/821">#821</a> from prometheus/beorn7/multierror</li>
<li><a href="34ca120377"><code>34ca120</code></a> Be more explicit about the multi-line properties of MultiError</li>
<li><a href="fd6d368676"><code>fd6d368</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/819">#819</a> from jubalh/sp</li>
<li><a href="cf6dc82780"><code>cf6dc82</code></a> Correct spelling: possibilites -> possibilities</li>
<li><a href="39b478e90c"><code>39b478e</code></a> Added example api code showing how to add auth tokens and user agents to prom...</li>
<li>Additional commits viewable in <a href="https://github.com/prometheus/client_golang/compare/v1.8.0...v1.9.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
time_iota_ms is intended to ensure that an honest validator always generates timestamps
with time increasing monotonically. For this purpose, it always suffices to have this parameter
set to `1ms`. Allowing users to choose different numbers increases bug surface area.
Thus the code now ignores the user provided time_iota_ms parameter (marking it as unused),
and uses 1ms internally.
The MTU (Maximum Transmission Unit) for Ethernet is 1500 bytes.
The IP header and the TCP header take up 20 bytes each at least (unless
optional header fields are used) and thus the max for (non-Jumbo frame)
Ethernet is 1500 - 20 -20 = 1460
Source: https://stackoverflow.com/a/3074427/820520
While debugging the mempool issue (#5796), I've noticed we're spending
quite a bit of time encoding blobs of data, which never get printed! The
reason is filtering occurs on the level below, so Go runtime rightfully
evaluates function arguments.
I think it's okay to not format raw bytes.
The `NodeInfo` interface does not appear to serve any purpose at all, so I removed it and renamed the `DefaultNodeInfo` struct to `NodeInfo` (including the Protobuf representations). Let me know if this is actually needed for anything.
Only the Protobuf rename is listed in the changelog, since we do not officially support API stability of the `p2p` package (according to `README.md`). The on-wire protocol remains compatible.
This implements a new `Transport` interface and related types for the P2P refactor in #5670. Previously, `conn.MConnection` was very tightly coupled to the `Peer` implementation -- in order to allow alternative non-multiplexed transports (e.g. QUIC), MConnection has now been moved below the `Transport` interface, as `MConnTransport`, and decoupled from the peer. Since the `p2p` package is not covered by our Go API stability, this is not considered a breaking change, and not listed in the changelog.
The initial approach was to implement the new interface in its final form (which also involved possible protocol changes, see https://github.com/tendermint/spec/pull/227). However, it turned out that this would require a large amount of changes to existing P2P code because of the previous tight coupling between `Peer` and `MConnection` and the reliance on subtleties in the MConnection behavior. Instead, I have broadened the `Transport` interface to expose much of the existing MConnection interface, preserved much of the existing MConnection logic and behavior in the transport implementation, and tried to make as few changes to the rest of the P2P stack as possible. We will instead reduce this interface gradually as we refactor other parts of the P2P stack.
The low-level transport code and protocol (e.g. MConnection, SecretConnection and so on) has not been significantly changed, and refactoring this is not a priority until we come up with a plan for QUIC adoption, as we may end up discarding the MConnection code entirely.
There are no tests of the new `MConnTransport`, as this code is likely to evolve as we proceed with the P2P refactor, but tests should be added before a final release. The E2E tests are sufficient for basic validation in the meanwhile.
Closes#5766
* memoize the scSchedulerFail error to avoid printing it every scheduleFreq
* blockchain/v2: modify switchIO funcs to accept peer instead of peerID
`Makefile` used `git branch --show-current` for branch detection. This option was introduced in Git 2.22. However, the current Debian release (Buster), which is used by the `golang:1.15` Docker image, uses Git 2.20. This gives spurious errors e.g. when running the E2E tests:
```
error: unknown option `show-current'
```
This PR changes the branch detection to be compatible with Git 2.20. The behavior appears to be the same as `git branch --show-current`, both when on a branch, on a tag, and on a detached HEAD.
closes: #5770closes: #5769
also, include node ID in the output (#5769) and modify NodeKey to use
value semantics (it makes perfect sense for NodeKey to not be a
pointer).
## Description
Bump version to get performance updates:
```
benchmark old ns/op new ns/op delta
BenchmarkVerification-8 174857 78376 -55.18%
```
Closes: #XXX
Bumps [vuepress-theme-cosmos](https://github.com/cosmos/vuepress-theme-cosmos) from 1.0.177 to 1.0.178.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/cosmos/vuepress-theme-cosmos/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [vuepress-theme-cosmos](https://github.com/cosmos/vuepress-theme-cosmos) from 1.0.176 to 1.0.177.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/cosmos/vuepress-theme-cosmos/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/configuring-github-dependabot-security-updates)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
After a reactor has failed to parse an incoming message, it shouldn't output the "bad" data into the logs, as that data is unfiltered and could have anything in it. (We also don't think this information is helpful to have in the logs anyways.)
Replace testing.T.Cleanup() with deferred function
calls in test helpers as those cleanup functions
need to be called once the helper returns and not
when the entire test ends.
This reverts a few of the changes introduced in #5723.
Thanks: @erikgrinaker for pointing this out.
Ref: #5732
*testing.T.TempDir() causes test cases to fail when
it is unable to remove the temporary directory once
the test case execution terminates. This seems to
happen often with pex reactor test cases.
I introduced a new variable - syncEnded, which is now used to prevent
sending new events to channels (which would block otherwise) if reactor
is finished syncing
Closes#4591
## Description
I'm just doing a self audit of the light client. There's a few things I've changed
- Validate trust level in `VerifyNonAdjacent` function
- Make errNoWitnesses public (it's something people running software on top of a light client should be able to parse)
- Remove `ChainID` check of witnesses on start up. We do this already when we compare the first header with witnesses
- Remove `ChainID()` from provider interface
Closes: #4538
`abci.Client`:
- Sync and Async methods now accept a context for cancellation
* grpc client uses context to cancel both Sync and Async requests
* local client ignores context parameter
* socket client uses context to cancel Sync requests and to drop Async requests before sending them if context was cancelled prior to that
- Async methods return an error
* socket client returns an error immediately if queue is full for Async requests
* local client always returns nil error
* grpc client returns an error if context was cancelled before we got response or the receiving queue had a space for response (do not confuse with the sending queue from the socket client)
- specify clients semantics in [doc.go](https://raw.githubusercontent.com/tendermint/tendermint/27112fffa62276bc016d56741f686f0f77931748/abci/client/doc.go)
`mempool.TxInfo`
- add optional `Context` to `TxInfo`, which can be used to cancel `CheckTx` request
Closes#5190
Replace defer with t.Cleanup().
Replace the combination of ioutil.TempDir, error checking
and defer os.RemoveAll() with Go testing.T's new TempDir()
helper.
Mark auxiliary functions as test helpers.
This fixes spurious `TestByzantinePrevoteEquivocation` failures by extending the block range and time spent waiting for evidence. I've seen many runs where the evidence isn't committed until e.g. height 27. Haven't looked into _why_ this happens, but as long as the evidence is committed eventually and the test doesn't spuriously fail I'm (mostly) happy. WDYT @cmwaters?
## Description
- separate docs related to running nodes into the nodes dir.
- keep old files but dont display them
- bring over debugging like a pro blog
Closes: #XXX
Closes#5444
Now we record the fact that a peer does not have a requested block and later use this information to make a new request for the same block from another peer.
## Description
Hardcode ed25519 to dialTCPFn in e2e tests.
I will backport `DefaultRequestHandler` fixes
This will be replaced when grpc is implemented.
## Description
- remove installation of protoc
- use buf protoc to generate proto stubs
prior to approving could someone test locally. I restarted my docker instance and its been stuck for 20+ minutes
Closes: #XXX
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.33.1 to 1.33.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/grpc/grpc-go/releases">google.golang.org/grpc's releases</a>.</em></p>
<blockquote>
<h2>Release 1.33.2</h2>
<ul>
<li>protobuf: update all generated code to google.golang.org/protobuf (<a href="https://github-redirect.dependabot.com/grpc/grpc-go/issues/3932">#3932</a>)</li>
<li>xdsclient: populate error details for NACK (<a href="https://github-redirect.dependabot.com/grpc/grpc-go/issues/3975">#3975</a>)</li>
<li>internal/credentials: fix a bug and add one more helper function SPIFFEIDFromCert (<a href="https://github-redirect.dependabot.com/grpc/grpc-go/issues/3929">#3929</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="56d63285d5"><code>56d6328</code></a> github: remove advancedtls examples test</li>
<li><a href="6396e4b7d7"><code>6396e4b</code></a> vet: ignore proto deprecation warnings</li>
<li><a href="0afe9d28d8"><code>0afe9d2</code></a> github: add Github Actions workflow for tests; support in vet.sh (<a href="https://github-redirect.dependabot.com/grpc/grpc-go/issues/4005">#4005</a>)</li>
<li><a href="8a0ca33b85"><code>8a0ca33</code></a> Change version to 1.33.2</li>
<li><a href="c1989b58a5"><code>c1989b5</code></a> protobuf: update all generated code to google.golang.org/protobuf (<a href="https://github-redirect.dependabot.com/grpc/grpc-go/issues/3932">#3932</a>)</li>
<li><a href="b205df69d4"><code>b205df6</code></a> xdsclient: populate error details for NACK (<a href="https://github-redirect.dependabot.com/grpc/grpc-go/issues/3975">#3975</a>)</li>
<li><a href="75e27683ed"><code>75e2768</code></a> internal/credentials: fix a bug and add one more helper function SPIFFEIDFrom...</li>
<li><a href="17493ac067"><code>17493ac</code></a> Change version to 1.33.2-dev</li>
<li>See full diff in <a href="https://github.com/grpc/grpc-go/compare/v1.33.1...v1.33.2">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/configuring-github-dependabot-security-updates)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
This is the first iteration of model-based testing in Go Tendermint. The test runner is using the static JSON fixtures located under the ./json directory. In the future, the Rust tensgen binary will be used to generate those (given the static intermediate scenarios and the test seed, which will be published along with each testgen release).
Closes: #5322
Removes `p2p.FuzzedConnection`, since it does not appear to be in use. While these sorts of test wrappers may be useful, they should be injected directly instead of bleeding through into the main application configuration. We'll implement something similar if and when necessary, for the new P2P abstractions in #2067.
* Don't use state sync for nodes starting at initial height.
* Also remove stopped containers when cleaning up.
* Start nodes in order of startAt, mode, name to avoid full nodes starting before their seeds.
* Tweak network waiting to avoid halts caused by validator changes and perturbations.
* Disable most tests for seed nodes, which aren't always able to join consensus.
* Disable `blockchain/v2` due to known bugs.
In #5488 the E2E testnet generator changed to setting explicit `StartAt` heights for initial nodes. This broke the runner, which expected all initial nodes to have `StartAt: 0`, as well as validator set scheduling in the generator. Testnet loading now normalizes initial nodes to have `StartAt: 0`.
This also tweaks waiting for misbehavior heights to only use an additional wait if there actually is any misbehavior in the testnet, and to output information when waiting.
Fixes#5540, fixes#2965. This is a hack that patches over the problem, but really the whole async handling in gRPC should be redesigned, as should ReqRes callback dispatch.
Before: scheduler receives psBlockProcessed event, but does not mark block as processed because peer timed out (or was removed for other reasons) and all associated blocks were rescheduled.
After: scheduler receives psBlockProcessed event and marks block as processed in any case (even if peer who provided this block errors).
Closes#5387
When a peer is stopped due to some network issue, the Reactor calls scheduler#handleRemovePeer, which removes the peer from the scheduler. BUT the peer stays in the processor, which sometimes could lead to "duplicate block enqueued by processor" panic WHEN the same block is requested by the scheduler again from a different peer. The solution is to return scPeerError, which will be propagated to the processor. The processor will clean up the blocks associated with the peer in purgePeer.
Closes#5513, #5517
Fixes#5439. This is really a workaround for #5519 (unless we require async implementations to return ordered responses, but that kind of defeats the purpose of having an async API).
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.1.0 to 1.1.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/spf13/cobra/releases">github.com/spf13/cobra's releases</a>.</em></p>
<blockquote>
<h2>v1.1.1</h2>
<ul>
<li><strong>Fix:</strong> yaml.v2 2.3.0 contained a unintended breaking change. This release reverts to yaml.v2 v2.2.8 which has recent critical CVE fixes, but does not have the breaking changes. See <a href="https://github-redirect.dependabot.com/spf13/cobra/pull/1259">spf13/cobra#1259</a> for context.</li>
<li><strong>Fix:</strong> correct internal formatting for go-md2man v2 (which caused man page generation to be broken). See <a href="https://github-redirect.dependabot.com/spf13/cobra/issues/1049">spf13/cobra#1049</a> for context.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="86f8bfd7fe"><code>86f8bfd</code></a> fix manpage building with new go-md2man (<a href="https://github-redirect.dependabot.com/spf13/cobra/issues/1255">#1255</a>)</li>
<li><a href="f32f4ef15b"><code>f32f4ef</code></a> Don't use yaml.v2 2.3.0 which has a breaking change (<a href="https://github-redirect.dependabot.com/spf13/cobra/issues/1259">#1259</a>)</li>
<li>See full diff in <a href="https://github.com/spf13/cobra/compare/v1.1.0...v1.1.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/configuring-github-dependabot-security-updates)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.4.2 to 1.4.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/golang/protobuf/releases">github.com/golang/protobuf's releases</a>.</em></p>
<blockquote>
<h2>v1.4.3</h2>
<p>Notable changes:</p>
<p>(<a href="https://github-redirect.dependabot.com/golang/protobuf/issues/1221">#1221</a>) jsonpb: Fix marshaling of Duration
(<a href="https://github-redirect.dependabot.com/golang/protobuf/issues/1210">#1210</a>) proto: convert integer to rune before converting to string</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="4846b58453"><code>4846b58</code></a> jsonpb: Fix marshaling of Duration (<a href="https://github-redirect.dependabot.com/golang/protobuf/issues/1221">#1221</a>)</li>
<li><a href="91c84e0db1"><code>91c84e0</code></a> travis.yml: update tested versions of Go (<a href="https://github-redirect.dependabot.com/golang/protobuf/issues/1211">#1211</a>)</li>
<li><a href="3860b2764f"><code>3860b27</code></a> proto: convert integer to rune before converting to string (<a href="https://github-redirect.dependabot.com/golang/protobuf/issues/1210">#1210</a>)</li>
<li>See full diff in <a href="https://github.com/golang/protobuf/compare/v1.4.2...v1.4.3">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/configuring-github-dependabot-security-updates)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Closes#5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
This was a missing test case from the old P2P tests removed in #5453, which makes sure that all nodes are able to peer with each other regardless of how they discover peers.
Fixes#2795, since the default CI testnet uses a combination of (partially meshed) persistent peers and PEX-based seed nodes.
## Description
In blocks_results we use the proto definition of abciResponses: 2672b91ab0/rpc/core/blocks.go (L152-L155), this leads to the use of the proto definition of the pubkey which is an interface in go (oneof). The interface must be registered with the JSON encoder to have it work correctly.
A clearer divide between proto types and native types is needed.
Closes: #XXX
Partial fix for #5291.
This adds a basic set of test cases for core network invariants. Although small, it is sufficient to replace and extend the current set of P2P tests. Further test cases can be added later.
## Description
Add simple `NoBlockResponse` handling to blockchain reactor v1. I tested before and after with erik's e2e testing and was not able to reproduce the inability to sync after the changes were applied
Closes: #5394
description:'"--dup-validators" (multiple validators share the same key) and(or) "--super-byzantine-validators" (byzantine validators have just shy of 2/3 the voting weight)'
required:false
default:''
concurrency:
description:'How many workers should we run? Must be an integer and >= 10, optionally followed by n (e.g. 3n) to multiply by the number of nodes.'
required:true
default:10
timeLimit:
description:'Excluding setup and teardown, how long should a test run for, in seconds?'
required:true
default:60
tendermintUrl:
description:'Where to grab the Tendermint tarball (w/ linux/amd64 binary)'
at the RFC stage will build collective understanding of the dimensions
of the problems and help structure conversations around trade-offs.
@@ -106,39 +107,23 @@ specify exactly the dependency you want to update, eg.
We use [Protocol Buffers](https://developers.google.com/protocol-buffers) along with [gogoproto](https://github.com/gogo/protobuf) to generate code for use across Tendermint Core.
For linting and checking breaking changes, we use [buf](https://buf.build/). If you would like to run linting and check if the changes you have made are breaking then you will need to have docker running locally. Then the linting cmd will be `make proto-lint` and the breaking changes check will be `make proto-check-breaking`.
For linting, checking breaking changes and generating proto stubs, we use [buf](https://buf.build/). If you would like to run linting and check if the changes you have made are breaking then you will need to have docker running locally. Then the linting cmd will be `make proto-lint` and the breaking changes check will be `make proto-check-breaking`.
There are two ways to generate your proto stubs.
We use [Docker](https://www.docker.com/) to generate the protobuf stubs. To generate the stubs yourself, make sure docker is running then run `make proto-gen`.
1. Use Docker, pull an image that will generate your proto stubs with no need to install anything. `make proto-gen-docker`
2. Run `make proto-gen` after installing `protoc` and gogoproto, you can do this by running `make protobuf`.
### Visual Studio Code
### Installation Instructions
If you are a VS Code user, you may want to add the following to your `.vscode/settings.json`:
To install `protoc`, download an appropriate release (<https://github.com/protocolbuffers/protobuf>) and then move the provided binaries into your PATH (follow instructions in README included with the download).
To install `gogoproto`, do the following:
```sh
go get github.com/gogo/protobuf/gogoproto
cd$GOPATH/pkg/mod/github.com/gogo/protobuf@v1.3.1 # or wherever go get installs things
make install
```
You should now be able to run `make proto-gen` from inside the root Tendermint directory to generate new files from proto files.
## Vagrant
If you are a [Vagrant](https://www.vagrantup.com/) user, you can get started
hacking Tendermint with the commands below.
NOTE: In case you installed Vagrant in 2017, you might need to run
`vagrant box update` to upgrade to the latest `ubuntu/xenial64`.
```sh
vagrant up
vagrant ssh
make test
```json
{
"protoc":{
"options":[
"--proto_path=${workspaceRoot}/proto",
"--proto_path=${workspaceRoot}/third_party/proto"
]
}
}
```
## Changelog
@@ -242,75 +227,221 @@ Fixes #nnnn
Each PR should have one commit once it lands on `master`; this can be accomplished by using the "squash and merge" button on Github. Be sure to edit your commit message, though!
### Release Procedure
### Release procedure
#### Major Release
#### A note about backport branches
Tendermint's `master` branch is under active development.
Releases are specified using tags and are built from long-lived "backport" branches.
Each release "line" (e.g. 0.34 or 0.33) has its own long-lived backport branch,
and the backport branches have names like `v0.34.x` or `v0.33.x`
(literally, `x`; it is not a placeholder in this case).
1. start on `master`
2. run integration tests (see `test_integrations` in Makefile)
3. prepare release in a pull request against `master` (to be squash merged):
- copy `CHANGELOG_PENDING.md` to top of `CHANGELOG.md`
- run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for
all issues
- run `bash ./scripts/authors.sh` to get a list of authors since the latest
release, and add the github aliases of external contributors to the top of
the changelog. To lookup an alias from an email, try `bash ./scripts/authors.sh <email>`
- reset the `CHANGELOG_PENDING.md`
- bump Tendermint version in `version.go`
- bump P2P and block protocol versions in `version.go`, if necessary
- bump ABCI protocol version in `version.go`, if necessary
- make sure all significant breaking changes are covered in `UPGRADING.md`
4. push your changes with prepared release details to `vX.X` (this will trigger the release `vX.X.0`)
5. merge back to master (don't squash merge!)
As non-breaking changes land on `master`, they should also be backported (cherry-picked)
to these backport branches.
#### Minor Release
We use Mergify's [backport feature](https://mergify.io/features/backports) to automatically backport
to the needed branch. There should be a label for any backport branch that you'll be targeting.
To notify the bot to backport a pull request, mark the pull request with
the label `S:backport-to-<backport_branch>`.
Once the original pull request is merged, the bot will try to cherry-pick the pull request
to the backport branch. If the bot fails to backport, it will open a pull request.
The author of the original pull request is responsible for solving the conflicts and
merging the pull request.
Minor releases are done differently from major releases: They are built off of long-lived release candidate branches, rather than from master.
#### Creating a backport branch
If this is the first release candidate for a major release, you get to have the honor of creating
the backport branch!
1. Checkout the long-lived release candidate branch: `git checkout rcX/vX.X.X`
2. Run integration tests: `make test_integrations`
Note that, after creating the backport branch, you'll also need to update the tags on `master`
so that `go mod` is able to order the branches correctly. You should tag `master` with a "dev" tag
that is "greater than" the backport branches tags. See #6072 for more context.
In the following example, we'll assume that we're making a backport branch for
the 0.35.x line.
1. Start on `master`
2. Create the backport branch:
`git checkout -b v0.35.x`
3. Go back to master and tag it as the dev branch for the _next_ major release and push it back up:
`git tag -a v0.36.0-dev; git push v0.36.0-dev`
4. Create a new workflow to run the e2e nightlies for this backport branch.
(See https://github.com/tendermint/tendermint/blob/master/.github/workflows/e2e-nightly-34x.yml
for an example.)
#### Release candidates
Before creating an official release, especially a major release, we may want to create a
release candidate (RC) for our friends and partners to test out. We use git tags to
create RCs, and we build them off of backport branches.
Tags for RCs should follow the "standard" release naming conventions, with `-rcX` at the end
(for example, `v0.35.0-rc0`).
(Note that branches and tags _cannot_ have the same names, so it's important that these branches
have distinct names from the tags/release names.)
If this is the first RC for a major release, you'll have to make a new backport branch (see above).
Otherwise:
1. Start from the backport branch (e.g. `v0.35.x`).
1. Run the integration tests and the e2e nightlies
- Move the changes included in `CHANGELOG_PENDING.md` into `CHANGELOG.md`.
- Run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for
all PRs
- Ensure that UPGRADING.md is up-to-date and includes notes on any breaking changes
or other upgrading flows.
- Bump TMVersionDefault version in `version.go`
- Bump P2P and block protocol versions in `version.go`, if necessary
- Bump ABCI protocol version in `version.go`, if necessary
1. Open a PR with these changes against the backport branch.
1. Once these changes have landed on the backport branch, be sure to pull them back down locally.
2. Once you have the changes locally, create the new tag, specifying a name and a tag "message":
`git tag -a v0.35.0-rc0 -m "Release Candidate v0.35.0-rc0`
3. Push the tag back up to origin:
`git push origin v0.35.0-rc0`
Now the tag should be available on the repo's releases page.
4. Future RCs will continue to be built off of this branch.
Note that this process should only be used for "true" RCs--
release candidates that, if successful, will be the next release.
For more experimental "RCs," create a new, short-lived branch and tag that instead.
#### Major release
This major release process assumes that this release was preceded by release candidates.
If there were no release candidates, begin by creating a backport branch, as described above.
1. Start on the backport branch (e.g. `v0.35.x`)
2. Run integration tests and the e2e nightlies.
3. Prepare the release:
-copy `CHANGELOG_PENDING.md` to top of `CHANGELOG.md`
- run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for all issues
- run `bash ./scripts/authors.sh` to get a list of authors since the latest release, and add the GitHub aliases of external contributors to the top of the CHANGELOG. To lookup an alias from an email, try `bash ./scripts/authors.sh <email>`
- reset the `CHANGELOG_PENDING.md`
-bump Tendermint version in `version.go`
- bump P2P and block protocol versions in `version.go`, if necessary
-bump ABCI protocol version in `version.go`, if necessary
- make sure all significant breaking changes are covered in `UPGRADING.md`
4. Create a release branch `release/vX.X.x` off the release candidate branch:
-`git checkout -b release/vX.X.x`
-`git push -u origin release/vX.X.x`
- Note that all branches prefixed with `release` are protected once pushed. You will need admin help to make any changes to the branch.
5. Open a pull request of the new minor release branch onto the latest major release branch `vX.X` and then rebase to merge. This will start the release process.
-"Squash" changes from the changelog entries for the RCs into a single entry,
and add all changes included in `CHANGELOG_PENDING.md`.
(Squashing includes both combining all entries, as well as removing or simplifying
any intra-RC changes. It may also help to alphabetize the entries by package name.)
-Run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for
all PRs
-Ensure that UPGRADING.md is up-to-date and includes notes on any breaking changes
or other upgrading flows.
- Bump TMVersionDefault version in `version.go`
-Bump P2P and block protocol versions in `version.go`, if necessary
-Bump ABCI protocol version in `version.go`, if necessary
4. Open a PR with these changes against the backport branch.
5. Once these changes are on the backport branch, push a tag with prepared release details.
This will trigger the actual release `v0.35.0`.
-`git tag -a v0.35.0 -m 'Release v0.35.0'`
-`git push origin v0.35.0`
7. Make sure that `master` is updated with the latest `CHANGELOG.md`, `CHANGELOG_PENDING.md`, and `UPGRADING.md`.
#### Minor release (point releases)
Minor releases are done differently from major releases: They are built off of long-lived backport branches, rather than from master.
As non-breaking changes land on `master`, they should also be backported (cherry-picked) to these backport branches.
Minor releases don't have release candidates by default, although any tricky changes may merit a release candidate.
To create a minor release:
1. Checkout the long-lived backport branch: `git checkout v0.35.x`
2. Run integration tests (`make test_integrations`) and the nightlies.
3. Check out a new branch and prepare the release:
- Copy `CHANGELOG_PENDING.md` to top of `CHANGELOG.md`
- Run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for all issues
- Run `bash ./scripts/authors.sh` to get a list of authors since the latest release, and add the GitHub aliases of external contributors to the top of the CHANGELOG. To lookup an alias from an email, try `bash ./scripts/authors.sh <email>`
- Reset the `CHANGELOG_PENDING.md`
- Bump the ABCI version number, if necessary.
(Note that ABCI follows semver, and that ABCI versions are the only versions
which can change during minor releases, and only field additions are valid minor changes.)
4. Open a PR with these changes that will land them back on `v0.35.x`
5. Once this change has landed on the backport branch, make sure to pull it locally, then push a tag.
-`git tag -a v0.35.1 -m 'Release v0.35.1'`
-`git push origin v0.35.1`
6. Create a pull request back to master with the CHANGELOG & version changes from the latest release.
- Remove all `R:minor` labels from the pull requests that were included in the release.
- Do not merge the release branch into master.
7. Delete the former long lived release candidate branch once the release has been made.
8. Create a new release candidate branch to be used for the next release.
#### Backport Release
1. start from the existing release branch you want to backport changes to (e.g. v0.30)
Branch to a release/vX.X.X branch locally (e.g. release/v0.30.7)
2. cherry pick the commit(s) that contain the changes you want to backport (usually these commits are from squash-merged PRs which were already reviewed)
3. steps 2 and 3 from [Major Release](#major-release)
4. push changes to release/vX.X.X branch
5. open a PR against the existing vX.X branch
- Do not merge the backport branch into master.
## Testing
All repos should be hooked up to [CircleCI](https://circleci.com/).
### Unit tests
If they have `.go` files in the root directory, they will be automatically
tested by circle using `go test -v -race ./...`. If not, they will need a
`circle.yml`. Ideally, every repo has a `Makefile` that defines `make test` and
includes its continuous integration status using a badge in the `README.md`.
Unit tests are located in `_test.go` files as directed by [the Go testing
package](https://golang.org/pkg/testing/). If you're adding or removing a
function, please check there's a `TestType_Method` test for it.
Run: `make test`
### Integration tests
Integration tests are also located in `_test.go` files. What differentiates
them is a more complicated setup, which usually involves setting up two or more
components.
Run: `make test_integrations`
### End-to-end tests
End-to-end tests are used to verify a fully integrated Tendermint network.
See [README](./test/e2e/README.md) for details.
Run:
```sh
cd test/e2e &&\
make &&\
./build/runner -f networks/ci.toml
```
### Model-based tests (ADVANCED)
*NOTE: if you're just submitting your first PR, you won't need to touch these
most probably (99.9%)*.
For components, that have been [formally
verified](https://en.wikipedia.org/wiki/Formal_verification) using
[TLA+](https://en.wikipedia.org/wiki/TLA%2B), it may be possible to generate
tests using a combination of the [Apalache Model
Checker](https://apalache.informal.systems/) and [tendermint-rs testgen
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language -
and securely replicates it on many machines.
Tendermint Core is a Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines.
For protocol details, see [the specification](https://github.com/tendermint/spec).
@@ -30,25 +29,26 @@ see our recent paper, "[The latest gossip on BFT consensus](https://arxiv.org/ab
Please do not depend on master as your production branch. Use [releases](https://github.com/tendermint/tendermint/releases) instead.
Tendermint is being used in production in both private and public environments,
most notably the blockchains of the [Cosmos Network](https://cosmos.network/).
However, we are still making breaking changes to the protocol and the APIs and have not yet released v1.0.
Tendermint has been in the production of private and public environments, most notably the blockchains of the Cosmos Network. we haven't released v1.0 yet since we are making breaking changes to the protocol and the APIs.
See below for more details about [versioning](#versioning).
In any case, if you intend to run Tendermint in production, we're happy to help. You can
contact us [over email](mailto:hello@interchain.berlin) or [join the chat](https://discord.gg/AzefAFd).
contact us [over email](mailto:hello@interchain.berlin) or [join the chat](https://discord.gg/cosmosnetwork).
## Security
To report a security vulnerability, see our [bug bounty
program](https://hackerone.com/tendermint).
For examples of the kinds of bugs we're looking for, see [our security policy](SECURITY.md)
For examples of the kinds of bugs we're looking for, see [our security policy](SECURITY.md).
We also maintain a dedicated mailing list for security updates. We will only ever use this mailing list
to notify you of vulnerabilities and fixes in Tendermint Core. You can subscribe [here](http://eepurl.com/gZ5hQD).
## Minimum requirements
| Requirement | Notes |
|----------- | ----------------|
| Go version | Go1.14 or higher |
|-------------|------------------|
| Go version | Go1.16 or higher |
## Documentation
@@ -82,33 +82,12 @@ and familiarize yourself with our
Tendermint uses [Semantic Versioning](http://semver.org/) to determine when and how the version changes.
According to SemVer, anything in the public API can change at any time before version 1.0.0
To provide some stability to Tendermint users in these 0.X.X days, the MINOR version is used
to signal breaking changes across a subset of the total public API. This subset includes all
interfaces exposed to other processes (cli, rpc, p2p, etc.), but does not
include the Go APIs.
To provide some stability to users of 0.X.X versions of Tendermint, the MINOR version is used
to signal breaking changes across Tendermint's API. This API includes all
publicly exposed types, functions, and methods in non-internal Go packages as well as
the types and methods accessible via the Tendermint RPC interface.
That said, breaking changes in the following packages will be documented in the
CHANGELOG even if they don't lead to MINOR version bumps:
- crypto
- config
- libs
- bech32
- bits
- bytes
- json
- log
- math
- net
- os
- protoio
- rand
- sync
- strings
- service
- node
- rpc/client
- types
Breaking changes to these public APIs will be documented in the CHANGELOG.
### Upgrades
@@ -155,4 +134,14 @@ Additional tooling can be found in [/docs/tools](/docs/tools).
- [The latest gossip on BFT consensus](https://arxiv.org/abs/1807.04938)
- [Master's Thesis on Tendermint](https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769)
- [Original Whitepaper: "Tendermint: Consensus Without Mining"](https://tendermint.com/static/docs/tendermint.pdf)
Tendermint Core is maintained by [Interchain GmbH](https://interchain.berlin).
If you'd like to work full-time on Tendermint Core, [we're hiring](https://interchain-gmbh.breezy.hr/p/682fb7e8a6f601-software-engineer-tendermint-core)!
Funding for Tendermint Core development comes primarily from the [Interchain Foundation](https://interchain.io),
a Swiss non-profit. The Tendermint trademark is owned by [Tendermint Inc.](https://tendermint.com), the for-profit entity
that also maintains [tendermint.com](https://tendermint.com).
@@ -7,54 +7,55 @@ Policy](https://tendermint.com/security), we operate a [bug
bounty](https://hackerone.com/tendermint).
See the policy for more details on submissions and rewards, and see "Example Vulnerabilities" (below) for examples of the kinds of bugs we're most interested in.
### Guidelines
### Guidelines
We require that all researchers:
* Use the bug bounty to disclose all vulnerabilities, and avoid posting vulnerability information in public places, including Github Issues, Discord channels, and Telegram groups
* Make every effort to avoid privacy violations, degradation of user experience, disruption to production systems (including but not limited to the Cosmos Hub), and destruction of data
* Keep any information about vulnerabilities that you’ve discovered confidential between yourself and the Tendermint Core engineering team until the issue has been resolved and disclosed
* Keep any information about vulnerabilities that you’ve discovered confidential between yourself and the Tendermint Core engineering team until the issue has been resolved and disclosed
* Avoid posting personally identifiable information, privately or publicly
If you follow these guidelines when reporting an issue to us, we commit to:
* Not pursue or support any legal action related to your research on this vulnerability
* Work with you to understand, resolve and ultimately disclose the issue in a timely fashion
* Work with you to understand, resolve and ultimately disclose the issue in a timely fashion
## Disclosure Process
## Disclosure Process
Tendermint Core uses the following disclosure process:
1. Once a security report is received, the Tendermint Core team works to verify the issue and confirm its severity level using CVSS.
2. The Tendermint Core team collaborates with the Gaia team to determine the vulnerability’s potential impact on the Cosmos Hub.
3. Patches are prepared for eligible releases of Tendermint in private repositories. See “Supported Releases” below for more information on which releases are considered eligible.
4. If it is determined that a CVE-ID is required, we request a CVE through a CVE Numbering Authority.
1. Once a security report is received, the Tendermint Core team works to verify the issue and confirm its severity level using CVSS.
2. The Tendermint Core team collaborates with the Gaia team to determine the vulnerability’s potential impact on the Cosmos Hub.
3. Patches are prepared for eligible releases of Tendermint in private repositories. See “Supported Releases” below for more information on which releases are considered eligible.
4. If it is determined that a CVE-ID is required, we request a CVE through a CVE Numbering Authority.
5. We notify the community that a security release is coming, to give users time to prepare their systems for the update. Notifications can include forum posts, tweets, and emails to partners and validators, including emails sent to the [Tendermint Security Mailing List](https://berlin.us4.list-manage.com/subscribe?u=431b35421ff7edcc77df5df10&id=3fe93307bc).
6. 24 hours following this notification, the fixes are applied publicly and new releases are issued.
7. Cosmos SDK and Gaia update their Tendermint Core dependencies to use these releases, and then themselves issue new releases.
8. Once releases are available for Tendermint Core, Cosmos SDK and Gaia, we notify the community, again, through the same channels as above. We also publish a Security Advisory on Github and publish the CVE, as long as neither the Security Advisory nor the CVE include any information on how to exploit these vulnerabilities beyond what information is already available in the patch itself.
9. Once the community is notified, we will pay out any relevant bug bounties to submitters.
10. One week after the releases go out, we will publish a post with further details on the vulnerability as well as our response to it.
6. 24 hours following this notification, the fixes are applied publicly and new releases are issued.
7. Cosmos SDK and Gaia update their Tendermint Core dependencies to use these releases, and then themselves issue new releases.
8. Once releases are available for Tendermint Core, Cosmos SDK and Gaia, we notify the community, again, through the same channels as above. We also publish a Security Advisory on Github and publish the CVE, as long as neither the Security Advisory nor the CVE include any information on how to exploit these vulnerabilities beyond what information is already available in the patch itself.
9. Once the community is notified, we will pay out any relevant bug bounties to submitters.
10. One week after the releases go out, we will publish a post with further details on the vulnerability as well as our response to it.
This process can take some time. Every effort will be made to handle the bug in as timely a manner as possible, however it's important that we follow the process described above to ensure that disclosures are handled consistently and to keep Tendermint Core and its downstream dependent projects--including but not limited to Gaia and the Cosmos Hub--as secure as possible.
This process can take some time. Every effort will be made to handle the bug in as timely a manner as possible, however it's important that we follow the process described above to ensure that disclosures are handled consistently and to keep Tendermint Core and its downstream dependent projects--including but not limited to Gaia and the Cosmos Hub--as secure as possible.
### Example Timeline
### Example Timeline
The following is an example timeline for the triage and response. The required roles and team members are described in parentheses after each task; however, multiple people can play each role and each person may play multiple roles.
The following is an example timeline for the triage and response. The required roles and team members are described in parentheses after each task; however, multiple people can play each role and each person may play multiple roles.
#### > 24 Hours Before Release Time
#### 24+ Hours Before Release Time
1. Request CVE number (ADMIN)
2. Gather emails and other contact info for validators (COMMS LEAD)
3.Test fixes on a testnet (TENDERMINT ENG, COSMOS ENG)
4.Write “Security Advisory” for forum (TENDERMINT LEAD)
1. Request CVE number (ADMIN)
2. Gather emails and other contact info for validators (COMMS LEAD)
3.Create patches in a private security repo, and ensure that PRs are open targeting all relevant release branches (TENDERMINT ENG, TENDERMINT LEAD)
4.Test fixes on a testnet (TENDERMINT ENG, COSMOS SDK ENG)
5. Write “Security Advisory” for forum (TENDERMINT LEAD)
#### 24 Hours Before Release Time
1. Post “Security Advisory” pre-notification on forum (TENDERMINT LEAD)
2. Post Tweet linking to forum post (COMMS LEAD)
3. Announce security advisory/link to post in various other social channels (Telegram, Discord) (COMMS LEAD)
4. Send emails to validators or other users (PARTNERSHIPS LEAD)
1. Post “Security Advisory” pre-notification on forum (TENDERMINT LEAD)
2. Post Tweet linking to forum post (COMMS LEAD)
3. Announce security advisory/link to post in various other social channels (Telegram, Discord) (COMMS LEAD)
4. Send emails to validators or other users (PARTNERSHIPS LEAD)
#### Release Time
@@ -64,36 +65,36 @@ The following is an example timeline for the triage and response. The required r
4. Post “Security releases” on forum (TENDERMINT LEAD)
5. Post new Tweet linking to forum post (COMMS LEAD)
6. Remind everyone via social channels (Telegram, Discord) that the release is out (COMMS LEAD)
7. Send emails to validators or other users (COMMS LEAD)
8. Publish Security Advisory and CVE, if CVE has no sensitive information (ADMIN)
7. Send emails to validators or other users (COMMS LEAD)
8. Publish Security Advisory and CVE, if CVE has no sensitive information (ADMIN)
#### After Release Time
1. Write forum post with exploit details (TENDERMINT LEAD)
2. Approve pay-out on HackerOne for submitter (ADMIN)
2. Approve pay-out on HackerOne for submitter (ADMIN)
#### 7 Days After Release Time
1. Publish CVE if it has not yet been published (ADMIN)
1. Publish CVE if it has not yet been published (ADMIN)
2. Publish forum post with exploit details (TENDERMINT ENG, TENDERMINT LEAD)
## Supported Releases
The Tendermint Core team commits to releasing security patch releases for both the latest minor release as well for the major/minor release that the Cosmos Hub is running.
The Tendermint Core team commits to releasing security patch releases for both the latest minor release as well for the major/minor release that the Cosmos Hub is running.
If you are running older versions of Tendermint Core, we encourage you to upgrade at your earliest opportunity so that you can receive security patches directly from the Tendermint repo. While you are welcome to backport security patches to older versions for your own use, we will not publish or promote these backports.
If you are running older versions of Tendermint Core, we encourage you to upgrade at your earliest opportunity so that you can receive security patches directly from the Tendermint repo. While you are welcome to backport security patches to older versions for your own use, we will not publish or promote these backports.
## Scope
The full scope of our bug bounty program is outlined on our [Hacker One program page](https://hackerone.com/tendermint). Please also note that, in the interest of the safety of our users and staff, a few things are explicitly excluded from scope:
* Any third-party services
* Findings from physical testing, such as office access
* Any third-party services
* Findings from physical testing, such as office access
* Findings derived from social engineering (e.g., phishing)
## Example Vulnerabilities
## Example Vulnerabilities
The following is a list of examples of the kinds of vulnerabilities that we’re most interested in. It is not exhaustive: there are other kinds of issues we may also be interested in!
The following is a list of examples of the kinds of vulnerabilities that we’re most interested in. It is not exhaustive: there are other kinds of issues we may also be interested in!
### Specification
@@ -114,6 +115,9 @@ Assuming less than 1/3 of the voting power is Byzantine (malicious):
* A node halting (liveness failure)
* Syncing new and old nodes
Assuming more than 1/3 the voting power is Byzantine:
* Attacks that go unpunished (unhandled evidence)
### Networking
@@ -139,7 +143,7 @@ Attacks may come through the P2P network or the RPC layer:
### Libraries
* Serialization (Amino)
* Serialization
* Reading/Writing files and databases
### Cryptography
@@ -150,5 +154,5 @@ Attacks may come through the P2P network or the RPC layer:
This guide provides instructions for upgrading to specific versions of Tendermint Core.
## v0.35
### ABCI Changes
* Added `AbciVersion` to `RequestInfo`. Applications should check that the ABCI version they expect is being used in order to avoid unimplemented changes errors.
* The method `SetOption` has been removed from the ABCI.Client interface. This feature was used in the early ABCI implementation's.
* Messages are written to a byte stream using uin64 length delimiters instead of int64.
* When mempool `v1` is enabled, transactions broadcasted via `sync` mode may return a successful
response with a transaction hash indicating that the transaction was successfully inserted into
the mempool. While this is true for `v0`, the `v1` mempool reactor may at a later point in time
evict or even drop this transaction after a hash has been returned. Thus, the user or client must
query for that transaction to check if it is still in the mempool.
### Config Changes
* The configuration file field `[fastsync]` has been renamed to `[blocksync]`.
* The top level configuration file field `fast-sync` has moved under the new `[blocksync]`
field as `blocksync.enable`.
*`blocksync.version = "v1"` and `blocksync.version = "v2"` (previously `fastsync`)
are no longer supported. Please use `v0` instead. During the v0.35 release cycle, `v0` was
determined to suit the existing needs and the cost of maintaining the `v1` and `v2` modules
was determined to be greater than necessary.
* All config parameters are now hyphen-case (also known as kebab-case) instead of snake_case. Before restarting the node make sure
you have updated all the variables in your `config.toml` file.
* Added `--mode` flag and `mode` config variable on `config.toml` for setting Mode of the Node: `full` | `validator` | `seed` (default: `full`)
if you want to learn more about State Sync, or if you'd like your application to use it.
(If you don't want to support State Sync in your application, you can just implement these new
ABCI methods as no-ops, leaving them empty.)
* `KV.Pair` has been replaced with `abci.EventAttribute`. The `EventAttribute.Index` field
allows ABCI applications to dictate which events should be indexed.
* The blockchain can now start from an arbitrary initial height,
* The blockchain can now start from an arbitrary initial height,
provided to the application via `RequestInitChain.InitialHeight`.
* ABCI evidence type is now an enum with two recognized types of evidence:
`DUPLICATE_VOTE` and `LIGHT_CLIENT_ATTACK`.
Applications should be able to handle these evidence types
* ABCI evidence type is now an enum with two recognized types of evidence:
`DUPLICATE_VOTE` and `LIGHT_CLIENT_ATTACK`.
Applications should be able to handle these evidence types
(i.e., through slashing or other accountability measures).
* The [`PublicKey` type](https://github.com/tendermint/tendermint/blob/master/proto/tendermint/crypto/keys.proto#L13-L15)
(used in ABCI as part of `ValidatorUpdate`) now uses a `oneof` protobuf type.
Note that since Tendermint only supports ed25519 validator keys, there's only one
* The [`PublicKey` type](https://github.com/tendermint/tendermint/blob/master/proto/tendermint/crypto/keys.proto#L13-L15)
(used in ABCI as part of `ValidatorUpdate`) now uses a `oneof` protobuf type.
Note that since Tendermint only supports ed25519 validator keys, there's only one
option in the `oneof`. For more, see "Protocol Buffers," below.
* The field `Proof`, on the ABCI type `ResponseQuery`, is now named `ProofOps`.
For more, see "Crypto," below.
* The field `Proof`, on the ABCI type `ResponseQuery`, is now named `ProofOps`.
For more, see "Crypto," below.
* The method `SetOption` has been removed from the ABCI.Client interface. This feature was used in the early ABCI implementation's.
### P2P Protocol
The default codec is now proto3, not amino. The schema files can be found in the `/proto`
directory. For more, see "Protobuf," below.
directory. For more, see "Protobuf," below.
### Blockchain Protocol
*`Header#LastResultsHash` previously was the root hash of a Merkle tree built from`ResponseDeliverTx(Code, Data)` responses.
As of 0.34,`Header#LastResultsHash` is now the root hash of a Merkle tree built from:
*`BeginBlock#Events`
* Root hash of a Merkle tree built from `ResponseDeliverTx(Code, Data,
GasWanted, GasUsed, Events)` responses
* `BeginBlock#Events`
* `Header#LastResultsHash`, which is the root hash of a Merkle tree built from
`ResponseDeliverTx(Code, Data)` as of v0.34 also includes `GasWanted` and `GasUsed`
fields.
* Merkle hashes of empty trees previously returned nothing, but now return the hash of an empty input,
to conform with [RFC-6962](https://tools.ietf.org/html/rfc6962).
to conform with [RFC-6962](https://tools.ietf.org/html/rfc6962).
This mainly affects `Header#DataHash`, `Header#LastResultsHash`, and
`Header#EvidenceHash`, which are often empty. Non-empty hashes can also be affected, e.g. if their
inputs depend on other (empty) Merkle hashes, giving different results.
### Transaction Indexing
Tendermint now relies on the application to tell it which transactions to index. This means that
in the `config.toml`, generated by Tendermint, there is no longer a way to specify which
transactions to index. `tx.height` & `tx.hash` will always be indexed when using the `kv` indexer.
Tendermint now relies on the application to tell it which transactions to index. This means that
in the `config.toml`, generated by Tendermint, there is no longer a way to specify which
transactions to index. `tx.height` and `tx.hash` will always be indexed when using the `kv` indexer.
Applications must now choose to either a) enable indexing for all transactions, or
Applications must now choose to either a) enable indexing for all transactions, or
b) allow node operators to decide which transactions to index.
Applications can notify Tendermint to index a specific transaction by setting
Applications can notify Tendermint to index a specific transaction by setting
`Index: bool` to `true` in the Event Attribute:
```go
@@ -82,19 +270,19 @@ Applications can notify Tendermint to index a specific transaction by setting
### Protocol Buffers
Tendermint 0.34 replaces Amino with Protocol Buffers for encoding.
This migration is extensive and results in a number of changes, however,
Tendermint 0.34 replaces Amino with Protocol Buffers for encoding.
This migration is extensive and results in a number of changes, however,
Tendermint only uses the types generated from Protocol Buffers for disk and
wire serialization.
wire serialization.
**This means that these changes should not affect you as a Tendermint user.**
However, Tendermint users and contributors may note the following changes:
* Directory layout changes: All proto files have been moved under one directory, `/proto`.
This is in line with the recommended file layout by [Buf](https://buf.build).
* Directory layout changes: All proto files have been moved under one directory, `/proto`.
This is in line with the recommended file layout by [Buf](https://buf.build).
For more, see the [Buf documentation](https://buf.build/docs/lint-checkers#file_layout).
* ABCI Changes: As noted in the "ABCI Changes" section above, the `PublicKey` type now uses
a `oneof` type.
* ABCI Changes: As noted in the "ABCI Changes" section above, the `PublicKey` type now uses
a `oneof` type.
For more on the Protobuf changes, please see our [blog post on this migration](https://medium.com/tendermint/tendermint-0-34-protocol-buffers-and-you-8c40558939ae).
@@ -108,36 +296,33 @@ Tendermint 0.34 includes new and updated consensus parameters.
#### Evidence Parameters
* `MaxNum`, which caps the total amount of evidence by a absolute number. The default is 50.
* `MaxBytes`, which caps the total amount of evidence. The default is 1048576 (1 MB).
### Crypto
#### Keys
* Keys no longer include a type prefix. For example, ed25519 pubkeys have been renamed from
`PubKeyEd25519` to `PubKey`. This reduces stutter (e.g., `ed25519.PubKey`).
* Keys no longer include a type prefix. For example, ed25519 pubkeys have been renamed from
`PubKeyEd25519` to `PubKey`. This reduces stutter (e.g., `ed25519.PubKey`).
* Keys are now byte slices (`[]byte`) instead of byte arrays (`[<size>]byte`).
* The multisig functionality that was previously in Tendermint now has
a new home within the Cosmos SDK:
* The multisig functionality that was previously in Tendermint now has
* Similarly, secp256k1 has been removed from the Tendermint repo.
There is still [a secp256k1 implementation in the Cosmos SDK](https://github.com/cosmos/cosmos-sdk/tree/443e0c1f89bd3730a731aea30453bd732f7efa35/crypto/keys/secp256k1),
and we recommend you use that package for all your secp256k1 needs.
#### `merkle` Package
* `SimpleHashFromMap()` and `SimpleProofsFromMap()` were removed.
* The prefix `Simple` has been removed. (For example, `SimpleProof` is now called `Proof`.)
* All protobuf messages have been moved to the `/proto` directory.
* The protobuf message `Proof` that contained multiple ProofOp's has been renamed to `ProofOps`.
As noted above, this affects the ABCI type `ResponseQuery`:
* The prefix `Simple` has been removed. (For example, `SimpleProof` is now called `Proof`.)
* All protobuf messages have been moved to the `/proto` directory.
* The protobuf message `Proof` that contained multiple ProofOp's has been renamed to `ProofOps`.
As noted above, this affects the ABCI type `ResponseQuery`:
The field that was named Proof is now named `ProofOps`.
* `HashFromByteSlices` and `ProofsFromByteSlices` now return a hash for empty inputs, to conform with
are now part of the interface. The interface returns errors on all methods and can be used by calling `state.NewStore(dbm.DB)`.
### `privval` Package
All requests are now accompanied by the chain ID from the network.
This is a optional field and can be ignored by key management systems.
It is recommended to check the chain ID if using the same key management system for multiple chains.
This is a optional field and can be ignored by key management systems;
however, if you are using the same key management system for multiple different
blockchains, we recommend that you check the chain ID.
### RPC
`/unsafe_start_cpu_profiler`, `/unsafe_stop_cpu_profiler` and
`/unsafe_write_heap_profile` were removed.
For profiling, please use the pprof server, which can
be enabled through `--rpc.pprof_laddr=X` flag or `pprof_laddr=X` config setting
in the rpc section.
* `/unsafe_start_cpu_profiler`, `/unsafe_stop_cpu_profiler` and
`/unsafe_write_heap_profile` were removed.
For profiling, please use the pprof server, which can
be enabled through `--rpc.pprof_laddr=X` flag or `pprof_laddr=X` config setting
in the rpc section.
* The `Content-Type` header returned on RPC calls is now (correctly) set as `application/json`.
### Version
Version is now set through Go linker flags `ld_flags`. Applications that are using tendermint as a library should set this at compile time.
Example:
```sh
go install -mod=readonly -ldflags "-X github.com/tendermint/tendermint/version.TMCoreSemVer=$(go list -m github.com/tendermint/tendermint | sed 's/ /\@/g') -s -w " -trimpath ./cmd
```
Additionally, the exported constant `version.Version` is now `version.TMCoreSemVer`.
## v0.33.4
@@ -272,12 +473,12 @@ Evidence Params has been changed to include duration.
### Go API
* `libs/common` has been removed in favor of specific pkgs.
* `async`
* `service`
* `rand`
* `net`
* `strings`
* `cmap`
* `async`
* `service`
* `rand`
* `net`
* `strings`
* `cmap`
* removal of `errors` pkg
### RPC Changes
@@ -346,9 +547,9 @@ Prior to the update, suppose your `ResponseDeliverTx` look like:
```go
abci.ResponseDeliverTx{
Tags: []kv.Pair{
{Key: []byte("sender"), Value: []byte("foo")},
{Key: []byte("recipient"), Value: []byte("bar")},
{Key: []byte("amount"), Value: []byte("35")},
{Key: []byte("sender"), Value: []byte("foo")},
{Key: []byte("recipient"), Value: []byte("bar")},
{Key: []byte("amount"), Value: []byte("35")},
}
}
```
@@ -367,14 +568,14 @@ the following `Events`:
```go
abci.ResponseDeliverTx{
Events: []abci.Event{
{
Type: "transfer",
Attributes: kv.Pairs{
{Key: []byte("sender"), Value: []byte("foo")},
{Key: []byte("recipient"), Value: []byte("bar")},
{Key: []byte("amount"), Value: []byte("35")},
},
}
{
Type: "transfer",
Attributes: kv.Pairs{
{Key: []byte("sender"), Value: []byte("foo")},
{Key: []byte("recipient"), Value: []byte("bar")},
{Key: []byte("amount"), Value: []byte("35")},
},
}
}
```
@@ -422,9 +623,9 @@ In this case, the WS client will receive an error with description:
"jsonrpc": "2.0",
"id": "{ID}#event",
"error": {
"code": -32000,
"msg": "Server error",
"data": "subscription was cancelled (reason: client is not pulling messages fast enough)" // or "subscription was cancelled (reason: Tendermint exited)"
"code": -32000,
"msg": "Server error",
"data": "subscription was canceled (reason: client is not pulling messages fast enough)" // or "subscription was canceled (reason: Tendermint exited)"
logger.Info("node WAL does not exist; continuing...")
}
logger.Info("copying node configuration...")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
