M. J. Fromberger 118bfe2087 abci: Flush socket requests and responses immediately. (#6997)
The main effect of this change is to flush the socket client and server message
encoding buffers immediately once the message is fully and correctly encoded.
This allows us to remove the timer and some other special cases, without
changing the observed behaviour of the system.

-- Background

The socket protocol client and server each use a buffered writer to encode
request and response messages onto the underlying connection. This reduces the
possibility of a single message being split across multiple writes, but has the
side-effect that a request may remain buffered for some time.

The implementation worked around this by keeping a ticker that occasionally
triggers a flush, and by flushing the writer in response to an explicit request
baked into the client/server protocol (see also #6994).

These workarounds are both unnecessary: Once a message has been dequeued for
sending and fully encoded in wire format, there is no real use keeping all or
part of it buffered locally.  Moreover, using an asynchronous process to flush
the buffer makes the round-trip performance of the request unpredictable.

-- Benchmarks

Code: https://play.golang.org/p/0ChUOxJOiHt

I found no pre-existing performance benchmarks to justify the flush pattern,
but a natural question is whether this will significantly harm client/server
performance.  To test this, I implemented a simple benchmark that transfers
randomly-sized byte buffers from a no-op "client" to a no-op "server" over a
Unix-domain socket, using a buffered writer, both with and without explicit
flushes after each write.

As the following data show, flushing every time (FLUSH=true) does reduce raw
throughput, but not by a significant amount except for very small request
sizes, where the transfer time is already trivial (1.9μs).  Given that the
client is calibrated for 1MiB transactions, the overhead is not meaningful.

The percentage in each section is the speedup for flushing only when the buffer
is full, relative to flushing every block.  The benchmark uses the default
buffer size (4096 bytes), which is the same value used by the socket client and
server implementation:

  FLUSH  NBLOCKS  MAX      AVG     TOTAL       ELAPSED       TIME/BLOCK
  false  3957471  512      255     1011165416  2.00018873s   505ns
  true   1068568  512      255     273064368   2.000217051s  1.871µs
                                                             (73%)

  false  536096   4096     2048    1098066401  2.000229108s  3.731µs
  true   477911   4096     2047    978746731   2.000177825s  4.185µs
                                                             (10.8%)

  false  124595   16384    8181    1019340160  2.000235086s  16.053µs
  true   120995   16384    8179    989703064   2.000329349s  16.532µs
                                                             (2.9%)

  false  2114     1048576  525693  1111316541  2.000479928s  946.3µs
  true   2083     1048576  526379  1096449173  2.001817137s  961.025µs
                                                             (1.5%)

Note also that the FLUSH=false baseline is actually faster than the production
code, which flushes more often than is required by the buffer filling up.

Moreover, the timer slows down the overall transaction rate of the client and
server, indepenedent of how fast the socket transfer is, so the loss on a real
workload is probably much less.
2021-09-24 15:37:25 -07:00
2021-09-24 14:59:10 -04:00
2021-09-21 11:22:55 -04:00
2017-12-04 15:01:28 -06:00
2016-07-18 11:51:37 -04:00
2021-09-21 10:35:40 +00:00

Tendermint

banner

Byzantine-Fault Tolerant State Machines. Or Blockchain, for short.

version API Reference Go version Discord chat license tendermint/tendermint Sourcegraph

Branch Tests Coverage Linting
master Tests codecov Lint

Tendermint Core is a Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines.

For protocol details, see the specification.

For detailed analysis of the consensus protocol, including safety and liveness proofs, see our recent paper, "The latest gossip on BFT consensus".

Releases

Please do not depend on master as your production branch. Use releases instead.

Tendermint has been in the production of private and public environments, most notably the blockchains of the Cosmos Network. we haven't released v1.0 yet since we are making breaking changes to the protocol and the APIs. See below for more details about versioning.

In any case, if you intend to run Tendermint in production, we're happy to help. You can contact us over email or join the chat.

Security

To report a security vulnerability, see our bug bounty program. For examples of the kinds of bugs we're looking for, see our security policy.

We also maintain a dedicated mailing list for security updates. We will only ever use this mailing list to notify you of vulnerabilities and fixes in Tendermint Core. You can subscribe here.

Minimum requirements

Requirement Notes
Go version Go1.16 or higher

Documentation

Complete documentation can be found on the website.

Install

See the install instructions.

Quick Start

Contributing

Please abide by the Code of Conduct in all interactions.

Before contributing to the project, please take a look at the contributing guidelines and the style guide. You may also find it helpful to read the specifications, watch the Developer Sessions, and familiarize yourself with our Architectural Decision Records.

Versioning

Semantic Versioning

Tendermint uses Semantic Versioning to determine when and how the version changes. According to SemVer, anything in the public API can change at any time before version 1.0.0

To provide some stability to users of 0.X.X versions of Tendermint, the MINOR version is used to signal breaking changes across Tendermint's API. This API includes all publicly exposed types, functions, and methods in non-internal Go packages as well as the types and methods accessible via the Tendermint RPC interface.

Breaking changes to these public APIs will be documented in the CHANGELOG.

Upgrades

In an effort to avoid accumulating technical debt prior to 1.0.0, we do not guarantee that breaking changes (ie. bumps in the MINOR version) will work with existing Tendermint blockchains. In these cases you will have to start a new blockchain, or write something custom to get the old data into the new chain. However, any bump in the PATCH version should be compatible with existing blockchain histories.

For more information on upgrading, see UPGRADING.md.

Supported Versions

Because we are a small core team, we only ship patch updates, including security updates, to the most recent minor release and the second-most recent minor release. Consequently, we strongly recommend keeping Tendermint up-to-date. Upgrading instructions can be found in UPGRADING.md.

Resources

Tendermint Core

For details about the blockchain data structures and the p2p protocols, see the Tendermint specification.

For details on using the software, see the documentation which is also hosted at: https://docs.tendermint.com/master/

Tools

Benchmarking is provided by tm-load-test. Additional tooling can be found in /docs/tools.

Applications

Research

Join us!

Tendermint Core is maintained by Interchain GmbH. If you'd like to work full-time on Tendermint Core, we're hiring!

Funding for Tendermint Core development comes primarily from the Interchain Foundation, a Swiss non-profit. The Tendermint trademark is owned by Tendermint Inc., the for-profit entity that also maintains tendermint.com.

Description
No description provided
Readme Apache-2.0 394 MiB
Languages
Go 85.9%
TeX 7.2%
TLA 4.9%
Shell 0.6%
Python 0.5%
Other 0.8%