mirrors/tendermint
1
0
Fork 0
mirror of https://github.com/tendermint/tendermint.git synced 2026-04-26 18:45:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

spec: migrate spec at v0.7.1. into main (#9098)

Browse Source
This commit is contained in:
Callum Waters
2022-07-27 00:12:36 +02:00
committed by GitHub
parent a11b7743d4 0323bd5365
commit a7049ed3fc
164 changed files with 39721 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.github/CODEOWNERS vendored
View File

@@ -7,4 +7,6 @@
# global owners are only requested if there isn't a more specific
# codeowner specified below. For this reason, the global codeowners
# are often repeated in package-level definitions.
* @ebuchman @cmwaters @tychoish @williambanfield @creachadair
* @ebuchman @cmwaters @tychoish @williambanfield @creachadair @thanethomson @sergio-mena @jmalicevic
/spec @cason @sergio-mena @jmalicevic @milosevic

37
.github/ISSUE_TEMPLATE/proposal.md vendored Normal file
View File

@@ -0,0 +1,37 @@
---
name: Protocol Change Proposal
about: Create a proposal to request a change to the protocol
---
<!-- < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < ☺
v ✰ Thanks for opening an issue! ✰
v Before smashing the submit button please review the template.
v Word of caution: Under-specified proposals may be rejected summarily
☺ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -->
# Protocol Change Proposal
## Summary
<!-- Short, concise description of the proposed change -->
## Problem Definition
<!-- Why do we need this change?
What problems may be addressed by introducing this change?
What benefits does Tendermint stand to gain by including this change?
Are there any disadvantages of including this change? -->
## Proposal
<!-- Detailed description of requirements of implementation -->
____
#### For Admin Use
- [ ] Not duplicate issue
- [ ] Appropriate labels applied
- [ ] Appropriate contributors tagged
- [ ] Contributor assigned/self-assigned

6
.gitignore vendored
View File

@@ -56,3 +56,9 @@ test/fuzz/**/corpus
test/fuzz/**/crashers
test/fuzz/**/suppressions
test/fuzz/**/*.zip
*.aux
*.bbl
*.blg
*.pdf
*.gz
*.dvi

15
.markdownlint.yml
View File

@@ -1,8 +1,11 @@
default: true,
MD007: { "indent": 4 }
default: true
MD001: false
MD007: {indent: 4}
MD013: false
MD024: { siblings_only: true }
MD024: {siblings_only: true}
MD025: false
MD033: { no-inline-html: false }
no-hard-tabs: false
whitespace: false
MD033: false
MD036: false
MD010: false
MD012: false
MD028: false

23
proto/README.md Normal file
View File

@@ -0,0 +1,23 @@
# Protocol Buffers
This sections defines the types and messages shared across implementations. The definition of the data structures are located in the [core/data_structures](../spec/core/data_structures.md) for the core data types and ABCI definitions are located in the [ABCI](../spec/abci/README.md) section.
## Process of Updates
The `.proto` files within this section are core to the protocol and updates must be treated as such.
### Steps
1. Make an issue with the proposed change.
- Within in the issue members from both the Tendermint-go and Tendermint-rs team will leave comments. If there is not consensus on the change an [RFC](../rfc/README.md) may be requested.
1a. Submission of an RFC as a pull request should be made to facilitate further discussion.
1b. Merge the RFC.
2. Make the necessary changes to the `.proto` file(s), [core data structures](../spec/core/data_structures.md) and/or [ABCI protocol](../spec/abci/apps.md).
3. Open issues within Tendermint-go and Tendermint-rs repos. This is used to notify the teams that a change occurred in the spec.
1. Tag the issue with a spec version label. This will notify the team the changed has been made on master but has not entered a release.
### Versioning
The spec repo aims to be versioned. Once it has been versioned, updates to the protobuf files will live on master. After a certain amount of time, decided on by Tendermint-go and Tendermint-rs team leads, a release will be made on the spec repo. The spec may contain minor releases as well, depending on the implementation these changes may lead to a breaking change. If so, the implementation team should open an issue within the spec repo requiring a major release of the spec.
If the steps above were followed each implementation should have issues tagged with a spec change label. Once all issues have been completed the team should signify their readiness for release.

93
spec/README.md Normal file
View File

@@ -0,0 +1,93 @@
---
order: 1
title: Overview
parent:
title: Spec
order: 7
---
# Tendermint Spec
This is a markdown specification of the Tendermint blockchain.
It defines the base data structures, how they are validated,
and how they are communicated over the network.
If you find discrepancies between the spec and the code that
do not have an associated issue or pull request on github,
please submit them to our [bug bounty](https://tendermint.com/security)!
## Contents
- [Overview](#overview)
### Data Structures
- [Encoding and Digests](./core/encoding.md)
- [Blockchain](./core/data_structures.md)
- [State](./core/state.md)
### Consensus Protocol
- [Consensus Algorithm](./consensus/consensus.md)
- [Creating a proposal](./consensus/creating-proposal.md)
- [Time](./consensus/bft-time.md)
- [Light-Client](./consensus/light-client/README.md)
### P2P and Network Protocols
- [The Base P2P Layer](./p2p/node.md): multiplex the protocols ("reactors") on authenticated and encrypted TCP connections
- [Peer Exchange (PEX)](./p2p/messages/pex.md): gossip known peer addresses so peers can find each other
- [Block Sync](./p2p/messages/block-sync.md): gossip blocks so peers can catch up quickly
- [Consensus](./p2p/messages/consensus.md): gossip votes and block parts so new blocks can be committed
- [Mempool](./p2p/messages/mempool.md): gossip transactions so they get included in blocks
- [Evidence](./p2p/messages/evidence.md): sending invalid evidence will stop the peer
### RPC
- [RPC SPEC](./rpc/README.md): Specification of the Tendermint remote procedure call interface.
### Software
- [ABCI](./abci/README.md): Details about interactions between the
application and consensus engine over ABCI
- [Write-Ahead Log](./consensus/wal.md): Details about how the consensus
engine preserves data and recovers from crash failures
## Overview
Tendermint provides Byzantine Fault Tolerant State Machine Replication using
hash-linked batches of transactions. Such transaction batches are called "blocks".
Hence, Tendermint defines a "blockchain".
Each block in Tendermint has a unique index - its Height.
Height's in the blockchain are monotonic.
Each block is committed by a known set of weighted Validators.
Membership and weighting within this validator set may change over time.
Tendermint guarantees the safety and liveness of the blockchain
so long as less than 1/3 of the total weight of the Validator set
is malicious or faulty.
A commit in Tendermint is a set of signed messages from more than 2/3 of
the total weight of the current Validator set. Validators take turns proposing
blocks and voting on them. Once enough votes are received, the block is considered
committed. These votes are included in the _next_ block as proof that the previous block
was committed - they cannot be included in the current block, as that block has already been
created.
Once a block is committed, it can be executed against an application.
The application returns results for each of the transactions in the block.
The application can also return changes to be made to the validator set,
as well as a cryptographic digest of its latest state.
Tendermint is designed to enable efficient verification and authentication
of the latest state of the blockchain. To achieve this, it embeds
cryptographic commitments to certain information in the block "header".
This information includes the contents of the block (eg. the transactions),
the validator set committing the block, as well as the various results returned by the application.
Note, however, that block execution only occurs _after_ a block is committed.
Thus, application results can only be included in the _next_ block.
Also note that information like the transaction results and the validator set are never
directly included in the block - only their cryptographic digests (Merkle roots) are.
Hence, verification of a block requires a separate data structure to store this information.
We call this the `State`. Block verification also requires access to the previous block.

27
spec/abci/README.md Normal file
View File

@@ -0,0 +1,27 @@
---
order: 1
parent:
title: ABCI
order: 2
---
# ABCI
ABCI stands for "**A**pplication **B**lock**c**hain **I**nterface".
ABCI is the interface between Tendermint (a state-machine replication engine)
and your application (the actual state machine). It consists of a set of
_methods_, each with a corresponding `Request` and `Response`message type.
To perform state-machine replication, Tendermint calls the ABCI methods on the
ABCI application by sending the `Request*` messages and receiving the `Response*` messages in return.
All ABCI messages and methods are defined in [protocol buffers](https://github.com/tendermint/spec/blob/master/proto/abci/types.proto).
This allows Tendermint to run with applications written in many programming languages.
This specification is split as follows:
- [Methods and Types](./abci.md) - complete details on all ABCI methods and
message types
- [Applications](./apps.md) - how to manage ABCI application state and other
details about building ABCI applications
- [Client and Server](./client-server.md) - for those looking to implement their
own ABCI application servers

775
spec/abci/abci.md Normal file
View File

@@ -0,0 +1,775 @@
---
order: 1
title: Method and Types
---
# Methods and Types
## Connections
ABCI applications can run either within the _same_ process as the Tendermint
state-machine replication engine, or as a _separate_ process from the state-machine
replication engine. When run within the same process, Tendermint will call the ABCI
application methods directly as Go method calls.
When Tendermint and the ABCI application are run as separate processes, Tendermint
opens four connections to the application for ABCI methods. The connections each
handle a subset of the ABCI method calls. These subsets are defined as follows:
#### **Consensus** connection
* Driven by a consensus protocol and is responsible for block execution.
* Handles the `InitChain`, `BeginBlock`, `DeliverTx`, `EndBlock`, and `Commit` method
calls.
#### **Mempool** connection
* For validating new transactions, before they're shared or included in a block.
* Handles the `CheckTx` calls.
#### **Info** connection
* For initialization and for queries from the user.
* Handles the `Info` and `Query` calls.
#### **Snapshot** connection
* For serving and restoring [state sync snapshots](apps.md#state-sync).
* Handles the `ListSnapshots`, `LoadSnapshotChunk`, `OfferSnapshot`, and `ApplySnapshotChunk` calls.
Additionally, there is a `Flush` method that is called on every connection,
and an `Echo` method that is just for debugging.
More details on managing state across connections can be found in the section on
[ABCI Applications](apps.md).
## Errors
The `Query`, `CheckTx` and `DeliverTx` methods include a `Code` field in their `Response*`.
This field is meant to contain an application-specific response code.
A response code of `0` indicates no error. Any other response code
indicates to Tendermint that an error occurred.
These methods also return a `Codespace` string to Tendermint. This field is
used to disambiguate `Code` values returned by different domains of the
application. The `Codespace` is a namespace for the `Code`.
The `Echo`, `Info`, `InitChain`, `BeginBlock`, `EndBlock`, `Commit` methods
do not return errors. An error in any of these methods represents a critical
issue that Tendermint has no reasonable way to handle. If there is an error in one
of these methods, the application must crash to ensure that the error is safely
handled by an operator.
The handling of non-zero response codes by Tendermint is described below
### CheckTx
The `CheckTx` ABCI method controls what transactions are considered for inclusion in a block.
When Tendermint receives a `ResponseCheckTx` with a non-zero `Code`, the associated
transaction will be not be added to Tendermint's mempool or it will be removed if
it is already included.
### DeliverTx
The `DeliverTx` ABCI method delivers transactions from Tendermint to the application.
When Tendermint recieves a `ResponseDeliverTx` with a non-zero `Code`, the response code is logged.
The transaction was already included in a block, so the `Code` does not influence
Tendermint consensus.
### Query
The `Query` ABCI method query queries the application for information about application state.
When Tendermint receives a `ResponseQuery` with a non-zero `Code`, this code is
returned directly to the client that initiated the query.
## Events
The `CheckTx`, `BeginBlock`, `DeliverTx`, `EndBlock` methods include an `Events`
field in their `Response*`. Applications may respond to these ABCI methods with a set of events.
Events allow applications to associate metadata about ABCI method execution with the
transactions and blocks this metadata relates to.
Events returned via these ABCI methods do not impact Tendermint consensus in any way
and instead exist to power subscriptions and queries of Tendermint state.
An `Event` contains a `type` and a list of `EventAttributes`, which are key-value
string pairs denoting metadata about what happened during the method's execution.
`Event` values can be used to index transactions and blocks according to what happened
during their execution. Note that the set of events returned for a block from
`BeginBlock` and `EndBlock` are merged. In case both methods return the same
key, only the value defined in `EndBlock` is used.
Each event has a `type` which is meant to categorize the event for a particular
`Response*` or `Tx`. A `Response*` or `Tx` may contain multiple events with duplicate
`type` values, where each distinct entry is meant to categorize attributes for a
particular event. Every key and value in an event's attributes must be UTF-8
encoded strings along with the event type itself.
```protobuf
message Event {
string type = 1;
repeated EventAttribute attributes = 2;
}
```
The attributes of an `Event` consist of a `key`, a `value`, and an `index` flag. The
index flag notifies the Tendermint indexer to index the attribute. The value of
the `index` flag is non-deterministic and may vary across different nodes in the network.
```protobuf
message EventAttribute {
bytes key = 1;
bytes value = 2;
bool index = 3; // nondeterministic
}
```
Example:
```go
abci.ResponseDeliverTx{
// ...
Events: []abci.Event{
{
Type: "validator.provisions",
Attributes: []abci.EventAttribute{
abci.EventAttribute{Key: []byte("address"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("amount"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("balance"), Value: []byte("..."), Index: true},
},
},
{
Type: "validator.provisions",
Attributes: []abci.EventAttribute{
abci.EventAttribute{Key: []byte("address"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("amount"), Value: []byte("..."), Index: false},
abci.EventAttribute{Key: []byte("balance"), Value: []byte("..."), Index: false},
},
},
{
Type: "validator.slashed",
Attributes: []abci.EventAttribute{
abci.EventAttribute{Key: []byte("address"), Value: []byte("..."), Index: false},
abci.EventAttribute{Key: []byte("amount"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("reason"), Value: []byte("..."), Index: true},
},
},
// ...
},
}
```
## EvidenceType
Tendermint's security model relies on the use of "evidence". Evidence is proof of
malicious behaviour by a network participant. It is the responsibility of Tendermint
to detect such malicious behaviour. When malicious behavior is detected, Tendermint
will gossip evidence of the behavior to other nodes and commit the evidence to
the chain once it is verified by all validators. This evidence will then be
passed it on to the application through the ABCI. It is the responsibility of the
application to handle the evidence and exercise punishment.
EvidenceType has the following protobuf format:
```proto
enum EvidenceType {
UNKNOWN = 0;
DUPLICATE_VOTE = 1;
LIGHT_CLIENT_ATTACK = 2;
}
```
There are two forms of evidence: Duplicate Vote and Light Client Attack. More
information can be found in either [data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md)
or [accountability](https://github.com/tendermint/spec/blob/master/spec/light-client/accountability/)
## Determinism
ABCI applications must implement deterministic finite-state machines to be
securely replicated by the Tendermint consensus engine. This means block execution
over the Consensus Connection must be strictly deterministic: given the same
ordered set of requests, all nodes will compute identical responses, for all
BeginBlock, DeliverTx, EndBlock, and Commit. This is critical, because the
responses are included in the header of the next block, either via a Merkle root
or directly, so all nodes must agree on exactly what they are.
For this reason, it is recommended that applications not be exposed to any
external user or process except via the ABCI connections to a consensus engine
like Tendermint Core. The application must only change its state based on input
from block execution (BeginBlock, DeliverTx, EndBlock, Commit), and not through
any other kind of request. This is the only way to ensure all nodes see the same
transactions and compute the same results.
If there is some non-determinism in the state machine, consensus will eventually
fail as nodes disagree over the correct values for the block header. The
non-determinism must be fixed and the nodes restarted.
Sources of non-determinism in applications may include:
* Hardware failures
* Cosmic rays, overheating, etc.
* Node-dependent state
* Random numbers
* Time
* Underspecification
* Library version changes
* Race conditions
* Floating point numbers
* JSON serialization
* Iterating through hash-tables/maps/dictionaries
* External Sources
* Filesystem
* Network calls (eg. some external REST API service)
See [#56](https://github.com/tendermint/abci/issues/56) for original discussion.
Note that some methods (`Query, CheckTx, DeliverTx`) return
explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is
intended for the literal output from the application's logger, while the
`Info` is any additional info that should be returned. These are the only fields
that are not included in block header computations, so we don't need agreement
on them. All other fields in the `Response*` must be strictly deterministic.
## Block Execution
The first time a new blockchain is started, Tendermint calls
`InitChain`. From then on, the following sequence of methods is executed for each
block:
`BeginBlock, [DeliverTx], EndBlock, Commit`
where one `DeliverTx` is called for each transaction in the block.
The result is an updated application state.
Cryptographic commitments to the results of DeliverTx, EndBlock, and
Commit are included in the header of the next block.
## State Sync
State sync allows new nodes to rapidly bootstrap by discovering, fetching, and applying
state machine snapshots instead of replaying historical blocks. For more details, see the
[state sync section](../spec/p2p/messages/state-sync.md).
New nodes will discover and request snapshots from other nodes in the P2P network.
A Tendermint node that receives a request for snapshots from a peer will call
`ListSnapshots` on its application to retrieve any local state snapshots. After receiving
snapshots from peers, the new node will offer each snapshot received from a peer
to its local application via the `OfferSnapshot` method.
Snapshots may be quite large and are thus broken into smaller "chunks" that can be
assembled into the whole snapshot. Once the application accepts a snapshot and
begins restoring it, Tendermint will fetch snapshot "chunks" from existing nodes.
The node providing "chunks" will fetch them from its local application using
the `LoadSnapshotChunk` method.
As the new node receives "chunks" it will apply them sequentially to the local
application with `ApplySnapshotChunk`. When all chunks have been applied, the application
`AppHash` is retrieved via an `Info` query. The `AppHash` is then compared to
the blockchain's `AppHash` which is verified via [light client verification](../spec/light-client/verification/README.md).
## Messages
### Echo
* **Request**:
* `Message (string)`: A string to echo back
* **Response**:
* `Message (string)`: The input string
* **Usage**:
* Echo a string to test an abci client/server implementation
### Flush
* **Usage**:
* Signals that messages queued on the client should be flushed to
the server. It is called periodically by the client
implementation to ensure asynchronous requests are actually
sent, and is called immediately to make a synchronous request,
which returns when the Flush response comes back.
### Info
* **Request**:
| Name | Type | Description | Field Number |
|---------------|--------|------------------------------------------|--------------|
| version | string | The Tendermint software semantic version | 1 |
| block_version | uint64 | The Tendermint Block Protocol version | 2 |
| p2p_version | uint64 | The Tendermint P2P Protocol version | 3 |
| abci_version | string | The Tendermint ABCI semantic version | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|---------------------|--------|--------------------------------------------------|--------------|
| data | string | Some arbitrary information | 1 |
| version | string | The application software semantic version | 2 |
| app_version | uint64 | The application protocol version | 3 |
| last_block_height | int64 | Latest block for which the app has called Commit | 4 |
| last_block_app_hash | bytes | Latest result of Commit | 5 |
* **Usage**:
* Return information about the application state.
* Used to sync Tendermint with the application during a handshake
that happens on startup.
* The returned `app_version` will be included in the Header of every block.
* Tendermint expects `last_block_app_hash` and `last_block_height` to
be updated during `Commit`, ensuring that `Commit` is never
called twice for the same block height.
> Note: Semantic version is a reference to [semantic versioning](https://semver.org/). Semantic versions in info will be displayed as X.X.x.
### InitChain
* **Request**:
| Name | Type | Description | Field Number |
|------------------|--------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|--------------|
| time | [google.protobuf.Timestamp](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp) | Genesis time | 1 |
| chain_id | string | ID of the blockchain. | 2 |
| consensus_params | [ConsensusParams](#consensusparams) | Initial consensus-critical parameters. | 3 |
| validators | repeated [ValidatorUpdate](#validatorupdate) | Initial genesis validators, sorted by voting power. | 4 |
| app_state_bytes | bytes | Serialized initial application state. JSON bytes. | 5 |
| initial_height | int64 | Height of the initial block (typically `1`). | 6 |
* **Response**:
| Name | Type | Description | Field Number |
|------------------|----------------------------------------------|-------------------------------------------------|--------------|
| consensus_params | [ConsensusParams](#consensusparams) | Initial consensus-critical parameters (optional | 1 |
| validators | repeated [ValidatorUpdate](#validatorupdate) | Initial validator set (optional). | 2 |
| app_hash | bytes | Initial application hash. | 3 |
* **Usage**:
* Called once upon genesis.
* If ResponseInitChain.Validators is empty, the initial validator set will be the RequestInitChain.Validators
* If ResponseInitChain.Validators is not empty, it will be the initial
validator set (regardless of what is in RequestInitChain.Validators).
* This allows the app to decide if it wants to accept the initial validator
set proposed by tendermint (ie. in the genesis file), or if it wants to use
a different one (perhaps computed based on some application specific
information in the genesis file).
### Query
* **Request**:
| Name | Type | Description | Field Number |
|--------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| data | bytes | Raw query bytes. Can be used with or in lieu of Path. | 1 |
| path | string | Path field of the request URI. Can be used with or in lieu of `data`. Apps MUST interpret `/store` as a query by key on the underlying store. The key SHOULD be specified in the `data` field. Apps SHOULD allow queries over specific types like `/accounts/...` or `/votes/...` | 2 |
| height | int64 | The block height for which you want the query (default=0 returns data for the latest committed block). Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 3 |
| prove | bool | Return Merkle proof with response if possible | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|-----------|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| index | int64 | The index of the key in the tree. | 5 |
| key | bytes | The key of the matching data. | 6 |
| value | bytes | The value of the matching data. | 7 |
| proof_ops | [ProofOps](#proofops) | Serialized proof for the value data, if requested, to be verified against the `app_hash` for the given Height. | 8 |
| height | int64 | The block height from which data was derived. Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 9 |
| codespace | string | Namespace for the `code`. | 10 |
* **Usage**:
* Query for data from the application at current or past height.
* Optionally return Merkle proof.
* Merkle proof includes self-describing `type` field to support many types
of Merkle trees and encoding formats.
### BeginBlock
* **Request**:
| Name | Type | Description | Field Number |
|----------------------|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block's hash. This can be derived from the block header. | 1 |
| header | [Header](../core/data_structures.md#header) | The block header. | 2 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round, and the list of validators and which ones signed the last block. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|---------------------------|-------------------------------------|--------------|
| events | repeated [Event](#events) | type & Key-Value events for indexing | 1 |
* **Usage**:
* Signals the beginning of a new block.
* Called prior to any `DeliverTx` method calls.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header. We may seek to generalize this in the future.
* The `LastCommitInfo` and `ByzantineValidators` can be used to determine
rewards and punishments for the validators.
### CheckTx
* **Request**:
| Name | Type | Description | Field Number |
|------|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| tx | bytes | The request transaction bytes | 1 |
| type | CheckTxType | One of `CheckTx_New` or `CheckTx_Recheck`. `CheckTx_New` is the default and means that a full check of the tranasaction is required. `CheckTx_Recheck` types are used when the mempool is initiating a normal recheck of a transaction. | 2 |
* **Response**:
| Name | Type | Description | Field Number |
|------------|---------------------------|-----------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| data | bytes | Result bytes, if any. | 2 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| gas_wanted | int64 | Amount of gas requested for transaction. | 5 |
| gas_used | int64 | Amount of gas consumed by transaction. | 6 |
| events | repeated [Event](#events) | Type & Key-Value events for indexing transactions (eg. by account). | 7 |
| codespace | string | Namespace for the `code`. | 8 |
| sender | string | The transaction's sender (e.g. the signer) | 9 |
| priority | int64 | The transaction's priority (for mempool ordering) | 10 |
* **Usage**:
* Technically optional - not involved in processing blocks.
* Guardian of the mempool: every node runs `CheckTx` before letting a
transaction into its local mempool.
* The transaction may come from an external user or another node
* `CheckTx` validates the transaction against the current state of the application,
for example, checking signatures and account balances, but does not apply any
of the state changes described in the transaction.
not running code in a virtual machine.
* Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast to
other nodes or included in a proposal block.
* Tendermint attributes no other value to the response code
### DeliverTx
* **Request**:
| Name | Type | Description | Field Number |
|------|-------|--------------------------------|--------------|
| tx | bytes | The request transaction bytes. | 1 |
* **Response**:
| Name | Type | Description | Field Number |
|------------|---------------------------|-----------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| data | bytes | Result bytes, if any. | 2 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| gas_wanted | int64 | Amount of gas requested for transaction. | 5 |
| gas_used | int64 | Amount of gas consumed by transaction. | 6 |
| events | repeated [Event](#events) | Type & Key-Value events for indexing transactions (eg. by account). | 7 |
| codespace | string | Namespace for the `code`. | 8 |
* **Usage**:
* [**Required**] The core method of the application.
* When `DeliverTx` is called, the application must execute the transaction in full before returning control to Tendermint.
* `ResponseDeliverTx.Code == 0` only if the transaction is fully valid.
### EndBlock
* **Request**:
| Name | Type | Description | Field Number |
|--------|-------|------------------------------------|--------------|
| height | int64 | Height of the block just executed. | 1 |
* **Response**:
| Name | Type | Description | Field Number |
|-------------------------|----------------------------------------------|-----------------------------------------------------------------|--------------|
| validator_updates | repeated [ValidatorUpdate](#validatorupdate) | Changes to validator set (set voting power to 0 to remove). | 1 |
| consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to consensus-critical time, size, and other parameters. | 2 |
| events | repeated [Event](#events) | Type & Key-Value events for indexing | 3 |
* **Usage**:
* Signals the end of a block.
* Called after all the transactions for the current block have been delivered, prior to the block's `Commit` message.
* Optional `validator_updates` triggered by block `H`. These updates affect validation
for blocks `H+1`, `H+2`, and `H+3`.
* Heights following a validator update are affected in the following way:
* `H+1`: `NextValidatorsHash` includes the new `validator_updates` value.
* `H+2`: The validator set change takes effect and `ValidatorsHash` is updated.
* `H+3`: `LastCommitInfo` is changed to include the altered validator set.
* `consensus_param_updates` returned for block `H` apply to the consensus
params for block `H+1`. For more information on the consensus parameters,
see the [application spec entry on consensus parameters](../spec/abci/apps.md#consensus-parameters).
### Commit
* **Request**:
| Name | Type | Description | Field Number |
|--------|-------|------------------------------------|--------------|
Commit signals the application to persist application state. It takes no parameters.
* **Response**:
| Name | Type | Description | Field Number |
|---------------|-------|------------------------------------------------------------------------|--------------|
| data | bytes | The Merkle root hash of the application state. | 2 |
| retain_height | int64 | Blocks below this height may be removed. Defaults to `0` (retain all). | 3 |
* **Usage**:
* Signal the application to persist the application state.
* Return an (optional) Merkle root hash of the application state
* `ResponseCommit.Data` is included as the `Header.AppHash` in the next block
* it may be empty
* Later calls to `Query` can return proofs about the application state anchored
in this Merkle root hash
* Note developers can return whatever they want here (could be nothing, or a
constant string, etc.), so long as it is deterministic - it must not be a
function of anything that did not come from the
BeginBlock/DeliverTx/EndBlock methods.
* Use `RetainHeight` with caution! If all nodes in the network remove historical
blocks then this data is permanently lost, and no new nodes will be able to
join the network and bootstrap. Historical blocks may also be required for
other purposes, e.g. auditing, replay of non-persisted heights, light client
verification, and so on.
### ListSnapshots
* **Request**:
| Name | Type | Description | Field Number |
|--------|-------|------------------------------------|--------------|
Empty request asking the application for a list of snapshots.
* **Response**:
| Name | Type | Description | Field Number |
|-----------|--------------------------------|--------------------------------|--------------|
| snapshots | repeated [Snapshot](#snapshot) | List of local state snapshots. | 1 |
* **Usage**:
* Used during state sync to discover available snapshots on peers.
* See `Snapshot` data type for details.
### LoadSnapshotChunk
* **Request**:
| Name | Type | Description | Field Number |
|--------|--------|-----------------------------------------------------------------------|--------------|
| height | uint64 | The height of the snapshot the chunks belongs to. | 1 |
| format | uint32 | The application-specific format of the snapshot the chunk belongs to. | 2 |
| chunk | uint32 | The chunk index, starting from `0` for the initial chunk. | 3 |
* **Response**:
| Name | Type | Description | Field Number |
|-------|-------|-------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| chunk | bytes | The binary chunk contents, in an arbitray format. Chunk messages cannot be larger than 16 MB _including metadata_, so 10 MB is a good starting point. | 1 |
* **Usage**:
* Used during state sync to retrieve snapshot chunks from peers.
### OfferSnapshot
* **Request**:
| Name | Type | Description | Field Number |
|----------|-----------------------|--------------------------------------------------------------------------|--------------|
| snapshot | [Snapshot](#snapshot) | The snapshot offered for restoration. | 1 |
| app_hash | bytes | The light client-verified app hash for this height, from the blockchain. | 2 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|-------------------|-----------------------------------|--------------|
| result | [Result](#result) | The result of the snapshot offer. | 1 |
#### Result
```proto
enum Result {
UNKNOWN = 0; // Unknown result, abort all snapshot restoration
ACCEPT = 1; // Snapshot is accepted, start applying chunks.
ABORT = 2; // Abort snapshot restoration, and don't try any other snapshots.
REJECT = 3; // Reject this specific snapshot, try others.
REJECT_FORMAT = 4; // Reject all snapshots with this `format`, try others.
REJECT_SENDER = 5; // Reject all snapshots from all senders of this snapshot, try others.
}
```
* **Usage**:
* `OfferSnapshot` is called when bootstrapping a node using state sync. The application may
accept or reject snapshots as appropriate. Upon accepting, Tendermint will retrieve and
apply snapshot chunks via `ApplySnapshotChunk`. The application may also choose to reject a
snapshot in the chunk response, in which case it should be prepared to accept further
`OfferSnapshot` calls.
* Only `AppHash` can be trusted, as it has been verified by the light client. Any other data
can be spoofed by adversaries, so applications should employ additional verification schemes
to avoid denial-of-service attacks. The verified `AppHash` is automatically checked against
the restored application at the end of snapshot restoration.
* For more information, see the `Snapshot` data type or the [state sync section](../spec/p2p/messages/state-sync.md).
### ApplySnapshotChunk
* **Request**:
| Name | Type | Description | Field Number |
|--------|--------|-----------------------------------------------------------------------------|--------------|
| index | uint32 | The chunk index, starting from `0`. Tendermint applies chunks sequentially. | 1 |
| chunk | bytes | The binary chunk contents, as returned by `LoadSnapshotChunk`. | 2 |
| sender | string | The P2P ID of the node who sent this chunk. | 3 |
* **Response**:
| Name | Type | Description | Field Number |
|----------------|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| result | Result (see below) | The result of applying this chunk. | 1 |
| refetch_chunks | repeated uint32 | Refetch and reapply the given chunks, regardless of `result`. Only the listed chunks will be refetched, and reapplied in sequential order. | 2 |
| reject_senders | repeated string | Reject the given P2P senders, regardless of `Result`. Any chunks already applied will not be refetched unless explicitly requested, but queued chunks from these senders will be discarded, and new chunks or other snapshots rejected. | 3 |
```proto
enum Result {
UNKNOWN = 0; // Unknown result, abort all snapshot restoration
ACCEPT = 1; // The chunk was accepted.
ABORT = 2; // Abort snapshot restoration, and don't try any other snapshots.
RETRY = 3; // Reapply this chunk, combine with `RefetchChunks` and `RejectSenders` as appropriate.
RETRY_SNAPSHOT = 4; // Restart this snapshot from `OfferSnapshot`, reusing chunks unless instructed otherwise.
REJECT_SNAPSHOT = 5; // Reject this snapshot, try a different one.
}
```
* **Usage**:
* The application can choose to refetch chunks and/or ban P2P peers as appropriate. Tendermint
will not do this unless instructed by the application.
* The application may want to verify each chunk, e.g. by attaching chunk hashes in
`Snapshot.Metadata` and/or incrementally verifying contents against `AppHash`.
* When all chunks have been accepted, Tendermint will make an ABCI `Info` call to verify that
`LastBlockAppHash` and `LastBlockHeight` matches the expected values, and record the
`AppVersion` in the node state. It then switches to fast sync or consensus and joins the
network.
* If Tendermint is unable to retrieve the next chunk after some time (e.g. because no suitable
peers are available), it will reject the snapshot and try a different one via `OfferSnapshot`.
The application should be prepared to reset and accept it or abort as appropriate.
## Data Types
Most of the data structures used in ABCI are shared [common data structures](../spec/core/data_structures.md). In certain cases, ABCI uses different data structures which are documented here:
### Validator
* **Fields**:
| Name | Type | Description | Field Number |
|---------|-------|---------------------------------------------------------------------|--------------|
| address | bytes | [Address](../core/data_structures.md#address) of validator | 1 |
| power | int64 | Voting power of the validator | 3 |
* **Usage**:
* Validator identified by address
* Used in RequestBeginBlock as part of VoteInfo
* Does not include PubKey to avoid sending potentially large quantum pubkeys
over the ABCI
### ValidatorUpdate
* **Fields**:
| Name | Type | Description | Field Number |
|---------|--------------------------------------------------|-------------------------------|--------------|
| pub_key | [Public Key](../core/data_structures.md#pub_key) | Public key of the validator | 1 |
| power | int64 | Voting power of the validator | 2 |
* **Usage**:
* Validator identified by PubKey
* Used to tell Tendermint to update the validator set
### VoteInfo
* **Fields**:
| Name | Type | Description | Field Number |
|-------------------|-------------------------|--------------------------------------------------------------|--------------|
| validator | [Validator](#validator) | A validator | 1 |
| signed_last_block | bool | Indicates whether or not the validator signed the last block | 2 |
* **Usage**:
* Indicates whether a validator signed the last block, allowing for rewards
based on validator availability
### Evidence
* **Fields**:
| Name | Type | Description | Field Number |
|--------------------|--------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|--------------|
| type | [EvidenceType](#evidencetype) | Type of the evidence. An enum of possible evidence's. | 1 |
| validator | [Validator](#validator) | The offending validator | 2 |
| height | int64 | Height when the offense occurred | 3 |
| time | [google.protobuf.Timestamp](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp) | Time of the block that was committed at the height that the offense occurred | 4 |
| total_voting_power | int64 | Total voting power of the validator set at height `Height` | 5 |
#### EvidenceType
* **Fields**
EvidenceType is an enum with the listed fields:
| Name | Field Number |
|---------------------|--------------|
| UNKNOWN | 0 |
| DUPLICATE_VOTE | 1 |
| LIGHT_CLIENT_ATTACK | 2 |
### LastCommitInfo
* **Fields**:
| Name | Type | Description | Field Number |
|-------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|--------------|
| round | int32 | Commit round. Reflects the total amount of rounds it took to come to consensus for the current block. | 1 |
| votes | repeated [VoteInfo](#voteinfo) | List of validators addresses in the last validator set with their voting power and whether or not they signed a vote. | 2 |
### ConsensusParams
* **Fields**:
| Name | Type | Description | Field Number |
|-----------|---------------------------------------------------------------|------------------------------------------------------------------------------|--------------|
| block | [BlockParams](../core/data_structures.md#blockparams) | Parameters limiting the size of a block and time between consecutive blocks. | 1 |
| evidence | [EvidenceParams](../core/data_structures.md#evidenceparams) | Parameters limiting the validity of evidence of byzantine behaviour. | 2 |
| validator | [ValidatorParams](../core/data_structures.md#validatorparams) | Parameters limiting the types of public keys validators can use. | 3 |
| version | [VersionsParams](../core/data_structures.md#versionparams) | The ABCI application version. | 4 |
### ProofOps
* **Fields**:
| Name | Type | Description | Field Number |
|------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| ops | repeated [ProofOp](#proofop) | List of chained Merkle proofs, of possibly different types. The Merkle root of one op is the value being proven in the next op. The Merkle root of the final op should equal the ultimate root hash being verified against.. | 1 |
### ProofOp
* **Fields**:
| Name | Type | Description | Field Number |
|------|--------|------------------------------------------------|--------------|
| type | string | Type of Merkle proof and how it's encoded. | 1 |
| key | bytes | Key in the Merkle tree that this proof is for. | 2 |
| data | bytes | Encoded Merkle proof for the key. | 3 |
### Snapshot
* **Fields**:
| Name | Type | Description | Field Number |
|----------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| height | uint64 | The height at which the snapshot was taken (after commit). | 1 |
| format | uint32 | An application-specific snapshot format, allowing applications to version their snapshot data format and make backwards-incompatible changes. Tendermint does not interpret this. | 2 |
| chunks | uint32 | The number of chunks in the snapshot. Must be at least 1 (even if empty). | 3 |
| hash | bytes | TAn arbitrary snapshot hash. Must be equal only for identical snapshots across nodes. Tendermint does not interpret the hash, it only compares them. | 3 |
| metadata | bytes | Arbitrary application metadata, for example chunk hashes or other verification data. | 3 |
* **Usage**:
* Used for state sync snapshots, see the [state sync section](../spec/p2p/messages/state-sync.md) for details.
* A snapshot is considered identical across nodes only if _all_ fields are equal (including
`Metadata`). Chunks may be retrieved from all nodes that have the same snapshot.
* When sent across the network, a snapshot message can be at most 4 MB.

671
spec/abci/apps.md Normal file
View File

@@ -0,0 +1,671 @@
---
order: 2
title: Applications
---
# Applications
Please ensure you've first read the spec for [ABCI Methods and Types](abci.md)
Here we cover the following components of ABCI applications:
- [Connection State](#connection-state) - the interplay between ABCI connections and application state
and the differences between `CheckTx` and `DeliverTx`.
- [Transaction Results](#transaction-results) - rules around transaction
results and validity
- [Validator Set Updates](#validator-updates) - how validator sets are
changed during `InitChain` and `EndBlock`
- [Query](#query) - standards for using the `Query` method and proofs about the
application state
- [Crash Recovery](#crash-recovery) - handshake protocol to synchronize
Tendermint and the application on startup.
- [State Sync](#state-sync) - rapid bootstrapping of new nodes by restoring state machine snapshots
## Connection State
Since Tendermint maintains four concurrent ABCI connections, it is typical
for an application to maintain a distinct state for each, and for the states to
be synchronized during `Commit`.
### Concurrency
In principle, each of the four ABCI connections operate concurrently with one
another. This means applications need to ensure access to state is
thread safe. In practice, both the
[default in-process ABCI client](https://github.com/tendermint/tendermint/blob/v0.34.4/abci/client/local_client.go#L18)
and the
[default Go ABCI
server](https://github.com/tendermint/tendermint/blob/v0.34.4/abci/server/socket_server.go#L32)
use global locks across all connections, so they are not
concurrent at all. This means if your app is written in Go, and compiled in-process with Tendermint
using the default `NewLocalClient`, or run out-of-process using the default `SocketServer`,
ABCI messages from all connections will be linearizable (received one at a
time).
The existence of this global mutex means Go application developers can get
thread safety for application state by routing *all* reads and writes through the ABCI
system. Thus it may be *unsafe* to expose application state directly to an RPC
interface, and unless explicit measures are taken, all queries should be routed through the ABCI Query method.
### BeginBlock
The BeginBlock request can be used to run some code at the beginning of
every block. It also allows Tendermint to send the current block hash
and header to the application, before it sends any of the transactions.
The app should remember the latest height and header (ie. from which it
has run a successful Commit) so that it can tell Tendermint where to
pick up from when it restarts. See information on the Handshake, below.
### Commit
Application state should only be persisted to disk during `Commit`.
Before `Commit` is called, Tendermint locks and flushes the mempool so that no new messages will
be received on the mempool connection. This provides an opportunity to safely update all four connection
states to the latest committed state at once.
When `Commit` completes, it unlocks the mempool.
WARNING: if the ABCI app logic processing the `Commit` message sends a
`/broadcast_tx_sync` or `/broadcast_tx_commit` and waits for the response
before proceeding, it will deadlock. Executing those `broadcast_tx` calls
involves acquiring a lock that is held during the `Commit` call, so it's not
possible. If you make the call to the `broadcast_tx` endpoints concurrently,
that's no problem, it just can't be part of the sequential logic of the
`Commit` function.
### Consensus Connection
The Consensus Connection should maintain a `DeliverTxState` - the working state
for block execution. It should be updated by the calls to `BeginBlock`, `DeliverTx`,
and `EndBlock` during block execution and committed to disk as the "latest
committed state" during `Commit`.
Updates made to the `DeliverTxState` by each method call must be readable by each subsequent method -
ie. the updates are linearizable.
### Mempool Connection
The mempool Connection should maintain a `CheckTxState`
to sequentially process pending transactions in the mempool that have
not yet been committed. It should be initialized to the latest committed state
at the end of every `Commit`.
Before calling `Commit`, Tendermint will lock and flush the mempool connection,
ensuring that all existing CheckTx are responded to and no new ones can begin.
The `CheckTxState` may be updated concurrently with the `DeliverTxState`, as
messages may be sent concurrently on the Consensus and Mempool connections.
After `Commit`, while still holding the mempool lock, CheckTx is run again on all transactions that remain in the
node's local mempool after filtering those included in the block.
An additional `Type` parameter is made available to the CheckTx function that
indicates whether an incoming transaction is new (`CheckTxType_New`), or a
recheck (`CheckTxType_Recheck`).
Finally, after re-checking transactions in the mempool, Tendermint will unlock
the mempool connection. New transactions are once again able to be processed through CheckTx.
Note that CheckTx is just a weak filter to keep invalid transactions out of the block chain.
CheckTx doesn't have to check everything that affects transaction validity; the
expensive things can be skipped. It's weak because a Byzantine node doesn't
care about CheckTx; it can propose a block full of invalid transactions if it wants.
#### Replay Protection
To prevent old transactions from being replayed, CheckTx must implement
replay protection.
It is possible for old transactions to be sent to the application. So
it is important CheckTx implements some logic to handle them.
### Query Connection
The Info Connection should maintain a `QueryState` for answering queries from the user,
and for initialization when Tendermint first starts up (both described further
below).
It should always contain the latest committed state associated with the
latest committed block.
`QueryState` should be set to the latest `DeliverTxState` at the end of every `Commit`,
after the full block has been processed and the state committed to disk.
Otherwise it should never be modified.
Tendermint Core currently uses the Query connection to filter peers upon
connecting, according to IP address or node ID. For instance,
returning non-OK ABCI response to either of the following queries will
cause Tendermint to not connect to the corresponding peer:
- `p2p/filter/addr/<ip addr>`, where `<ip addr>` is an IP address.
- `p2p/filter/id/<id>`, where `<is>` is the hex-encoded node ID (the hash of
the node's p2p pubkey).
Note: these query formats are subject to change!
### Snapshot Connection
The Snapshot Connection is optional, and is only used to serve state sync snapshots for other nodes
and/or restore state sync snapshots to a local node being bootstrapped.
For more information, see [the state sync section of this document](#state-sync).
## Transaction Results
The `Info` and `Log` fields are non-deterministic values for debugging/convenience purposes
that are otherwise ignored.
The `Data` field must be strictly deterministic, but can be arbitrary data.
### Gas
Ethereum introduced the notion of `gas` as an abstract representation of the
cost of resources used by nodes when processing transactions. Every operation in the
Ethereum Virtual Machine uses some amount of gas, and gas can be accepted at a market-variable price.
Users propose a maximum amount of gas for their transaction; if the tx uses less, they get
the difference credited back. Tendermint adopts a similar abstraction,
though uses it only optionally and weakly, allowing applications to define
their own sense of the cost of execution.
In Tendermint, the [ConsensusParams.Block.MaxGas](../proto/types/params.proto) limits the amount of `gas` that can be used in a block.
The default value is `-1`, meaning no limit, or that the concept of gas is
meaningless.
Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum
amount of gas the sender of a tx is willing to use, and the later is how much it actually
used. Applications should enforce that `GasUsed <= GasWanted` - ie. tx execution
should halt before it can use more resources than it requested.
When `MaxGas > -1`, Tendermint enforces the following rules:
- `GasWanted <= MaxGas` for all txs in the mempool
- `(sum of GasWanted in a block) <= MaxGas` when proposing a block
If `MaxGas == -1`, no rules about gas are enforced.
Note that Tendermint does not currently enforce anything about Gas in the consensus, only the mempool.
This means it does not guarantee that committed blocks satisfy these rules!
It is the application's responsibility to return non-zero response codes when gas limits are exceeded.
The `GasUsed` field is ignored completely by Tendermint. That said, applications should enforce:
- `GasUsed <= GasWanted` for any given transaction
- `(sum of GasUsed in a block) <= MaxGas` for every block
In the future, we intend to add a `Priority` field to the responses that can be
used to explicitly prioritize txs in the mempool for inclusion in a block
proposal. See [#1861](https://github.com/tendermint/tendermint/issues/1861).
### CheckTx
If `Code != 0`, it will be rejected from the mempool and hence
not broadcasted to other peers and not included in a proposal block.
`Data` contains the result of the CheckTx transaction execution, if any. It is
semantically meaningless to Tendermint.
`Events` include any events for the execution, though since the transaction has not
been committed yet, they are effectively ignored by Tendermint.
### DeliverTx
DeliverTx is the workhorse of the blockchain. Tendermint sends the
DeliverTx requests asynchronously but in order, and relies on the
underlying socket protocol (ie. TCP) to ensure they are received by the
app in order. They have already been ordered in the global consensus by
the Tendermint protocol.
If DeliverTx returns `Code != 0`, the transaction will be considered invalid,
though it is still included in the block.
DeliverTx also returns a [Code, Data, and Log](../../proto/abci/types.proto#L189-L191).
`Data` contains the result of the CheckTx transaction execution, if any. It is
semantically meaningless to Tendermint.
Both the `Code` and `Data` are included in a structure that is hashed into the
`LastResultsHash` of the next block header.
`Events` include any events for the execution, which Tendermint will use to index
the transaction by. This allows transactions to be queried according to what
events took place during their execution.
## Updating the Validator Set
The application may set the validator set during InitChain, and may update it during
EndBlock.
Note that the maximum total power of the validator set is bounded by
`MaxTotalVotingPower = MaxInt64 / 8`. Applications are responsible for ensuring
they do not make changes to the validator set that cause it to exceed this
limit.
Additionally, applications must ensure that a single set of updates does not contain any duplicates -
a given public key can only appear once within a given update. If an update includes
duplicates, the block execution will fail irrecoverably.
### InitChain
The `InitChain` method can return a list of validators.
If the list is empty, Tendermint will use the validators loaded in the genesis
file.
If the list returned by `InitChain` is not empty, Tendermint will use its contents as the validator set.
This way the application can set the initial validator set for the
blockchain.
### EndBlock
Updates to the Tendermint validator set can be made by returning
`ValidatorUpdate` objects in the `ResponseEndBlock`:
```protobuf
message ValidatorUpdate {
tendermint.crypto.keys.PublicKey pub_key
int64 power
}
message PublicKey {
oneof {
ed25519 bytes = 1;
}
```
The `pub_key` currently supports only one type:
- `type = "ed25519"`
The `power` is the new voting power for the validator, with the
following rules:
- power must be non-negative
- if power is 0, the validator must already exist, and will be removed from the
validator set
- if power is non-0:
- if the validator does not already exist, it will be added to the validator
set with the given power
- if the validator does already exist, its power will be adjusted to the given power
- the total power of the new validator set must not exceed MaxTotalVotingPower
Note the updates returned in block `H` will only take effect at block `H+2`.
## Consensus Parameters
ConsensusParams enforce certain limits in the blockchain, like the maximum size
of blocks, amount of gas used in a block, and the maximum acceptable age of
evidence. They can be set in InitChain and updated in EndBlock.
### BlockParams.MaxBytes
The maximum size of a complete Protobuf encoded block.
This is enforced by Tendermint consensus.
This implies a maximum transaction size that is this MaxBytes, less the expected size of
the header, the validator set, and any included evidence in the block.
Must have `0 < MaxBytes < 100 MB`.
### BlockParams.MaxGas
The maximum of the sum of `GasWanted` that will be allowed in a proposed block.
This is *not* enforced by Tendermint consensus.
It is left to the app to enforce (ie. if txs are included past the
limit, they should return non-zero codes). It is used by Tendermint to limit the
txs included in a proposed block.
Must have `MaxGas >= -1`.
If `MaxGas == -1`, no limit is enforced.
### EvidenceParams.MaxAgeDuration
This is the maximum age of evidence in time units.
This is enforced by Tendermint consensus.
If a block includes evidence older than this (AND the evidence was created more
than `MaxAgeNumBlocks` ago), the block will be rejected (validators won't vote
for it).
Must have `MaxAgeDuration > 0`.
### EvidenceParams.MaxAgeNumBlocks
This is the maximum age of evidence in blocks.
This is enforced by Tendermint consensus.
If a block includes evidence older than this (AND the evidence was created more
than `MaxAgeDuration` ago), the block will be rejected (validators won't vote
for it).
Must have `MaxAgeNumBlocks > 0`.
### EvidenceParams.MaxNum
This is the maximum number of evidence that can be committed to a single block.
The product of this and the `MaxEvidenceBytes` must not exceed the size of
a block minus it's overhead ( ~ `MaxBytes`).
Must have `MaxNum > 0`.
### Updates
The application may set the ConsensusParams during InitChain, and update them during
EndBlock. If the ConsensusParams is empty, it will be ignored. Each field
that is not empty will be applied in full. For instance, if updating the
Block.MaxBytes, applications must also set the other Block fields (like
Block.MaxGas), even if they are unchanged, as they will otherwise cause the
value to be updated to 0.
#### InitChain
ResponseInitChain includes a ConsensusParams.
If ConsensusParams is nil, Tendermint will use the params loaded in the genesis
file. If ConsensusParams is not nil, Tendermint will use it.
This way the application can determine the initial consensus params for the
blockchain.
#### EndBlock
ResponseEndBlock includes a ConsensusParams.
If ConsensusParams nil, Tendermint will do nothing.
If ConsensusParam is not nil, Tendermint will use it.
This way the application can update the consensus params over time.
Note the updates returned in block `H` will take effect right away for block
`H+1`.
## Query
Query is a generic method with lots of flexibility to enable diverse sets
of queries on application state. Tendermint makes use of Query to filter new peers
based on ID and IP, and exposes Query to the user over RPC.
Note that calls to Query are not replicated across nodes, but rather query the
local node's state - hence they may return stale reads. For reads that require
consensus, use a transaction.
The most important use of Query is to return Merkle proofs of the application state at some height
that can be used for efficient application-specific light-clients.
Note Tendermint has technically no requirements from the Query
message for normal operation - that is, the ABCI app developer need not implement
Query functionality if they do not wish too.
### Query Proofs
The Tendermint block header includes a number of hashes, each providing an
anchor for some type of proof about the blockchain. The `ValidatorsHash` enables
quick verification of the validator set, the `DataHash` gives quick
verification of the transactions included in the block, etc.
The `AppHash` is unique in that it is application specific, and allows for
application-specific Merkle proofs about the state of the application.
While some applications keep all relevant state in the transactions themselves
(like Bitcoin and its UTXOs), others maintain a separated state that is
computed deterministically *from* transactions, but is not contained directly in
the transactions themselves (like Ethereum contracts and accounts).
For such applications, the `AppHash` provides a much more efficient way to verify light-client proofs.
ABCI applications can take advantage of more efficient light-client proofs for
their state as follows:
- return the Merkle root of the deterministic application state in
`ResponseCommit.Data`. This Merkle root will be included as the `AppHash` in the next block.
- return efficient Merkle proofs about that application state in `ResponseQuery.Proof`
that can be verified using the `AppHash` of the corresponding block.
For instance, this allows an application's light-client to verify proofs of
absence in the application state, something which is much less efficient to do using the block hash.
Some applications (eg. Ethereum, Cosmos-SDK) have multiple "levels" of Merkle trees,
where the leaves of one tree are the root hashes of others. To support this, and
the general variability in Merkle proofs, the `ResponseQuery.Proof` has some minimal structure:
```protobuf
message ProofOps {
repeated ProofOp ops
}
message ProofOp {
string type = 1;
bytes key = 2;
bytes data = 3;
}
```
Each `ProofOp` contains a proof for a single key in a single Merkle tree, of the specified `type`.
This allows ABCI to support many different kinds of Merkle trees, encoding
formats, and proofs (eg. of presence and absence) just by varying the `type`.
The `data` contains the actual encoded proof, encoded according to the `type`.
When verifying the full proof, the root hash for one ProofOp is the value being
verified for the next ProofOp in the list. The root hash of the final ProofOp in
the list should match the `AppHash` being verified against.
### Peer Filtering
When Tendermint connects to a peer, it sends two queries to the ABCI application
using the following paths, with no additional data:
- `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and
the port of the connection
- `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the
pubkey.Address() for the peer's PubKey)
If either of these queries return a non-zero ABCI code, Tendermint will refuse
to connect to the peer.
### Paths
Queries are directed at paths, and may optionally include additional data.
The expectation is for there to be some number of high level paths
differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently,
Tendermint only uses `/p2p`, for filtering peers. For more advanced use, see the
implementation of
[Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/v0.23.1/baseapp/baseapp.go#L333).
## Crash Recovery
On startup, Tendermint calls the `Info` method on the Info Connection to get the latest
committed state of the app. The app MUST return information consistent with the
last block it succesfully completed Commit for.
If the app succesfully committed block H, then `last_block_height = H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app
failed during the Commit of block H, then `last_block_height = H-1` and
`last_block_app_hash = <hash returned by Commit for block H-1, which is the hash in the header of block H>`.
We now distinguish three heights, and describe how Tendermint syncs itself with
the app.
```md
storeBlockHeight = height of the last block Tendermint saw a commit for
stateBlockHeight = height of the last block for which Tendermint completed all
block processing and saved all ABCI results to disk
appBlockHeight = height of the last block for which ABCI app succesfully
completed Commit
```
Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight`
Note also Tendermint never calls Commit on an ABCI app twice for the same height.
The procedure is as follows.
First, some simple start conditions:
If `appBlockHeight == 0`, then call InitChain.
If `storeBlockHeight == 0`, we're done.
Now, some sanity checks:
If `storeBlockHeight < appBlockHeight`, error
If `storeBlockHeight < stateBlockHeight`, panic
If `storeBlockHeight > stateBlockHeight+1`, panic
Now, the meat:
If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`,
replay all blocks in full from `appBlockHeight` to `storeBlockHeight`.
This happens if we completed processing the block, but the app forgot its height.
If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done.
This happens if we crashed at an opportune spot.
If `storeBlockHeight == stateBlockHeight+1`
This happens if we started processing the block but didn't finish.
If `appBlockHeight < stateBlockHeight`
replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`,
and replay the block at `storeBlockHeight` using the WAL.
This happens if the app forgot the last block it committed.
If `appBlockHeight == stateBlockHeight`,
replay the last block (storeBlockHeight) in full.
This happens if we crashed before the app finished Commit
If `appBlockHeight == storeBlockHeight`
update the state using the saved ABCI responses but dont run the block against the real app.
This happens if we crashed after the app finished Commit but before Tendermint saved the state.
## State Sync
A new node joining the network can simply join consensus at the genesis height and replay all
historical blocks until it is caught up. However, for large chains this can take a significant
amount of time, often on the order of days or weeks.
State sync is an alternative mechanism for bootstrapping a new node, where it fetches a snapshot
of the state machine at a given height and restores it. Depending on the application, this can
be several orders of magnitude faster than replaying blocks.
Note that state sync does not currently backfill historical blocks, so the node will have a
truncated block history - users are advised to consider the broader network implications of this in
terms of block availability and auditability. This functionality may be added in the future.
For details on the specific ABCI calls and types, see the [methods and types section](abci.md).
### Taking Snapshots
Applications that want to support state syncing must take state snapshots at regular intervals. How
this is accomplished is entirely up to the application. A snapshot consists of some metadata and
a set of binary chunks in an arbitrary format:
- `Height (uint64)`: The height at which the snapshot is taken. It must be taken after the given
height has been committed, and must not contain data from any later heights.
- `Format (uint32)`: An arbitrary snapshot format identifier. This can be used to version snapshot
formats, e.g. to switch from Protobuf to MessagePack for serialization. The application can use
this when restoring to choose whether to accept or reject a snapshot.
- `Chunks (uint32)`: The number of chunks in the snapshot. Each chunk contains arbitrary binary
data, and should be less than 16 MB; 10 MB is a good starting point.
- `Hash ([]byte)`: An arbitrary hash of the snapshot. This is used to check whether a snapshot is
the same across nodes when downloading chunks.
- `Metadata ([]byte)`: Arbitrary snapshot metadata, e.g. chunk hashes for verification or any other
necessary info.
For a snapshot to be considered the same across nodes, all of these fields must be identical. When
sent across the network, snapshot metadata messages are limited to 4 MB.
When a new node is running state sync and discovering snapshots, Tendermint will query an existing
application via the ABCI `ListSnapshots` method to discover available snapshots, and load binary
snapshot chunks via `LoadSnapshotChunk`. The application is free to choose how to implement this
and which formats to use, but must provide the following guarantees:
- **Consistent:** A snapshot must be taken at a single isolated height, unaffected by
concurrent writes. This can be accomplished by using a data store that supports ACID
transactions with snapshot isolation.
- **Asynchronous:** Taking a snapshot can be time-consuming, so it must not halt chain progress,
for example by running in a separate thread.
- **Deterministic:** A snapshot taken at the same height in the same format must be identical
(at the byte level) across nodes, including all metadata. This ensures good availability of
chunks, and that they fit together across nodes.
A very basic approach might be to use a datastore with MVCC transactions (such as RocksDB),
start a transaction immediately after block commit, and spawn a new thread which is passed the
transaction handle. This thread can then export all data items, serialize them using e.g.
Protobuf, hash the byte stream, split it into chunks, and store the chunks in the file system
along with some metadata - all while the blockchain is applying new blocks in parallel.
A more advanced approach might include incremental verification of individual chunks against the
chain app hash, parallel or batched exports, compression, and so on.
Old snapshots should be removed after some time - generally only the last two snapshots are needed
(to prevent the last one from being removed while a node is restoring it).
### Bootstrapping a Node
An empty node can be state synced by setting the configuration option `statesync.enabled =
true`. The node also needs the chain genesis file for basic chain info, and configuration for
light client verification of the restored snapshot: a set of Tendermint RPC servers, and a
trusted header hash and corresponding height from a trusted source, via the `statesync`
configuration section.
Once started, the node will connect to the P2P network and begin discovering snapshots. These
will be offered to the local application via the `OfferSnapshot` ABCI method. Once a snapshot
is accepted Tendermint will fetch and apply the snapshot chunks. After all chunks have been
successfully applied, Tendermint verifies the app's `AppHash` against the chain using the light
client, then switches the node to normal consensus operation.
#### Snapshot Discovery
When the empty node join the P2P network, it asks all peers to report snapshots via the
`ListSnapshots` ABCI call (limited to 10 per node). After some time, the node picks the most
suitable snapshot (generally prioritized by height, format, and number of peers), and offers it
to the application via `OfferSnapshot`. The application can choose a number of responses,
including accepting or rejecting it, rejecting the offered format, rejecting the peer who sent
it, and so on. Tendermint will keep discovering and offering snapshots until one is accepted or
the application aborts.
#### Snapshot Restoration
Once a snapshot has been accepted via `OfferSnapshot`, Tendermint begins downloading chunks from
any peers that have the same snapshot (i.e. that have identical metadata fields). Chunks are
spooled in a temporary directory, and then given to the application in sequential order via
`ApplySnapshotChunk` until all chunks have been accepted.
The method for restoring snapshot chunks is entirely up to the application.
During restoration, the application can respond to `ApplySnapshotChunk` with instructions for how
to continue. This will typically be to accept the chunk and await the next one, but it can also
ask for chunks to be refetched (either the current one or any number of previous ones), P2P peers
to be banned, snapshots to be rejected or retried, and a number of other responses - see the ABCI
reference for details.
If Tendermint fails to fetch a chunk after some time, it will reject the snapshot and try a
different one via `OfferSnapshot` - the application can choose whether it wants to support
restarting restoration, or simply abort with an error.
#### Snapshot Verification
Once all chunks have been accepted, Tendermint issues an `Info` ABCI call to retrieve the
`LastBlockAppHash`. This is compared with the trusted app hash from the chain, retrieved and
verified using the light client. Tendermint also checks that `LastBlockHeight` corresponds to the
height of the snapshot.
This verification ensures that an application is valid before joining the network. However, the
snapshot restoration may take a long time to complete, so applications may want to employ additional
verification during the restore to detect failures early. This might e.g. include incremental
verification of each chunk against the app hash (using bundled Merkle proofs), checksums to
protect against data corruption by the disk or network, and so on. However, it is important to
note that the only trusted information available is the app hash, and all other snapshot metadata
can be spoofed by adversaries.
Apps may also want to consider state sync denial-of-service vectors, where adversaries provide
invalid or harmful snapshots to prevent nodes from joining the network. The application can
counteract this by asking Tendermint to ban peers. As a last resort, node operators can use
P2P configuration options to whitelist a set of trusted peers that can provide valid snapshots.
#### Transition to Consensus
Once the snapshots have all been restored, Tendermint gathers additional information necessary for
bootstrapping the node (e.g. chain ID, consensus parameters, validator sets, and block headers)
from the genesis file and light client RPC servers. It also fetches and records the `AppVersion`
from the ABCI application.
Once the state machine has been restored and Tendermint has gathered this additional
information, it transitions to block sync (if enabled) to fetch any remaining blocks up the chain
head, and then transitions to regular consensus operation. At this point the node operates like
any other node, apart from having a truncated block history at the height of the restored snapshot.

113
spec/abci/client-server.md Normal file
View File

@@ -0,0 +1,113 @@
---
order: 3
title: Client and Server
---
# Client and Server
This section is for those looking to implement their own ABCI Server, perhaps in
a new programming language.
You are expected to have read [ABCI Methods and Types](./abci.md) and [ABCI
Applications](./apps.md).
## Message Protocol
The message protocol consists of pairs of requests and responses defined in the
[protobuf file](https://github.com/tendermint/tendermint/blob/master/proto/tendermint/abci/types.proto).
Some messages have no fields, while others may include byte-arrays, strings, integers,
or custom protobuf types.
For more details on protobuf, see the [documentation](https://developers.google.com/protocol-buffers/docs/overview).
For each request, a server should respond with the corresponding
response, where the order of requests is preserved in the order of
responses.
## Server Implementations
To use ABCI in your programming language of choice, there must be a ABCI
server in that language. Tendermint supports three implementations of the ABCI, written in Go:
- In-process ([Golang](https://github.com/tendermint/tendermint/tree/master/abci), [Rust](https://github.com/tendermint/rust-abci))
- ABCI-socket
- GRPC
The latter two can be tested using the `abci-cli` by setting the `--abci` flag
appropriately (ie. to `socket` or `grpc`).
See examples, in various stages of maintenance, in
[Go](https://github.com/tendermint/tendermint/tree/master/abci/server),
[JavaScript](https://github.com/tendermint/js-abci),
[C++](https://github.com/mdyring/cpp-tmsp), and
[Java](https://github.com/jTendermint/jabci).
### In Process
The simplest implementation uses function calls within Golang.
This means ABCI applications written in Golang can be compiled with Tendermint Core and run as a single binary.
### GRPC
If GRPC is available in your language, this is the easiest approach,
though it will have significant performance overhead.
To get started with GRPC, copy in the [protobuf
file](https://github.com/tendermint/tendermint/blob/master/proto/tendermint/abci/types.proto)
and compile it using the GRPC plugin for your language. For instance,
for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`.
See the [grpc documentation for more details](http://www.grpc.io/docs/).
`protoc` will autogenerate all the necessary code for ABCI client and
server in your language, including whatever interface your application
must satisfy to be used by the ABCI server for handling requests.
Note the length-prefixing used in the socket implementation (TSP) does not apply for GRPC.
### TSP
Tendermint Socket Protocol is an asynchronous, raw socket server which provides ordered message passing over unix or tcp.
Messages are serialized using Protobuf3 and length-prefixed with a [signed Varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-integers)
If GRPC is not available in your language, or you require higher
performance, or otherwise enjoy programming, you may implement your own
ABCI server using the Tendermint Socket Protocol. The first step is still to auto-generate the relevant data
types and codec in your language using `protoc`. In addition to being proto3 encoded, messages coming over
the socket are length-prefixed to facilitate use as a streaming protocol. proto3 doesn't have an
official length-prefix standard, so we use our own. The first byte in
the prefix represents the length of the Big Endian encoded length. The
remaining bytes in the prefix are the Big Endian encoded length.
For example, if the proto3 encoded ABCI message is 0xDEADBEEF (4
bytes), the length-prefixed message is 0x0104DEADBEEF. If the proto3
encoded ABCI message is 65535 bytes long, the length-prefixed message
would be like 0x02FFFF....
The benefit of using this `varint` encoding over the old version (where integers were encoded as `<len of len><big endian len>` is that
it is the standard way to encode integers in Protobuf. It is also generally shorter.
As noted above, this prefixing does not apply for GRPC.
An ABCI server must also be able to support multiple connections, as
Tendermint uses four connections.
### Async vs Sync
The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages.
This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward
transactions to the app before it's finished processing previous ones.
Thus, DeliverTx and CheckTx messages are sent asynchronously, while all other
messages are sent synchronously.
## Client
There are currently two use-cases for an ABCI client. One is a testing
tool, as in the `abci-cli`, which allows ABCI requests to be sent via
command line. The other is a consensus engine, such as Tendermint Core,
which makes requests to the application every time a new transaction is
received or a block is committed.
It is unlikely that you will need to implement a client. For details of
our client, see
[here](https://github.com/tendermint/tendermint/tree/master/abci/client).

3
spec/blockchain/blockchain.md Normal file
View File

@@ -0,0 +1,3 @@
# Blockchain
Deprecated see [core/data_structures.md](../core/data_structures.md)

3
spec/blockchain/encoding.md Normal file
View File

@@ -0,0 +1,3 @@
# Encoding
Deprecated see [core/data_structures.md](../core/encoding.md)

14
spec/blockchain/readme.md Normal file
View File

@@ -0,0 +1,14 @@
---
order: 1
parent:
title: Blockchain
order: false
---
# Blockchain
This section describes the core types and functionality of the Tendermint protocol implementation.
[Core Data Structures](../core/data_structures.md)
[Encoding](../core/encoding.md)
[State](../core/state.md)

3
spec/blockchain/state.md Normal file
View File

@@ -0,0 +1,3 @@
# State
Deprecated see [core/state.md](../core/state.md)

55
spec/consensus/bft-time.md Normal file
View File

@@ -0,0 +1,55 @@
---
order: 2
---
# BFT Time
Tendermint provides a deterministic, Byzantine fault-tolerant, source of time.
Time in Tendermint is defined with the Time field of the block header.
It satisfies the following properties:
- Time Monotonicity: Time is monotonically increasing, i.e., given
a header H1 for height h1 and a header H2 for height `h2 = h1 + 1`, `H1.Time < H2.Time`.
- Time Validity: Given a set of Commit votes that forms the `block.LastCommit` field, a range of
valid values for the Time field of the block header is defined only by
Precommit messages (from the LastCommit field) sent by correct processes, i.e.,
a faulty process cannot arbitrarily increase the Time value.
In the context of Tendermint, time is of type int64 and denotes UNIX time in milliseconds, i.e.,
corresponds to the number of milliseconds since January 1, 1970. Before defining rules that need to be enforced by the
Tendermint consensus protocol, so the properties above holds, we introduce the following definition:
- median of a Commit is equal to the median of `Vote.Time` fields of the `Vote` messages,
where the value of `Vote.Time` is counted number of times proportional to the process voting power. As in Tendermint
the voting power is not uniform (one process one vote), a vote message is actually an aggregator of the same votes whose
number is equal to the voting power of the process that has casted the corresponding votes message.
Let's consider the following example:
- we have four processes p1, p2, p3 and p4, with the following voting power distribution (p1, 23), (p2, 27), (p3, 10)
and (p4, 10). The total voting power is 70 (`N = 3f+1`, where `N` is the total voting power, and `f` is the maximum voting
power of the faulty processes), so we assume that the faulty processes have at most 23 of voting power.
Furthermore, we have the following vote messages in some LastCommit field (we ignore all fields except Time field):
- (p1, 100), (p2, 98), (p3, 1000), (p4, 500). We assume that p3 and p4 are faulty processes. Let's assume that the
`block.LastCommit` message contains votes of processes p2, p3 and p4. Median is then chosen the following way:
the value 98 is counted 27 times, the value 1000 is counted 10 times and the value 500 is counted also 10 times.
So the median value will be the value 98. No matter what set of messages with at least `2f+1` voting power we
choose, the median value will always be between the values sent by correct processes.
We ensure Time Monotonicity and Time Validity properties by the following rules:
- let rs denotes `RoundState` (consensus internal state) of some process. Then
`rs.ProposalBlock.Header.Time == median(rs.LastCommit) &&
rs.Proposal.Timestamp == rs.ProposalBlock.Header.Time`.
- Furthermore, when creating the `vote` message, the following rules for determining `vote.Time` field should hold:
- if `rs.LockedBlock` is defined then
`vote.Time = max(rs.LockedBlock.Timestamp + time.Millisecond, time.Now())`, where `time.Now()`
denotes local Unix time in milliseconds
- else if `rs.Proposal` is defined then
`vote.Time = max(rs.Proposal.Timestamp + time.Millisecond,, time.Now())`,
- otherwise, `vote.Time = time.Now())`. In this case vote is for `nil` so it is not taken into account for
the timestamp of the next block.

2417
spec/consensus/consensus-paper/IEEEtran.bst Normal file
View File

File diff suppressed because it is too large Load Diff

4733
spec/consensus/consensus-paper/IEEEtran.cls Normal file
View File

File diff suppressed because it is too large Load Diff

24
spec/consensus/consensus-paper/README.md Normal file
View File

@@ -0,0 +1,24 @@
# Tendermint-spec
The repository contains the specification (and the proofs) of the Tendermint
consensus protocol.
## How to install Latex on Mac OS
MacTex is Latex distribution for Mac OS. You can download it [here](http://www.tug.org/mactex/mactex-download.html).
Popular IDE for Latex-based projects is TexStudio. It can be downloaded
[here](https://www.texstudio.org/).
## How to build project
In order to compile the latex files (and write bibliography), execute
`$ pdflatex paper` <br/>
`$ bibtex paper` <br/>
`$ pdflatex paper` <br/>
`$ pdflatex paper` <br/>
The generated file is paper.pdf. You can open it with
`$ open paper.pdf`

195
spec/consensus/consensus-paper/algorithmicplus.sty Normal file
View File

@@ -0,0 +1,195 @@
% ALGORITHMICPLUS STYLE
% for LaTeX version 2e
% Original ``algorithmic.sty'' by -- 1994 Peter Williams <pwil3058@bigpond.net.au>
% Bug fix (13 July 2004) by Arnaud Giersch <giersch@icps.u-strasbg.fr>
% Includes ideas from 'algorithmicext' by Martin Biely
% <biely@ecs.tuwien.ac.at> and 'distribalgo' by Xavier Defago
% Modifications: Martin Hutle <martin.hutle@epfl.ch>
%
% This style file is free software; you can redistribute it and/or
% modify it under the terms of the GNU Lesser General Public
% License as published by the Free Software Foundation; either
% version 2 of the License, or (at your option) any later version.
%
% This style file is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
% Lesser General Public License for more details.
%
% You should have received a copy of the GNU Lesser General Public
% License along with this style file; if not, write to the
% Free Software Foundation, Inc., 59 Temple Place - Suite 330,
% Boston, MA 02111-1307, USA.
%
\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{algorithmicplus}
\typeout{Document Style `algorithmicplus' - environment, replaces `algorithmic'}
%
\RequirePackage{ifthen}
\RequirePackage{calc}
\newboolean{ALC@noend}
\setboolean{ALC@noend}{false}
\newcounter{ALC@line}
\newcounter{ALC@rem}
\newcounter{ALC@depth}
\newcounter{ALCPLUS@lastline}
\newlength{\ALC@tlm}
%
\DeclareOption{noend}{\setboolean{ALC@noend}{true}}
%
\ProcessOptions
%
% ALGORITHMIC
\newcommand{\algorithmiclnosize}{\small}
\newcommand{\algorithmiclnofont}{\tt}
\newcommand{\algorithmiclnodelimiter}{:}
%
\newcommand{\algorithmicrequire}{\textbf{Require:}}
\newcommand{\algorithmicensure}{\textbf{Ensure:}}
\newcommand{\algorithmiccomment}[1]{\{#1\}}
\newcommand{\algorithmicend}{\textbf{end}}
\newcommand{\algorithmicif}{\textbf{if}}
\newcommand{\algorithmicthen}{\textbf{then}}
\newcommand{\algorithmicelse}{\textbf{else}}
\newcommand{\algorithmicelsif}{\algorithmicelse\ \algorithmicif}
\newcommand{\algorithmicendif}{\algorithmicend\ \algorithmicif}
\newcommand{\algorithmicfor}{\textbf{for}}
\newcommand{\algorithmicforall}{\textbf{for all}}
\newcommand{\algorithmicdo}{\textbf{do}}
\newcommand{\algorithmicendfor}{\algorithmicend\ \algorithmicfor}
\newcommand{\algorithmicwhile}{\textbf{while}}
\newcommand{\algorithmicendwhile}{\algorithmicend\ \algorithmicwhile}
\newcommand{\algorithmicloop}{\textbf{loop}}
\newcommand{\algorithmicendloop}{\algorithmicend\ \algorithmicloop}
\newcommand{\algorithmicrepeat}{\textbf{repeat}}
\newcommand{\algorithmicuntil}{\textbf{until}}
\def\ALC@item[#1]{%
\if@noparitem \@donoparitem
\else \if@inlabel \indent \par \fi
\ifhmode \unskip\unskip \par \fi
\if@newlist \if@nobreak \@nbitem \else
\addpenalty\@beginparpenalty
\addvspace\@topsep \addvspace{-\parskip}\fi
\else \addpenalty\@itempenalty \addvspace\itemsep
\fi
\global\@inlabeltrue
\fi
\everypar{\global\@minipagefalse\global\@newlistfalse
\if@inlabel\global\@inlabelfalse \hskip -\parindent \box\@labels
\penalty\z@ \fi
\everypar{}}\global\@nobreakfalse
\if@noitemarg \@noitemargfalse \if@nmbrlist \refstepcounter{\@listctr}\fi \fi
\sbox\@tempboxa{\makelabel{#1}}%
\global\setbox\@labels
\hbox{\unhbox\@labels \hskip \itemindent
\hskip -\labelwidth \hskip -\ALC@tlm
\ifdim \wd\@tempboxa >\labelwidth
\box\@tempboxa
\else \hbox to\labelwidth {\unhbox\@tempboxa}\fi
\hskip \ALC@tlm}\ignorespaces}
%
\newenvironment{algorithmic}[1][0]{
\setcounter{ALC@depth}{\@listdepth}%
\let\@listdepth\c@ALC@depth%
\let\@item\ALC@item
\newcommand{\ALC@lno}{%
\ifthenelse{\equal{\arabic{ALC@rem}}{0}}
{{\algorithmiclnosize\algorithmiclnofont \arabic{ALC@line}\algorithmiclnodelimiter}}{}%
}
\let\@listii\@listi
\let\@listiii\@listi
\let\@listiv\@listi
\let\@listv\@listi
\let\@listvi\@listi
\let\@listvii\@listi
\newenvironment{ALC@g}{
\begin{list}{\ALC@lno}{ \itemsep\z@ \itemindent\z@
\listparindent\z@ \rightmargin\z@
\topsep\z@ \partopsep\z@ \parskip\z@\parsep\z@
\leftmargin 1em
\addtolength{\ALC@tlm}{\leftmargin}
}
}
{\end{list}}
\newcommand{\ALC@it}{\refstepcounter{ALC@line}\addtocounter{ALC@rem}{1}\ifthenelse{\equal{\arabic{ALC@rem}}{#1}}{\setcounter{ALC@rem}{0}}{}\item}
\newcommand{\ALC@com}[1]{\ifthenelse{\equal{##1}{default}}%
{}{\ \algorithmiccomment{##1}}}
\newcommand{\REQUIRE}{\item[\algorithmicrequire]}
\newcommand{\ENSURE}{\item[\algorithmicensure]}
\newcommand{\STATE}{\ALC@it}
\newcommand{\COMMENT}[1]{\algorithmiccomment{##1}}
\newenvironment{ALC@if}{\begin{ALC@g}}{\end{ALC@g}}
\newenvironment{ALC@for}{\begin{ALC@g}}{\end{ALC@g}}
\newenvironment{ALC@whl}{\begin{ALC@g}}{\end{ALC@g}}
\newenvironment{ALC@loop}{\begin{ALC@g}}{\end{ALC@g}}
\newenvironment{ALC@rpt}{\begin{ALC@g}}{\end{ALC@g}}
\renewcommand{\\}{\@centercr}
\newcommand{\IF}[2][default]{\ALC@it\algorithmicif\ ##2\ \algorithmicthen%
\ALC@com{##1}\begin{ALC@if}}
\newcommand{\ELSE}[1][default]{\end{ALC@if}\ALC@it\algorithmicelse%
\ALC@com{##1}\begin{ALC@if}}
\newcommand{\ELSIF}[2][default]%
{\end{ALC@if}\ALC@it\algorithmicelsif\ ##2\ \algorithmicthen%
\ALC@com{##1}\begin{ALC@if}}
\newcommand{\FOR}[2][default]{\ALC@it\algorithmicfor\ ##2\ \algorithmicdo%
\ALC@com{##1}\begin{ALC@for}}
\newcommand{\FORALL}[2][default]{\ALC@it\algorithmicforall\ ##2\ %
\algorithmicdo%
\ALC@com{##1}\begin{ALC@for}}
\newcommand{\WHILE}[2][default]{\ALC@it\algorithmicwhile\ ##2\ %
\algorithmicdo%
\ALC@com{##1}\begin{ALC@whl}}
\newcommand{\LOOP}[1][default]{\ALC@it\algorithmicloop%
\ALC@com{##1}\begin{ALC@loop}}
\newcommand{\REPEAT}[1][default]{\ALC@it\algorithmicrepeat%
\ALC@com{##1}\begin{ALC@rpt}}
\newcommand{\UNTIL}[1]{\end{ALC@rpt}\ALC@it\algorithmicuntil\ ##1}
\ifthenelse{\boolean{ALC@noend}}{
\newcommand{\ENDIF}{\end{ALC@if}}
\newcommand{\ENDFOR}{\end{ALC@for}}
\newcommand{\ENDWHILE}{\end{ALC@whl}}
\newcommand{\ENDLOOP}{\end{ALC@loop}}
}{
\newcommand{\ENDIF}{\end{ALC@if}\ALC@it\algorithmicendif}
\newcommand{\ENDFOR}{\end{ALC@for}\ALC@it\algorithmicendfor}
\newcommand{\ENDWHILE}{\end{ALC@whl}\ALC@it\algorithmicendwhile}
\newcommand{\ENDLOOP}{\end{ALC@loop}\ALC@it\algorithmicendloop}
}
\renewcommand{\@toodeep}{}
\begin{list}{\ALC@lno}{\setcounter{ALC@line}{0}\setcounter{ALC@rem}{0}%
\itemsep\z@ \itemindent\z@ \listparindent\z@%
\partopsep\z@ \parskip\z@ \parsep\z@%
\labelsep 0.5em \topsep 0.2em%
\ifthenelse{\equal{#1}{0}}
{\labelwidth 0.5em }
{\labelwidth 1.2em }
\leftmargin\labelwidth \addtolength{\leftmargin}{\labelsep}
\ALC@tlm\labelsep
}
}
{%
\setcounter{ALCPLUS@lastline}{\value{ALC@line}}%
\end{list}}
\newcommand{\continuecounting}{\setcounter{ALC@line}{\value{ALCPLUS@lastline}}}
\newcommand{\startcounting}[1]{\setcounter{ALC@line}{#1}\addtocounter{ALC@line}{-1}}
\newcommand{\EMPTY}{\item[]}
\newcommand{\SPACE}{\vspace{3mm}}
\newcommand{\SHORTSPACE}{\vspace{1mm}}
\newcommand{\newlinetag}[3]{\newcommand{#1}[#2]{\item[#3]}}
\newcommand{\newconstruct}[5]{%
\newenvironment{ALC@\string#1}{\begin{ALC@g}}{\end{ALC@g}}
\newcommand{#1}[2][default]{\ALC@it#2\ ##2\ #3%
\ALC@com{##1}\begin{ALC@\string#1}}
\ifthenelse{\boolean{ALC@noend}}{
\newcommand{#4}{\end{ALC@\string#1}}
}{
\newcommand{#4}{\end{ALC@\string#1}\ALC@it#5}
}
}
\newconstruct{\INDENT}{}{}{\ENDINDENT}{}
\newcommand{\setlinenosize}[1]{\renewcommand{\algorithmiclnosize}{#1}}
\newcommand{\setlinenofont}[1]{\renewcommand{\algorithmiclnofont}{#1}}

16
spec/consensus/consensus-paper/conclusion.tex Normal file
View File

@@ -0,0 +1,16 @@
\section{Conclusion} \label{sec:conclusion}
We have proposed a new Byzantine-fault tolerant consensus algorithm that is the
core of the Tendermint BFT SMR platform. The algorithm is designed for the wide
area network with high number of mutually distrusted nodes that communicate
over gossip based peer-to-peer network. It has only a single mode of execution
and the communication pattern is very similar to the "normal" case of the
state-of-the art PBFT algorithm. The algorithm ensures termination with a novel
mechanism that takes advantage of the gossip based communication between nodes.
The proposed algorithm and the proofs are simple and elegant, and we believe
that this makes it easier to understand and implement correctly.
\section*{Acknowledgment}
We would like to thank Anton Kaliaev, Ismail Khoffi and Dahlia Malkhi for comments on an earlier version of the paper. We also want to thank Marko Vukolic, Ming Chuan Lin, Maria Potop-Butucaru, Sara Tucci, Antonella Del Pozzo and Yackolley Amoussou-Guenou for pointing out the liveness issues
in the previous version of the algorithm. Finally, we want to thank the Tendermint team members and all project contributors for making Tendermint such a great platform.

397
spec/consensus/consensus-paper/consensus.tex Normal file
View File

@@ -0,0 +1,397 @@
\section{Tendermint consensus algorithm} \label{sec:tendermint}
\newcommand\Disseminate{\textbf{Disseminate}}
\newcommand\Proposal{\mathsf{PROPOSAL}}
\newcommand\ProposalPart{\mathsf{PROPOSAL\mbox{-}PART}}
\newcommand\PrePrepare{\mathsf{INIT}} \newcommand\Prevote{\mathsf{PREVOTE}}
\newcommand\Precommit{\mathsf{PRECOMMIT}}
\newcommand\Decision{\mathsf{DECISION}}
\newcommand\ViewChange{\mathsf{VC}}
\newcommand\ViewChangeAck{\mathsf{VC\mbox{-}ACK}}
\newcommand\NewPrePrepare{\mathsf{VC\mbox{-}INIT}}
\newcommand\coord{\mathsf{proposer}}
\newcommand\newHeight{newHeight} \newcommand\newRound{newRound}
\newcommand\nil{nil} \newcommand\id{id} \newcommand{\propose}{propose}
\newcommand\prevote{prevote} \newcommand\prevoteWait{prevoteWait}
\newcommand\precommit{precommit} \newcommand\precommitWait{precommitWait}
\newcommand\commit{commit}
\newcommand\timeoutPropose{timeoutPropose}
\newcommand\timeoutPrevote{timeoutPrevote}
\newcommand\timeoutPrecommit{timeoutPrecommit}
\newcommand\proofOfLocking{proof\mbox{-}of\mbox{-}locking}
\begin{algorithm}[htb!] \def\baselinestretch{1} \scriptsize\raggedright
\begin{algorithmic}[1]
\SHORTSPACE
\INIT{}
\STATE $h_p := 0$
\COMMENT{current height, or consensus instance we are currently executing}
\STATE $round_p := 0$ \COMMENT{current round number}
\STATE $step_p \in \set{\propose, \prevote, \precommit}$
\STATE $decision_p[] := nil$
\STATE $lockedValue_p := nil$
\STATE $lockedRound_p := -1$
\STATE $validValue_p := nil$
\STATE $validRound_p := -1$
\ENDINIT
\SHORTSPACE
\STATE \textbf{upon} start \textbf{do} $StartRound(0)$
\SHORTSPACE
\FUNCTION{$StartRound(round)$} \label{line:tab:startRound}
\STATE $round_p \assign round$
\STATE $step_p \assign \propose$
\IF{$\coord(h_p, round_p) = p$}
\IF{$validValue_p \neq \nil$} \label{line:tab:isThereLockedValue}
\STATE $proposal \assign validValue_p$ \ELSE \STATE $proposal \assign
getValue()$
\label{line:tab:getValidValue}
\ENDIF
\STATE \Broadcast\ $\li{\Proposal,h_p, round_p, proposal, validRound_p}$
\label{line:tab:send-proposal}
\ELSE
\STATE \textbf{schedule} $OnTimeoutPropose(h_p,
round_p)$ to be executed \textbf{after} $\timeoutPropose(round_p)$
\ENDIF
\ENDFUNCTION
\SPACE
\UPON{$\li{\Proposal,h_p,round_p, v, -1}$ \From\ $\coord(h_p,round_p)$
\With\ $step_p = \propose$} \label{line:tab:recvProposal}
\IF{$valid(v) \wedge (lockedRound_p = -1 \vee lockedValue_p = v$)}
\label{line:tab:accept-proposal-2}
\STATE \Broadcast \ $\li{\Prevote,h_p,round_p,id(v)}$
\label{line:tab:prevote-proposal}
\ELSE
\label{line:tab:acceptProposal1}
\STATE \Broadcast \ $\li{\Prevote,h_p,round_p,\nil}$
\label{line:tab:prevote-nil}
\ENDIF
\STATE $step_p \assign \prevote$ \label{line:tab:setStateToPrevote1}
\ENDUPON
\SPACE
\UPON{$\li{\Proposal,h_p,round_p, v, vr}$ \From\ $\coord(h_p,round_p)$
\textbf{AND} $2f+1$ $\li{\Prevote,h_p, vr,id(v)}$ \With\ $step_p = \propose \wedge (vr \ge 0 \wedge vr < round_p)$}
\label{line:tab:acceptProposal}
\IF{$valid(v) \wedge (lockedRound_p \le vr
\vee lockedValue_p = v)$} \label{line:tab:cond-prevote-higher-proposal}
\STATE \Broadcast \ $\li{\Prevote,h_p,round_p,id(v)}$
\label{line:tab:prevote-higher-proposal}
\ELSE
\label{line:tab:acceptProposal2}
\STATE \Broadcast \ $\li{\Prevote,h_p,round_p,\nil}$
\label{line:tab:prevote-nil2}
\ENDIF
\STATE $step_p \assign \prevote$ \label{line:tab:setStateToPrevote3}
\ENDUPON
\SPACE
\UPON{$2f+1$ $\li{\Prevote,h_p, round_p,*}$ \With\ $step_p = \prevote$ for the first time}
\label{line:tab:recvAny2/3Prevote}
\STATE \textbf{schedule} $OnTimeoutPrevote(h_p, round_p)$ to be executed \textbf{after} $\timeoutPrevote(round_p)$ \label{line:tab:timeoutPrevote}
\ENDUPON
\SPACE
\UPON{$\li{\Proposal,h_p,round_p, v, *}$ \From\ $\coord(h_p,round_p)$
\textbf{AND} $2f+1$ $\li{\Prevote,h_p, round_p,id(v)}$ \With\ $valid(v) \wedge step_p \ge \prevote$ for the first time}
\label{line:tab:recvPrevote}
\IF{$step_p = \prevote$}
\STATE $lockedValue_p \assign v$ \label{line:tab:setLockedValue}
\STATE $lockedRound_p \assign round_p$ \label{line:tab:setLockedRound}
\STATE \Broadcast \ $\li{\Precommit,h_p,round_p,id(v))}$
\label{line:tab:precommit-v}
\STATE $step_p \assign \precommit$ \label{line:tab:setStateToCommit}
\ENDIF
\STATE $validValue_p \assign v$ \label{line:tab:setValidRound}
\STATE $validRound_p \assign round_p$ \label{line:tab:setValidValue}
\ENDUPON
\SHORTSPACE
\UPON{$2f+1$ $\li{\Prevote,h_p,round_p, \nil}$
\With\ $step_p = \prevote$}
\STATE \Broadcast \ $\li{\Precommit,h_p,round_p, \nil}$
\label{line:tab:precommit-v-1}
\STATE $step_p \assign \precommit$
\ENDUPON
\SPACE
\UPON{$2f+1$ $\li{\Precommit,h_p,round_p,*}$ for the first time}
\label{line:tab:startTimeoutPrecommit}
\STATE \textbf{schedule} $OnTimeoutPrecommit(h_p, round_p)$ to be executed \textbf{after} $\timeoutPrecommit(round_p)$
\ENDUPON
\SPACE
\UPON{$\li{\Proposal,h_p,r, v, *}$ \From\ $\coord(h_p,r)$ \textbf{AND}
$2f+1$ $\li{\Precommit,h_p,r,id(v)}$ \With\ $decision_p[h_p] = \nil$}
\label{line:tab:onDecideRule}
\IF{$valid(v)$} \label{line:tab:validDecisionValue}
\STATE $decision_p[h_p] = v$ \label{line:tab:decide}
\STATE$h_p \assign h_p + 1$ \label{line:tab:increaseHeight}
\STATE reset $lockedRound_p$, $lockedValue_p$, $validRound_p$ and $validValue_p$ to initial values
and empty message log
\STATE $StartRound(0)$
\ENDIF
\ENDUPON
\SHORTSPACE
\UPON{$f+1$ $\li{*,h_p,round, *, *}$ \textbf{with} $round > round_p$}
\label{line:tab:skipRounds}
\STATE $StartRound(round)$ \label{line:tab:nextRound2}
\ENDUPON
\SHORTSPACE
\FUNCTION{$OnTimeoutPropose(height,round)$} \label{line:tab:onTimeoutPropose}
\IF{$height = h_p \wedge round = round_p \wedge step_p = \propose$}
\STATE \Broadcast \ $\li{\Prevote,h_p,round_p, \nil}$
\label{line:tab:prevote-nil-on-timeout}
\STATE $step_p \assign \prevote$
\ENDIF
\ENDFUNCTION
\SHORTSPACE
\FUNCTION{$OnTimeoutPrevote(height,round)$} \label{line:tab:onTimeoutPrevote}
\IF{$height = h_p \wedge round = round_p \wedge step_p = \prevote$}
\STATE \Broadcast \ $\li{\Precommit,h_p,round_p,\nil}$
\label{line:tab:precommit-nil-onTimeout}
\STATE $step_p \assign \precommit$
\ENDIF
\ENDFUNCTION
\SHORTSPACE
\FUNCTION{$OnTimeoutPrecommit(height,round)$} \label{line:tab:onTimeoutPrecommit}
\IF{$height = h_p \wedge round = round_p$}
\STATE $StartRound(round_p + 1)$ \label{line:tab:nextRound}
\ENDIF
\ENDFUNCTION
\end{algorithmic} \caption{Tendermint consensus algorithm}
\label{alg:tendermint}
\end{algorithm}
In this section we present the Tendermint Byzantine fault-tolerant consensus
algorithm. The algorithm is specified by the pseudo-code shown in
Algorithm~\ref{alg:tendermint}. We present the algorithm as a set of \emph{upon
rules} that are executed atomically\footnote{In case several rules are active
at the same time, the first rule to be executed is picked randomly. The
correctness of the algorithm does not depend on the order in which rules are
executed.}. We assume that processes exchange protocol messages using a gossip
protocol and that both sent and received messages are stored in a local message
log for every process. An upon rule is triggered once the message log contains
messages such that the corresponding condition evaluates to $\tt{true}$. The
condition that assumes reception of $X$ messages of a particular type and
content denotes reception of messages whose senders have aggregate voting power at
least equal to $X$. For example, the condition $2f+1$ $\li{\Precommit,h_p,r,id(v)}$,
evaluates to true upon reception of $\Precommit$ messages for height $h_p$,
a round $r$ and with value equal to $id(v)$ whose senders have aggregate voting
power at least equal to $2f+1$. Some of the rules ends with "for the first time" constraint
to denote that it is triggered only the first time a corresponding condition evaluates
to $\tt{true}$. This is because those rules do not always change the state of algorithm
variables so without this constraint, the algorithm could keep
executing those rules forever. The variables with index $p$ are process local state
variables, while variables without index $p$ are value placeholders. The sign
$*$ denotes any value.
We denote with $n$ the total voting power of processes in the system, and we
assume that the total voting power of faulty processes in the system is bounded
with a system parameter $f$. The algorithm assumes that $n > 3f$, i.e., it
requires that the total voting power of faulty processes is smaller than one
third of the total voting power. For simplicity we present the algorithm for
the case $n = 3f + 1$.
The algorithm proceeds in rounds, where each round has a dedicated
\emph{proposer}. The mapping of rounds to proposers is known to all processes
and is given as a function $\coord(h, round)$, returning the proposer for
the round $round$ in the consensus instance $h$. We
assume that the proposer selection function is weighted round-robin, where
processes are rotated proportional to their voting power\footnote{A validator
with more voting power is selected more frequently, proportional to its power.
More precisely, during a sequence of rounds of size $n$, every process is
proposer in a number of rounds equal to its voting power.}.
The internal protocol state transitions are triggered by message reception and
by expiration of timeouts. There are three timeouts in Algorithm \ref{alg:tendermint}:
$\timeoutPropose$, $\timeoutPrevote$ and $\timeoutPrecommit$.
The timeouts prevent the algorithm from blocking and
waiting forever for some condition to be true, ensure that processes continuously
transition between rounds, and guarantee that eventually (after GST) communication
between correct processes is timely and reliable so they can decide.
The last role is achieved by increasing the timeouts with every new round $r$,
i.e, $timeoutX(r) = initTimeoutX + r*timeoutDelta$;
they are reset for every new height (consensus
instance).
Processes exchange the following messages in Tendermint: $\Proposal$,
$\Prevote$ and $\Precommit$. The $\Proposal$ message is used by the proposer of
the current round to suggest a potential decision value, while $\Prevote$ and
$\Precommit$ are votes for a proposed value. According to the classification of
consensus algorithms from \cite{RMS10:dsn}, Tendermint, like PBFT
\cite{CL02:tcs} and DLS \cite{DLS88:jacm}, belongs to class 3, so it requires
two voting steps (three communication exchanges in total) to decide a value.
The Tendermint consensus algorithm is designed for the blockchain context where
the value to decide is a block of transactions (ie. it is potentially quite
large, consisting of many transactions). Therefore, in the Algorithm
\ref{alg:tendermint} (similar as in \cite{CL02:tcs}) we are explicit about
sending a value (block of transactions) and a small, constant size value id (a
unique value identifier, normally a hash of the value, i.e., if $\id(v) =
\id(v')$, then $v=v'$). The $\Proposal$ message is the only one carrying the
value; $\Prevote$ and $\Precommit$ messages carry the value id. A correct
process decides on a value $v$ in Tendermint upon receiving the $\Proposal$ for
$v$ and $2f+1$ voting-power equivalent $\Precommit$ messages for $\id(v)$ in
some round $r$. In order to send $\Precommit$ message for $v$ in a round $r$, a
correct process waits to receive the $\Proposal$ and $2f+1$ of the
corresponding $\Prevote$ messages in the round $r$. Otherwise,
it sends $\Precommit$ message with a special $\nil$ value.
This ensures that correct processes can $\Precommit$ only a
single value (or $\nil$) in a round. As
proposers may be faulty, the proposed value is treated by correct processes as
a suggestion (it is not blindly accepted), and a correct process tells others
if it accepted the $\Proposal$ for value $v$ by sending $\Prevote$ message for
$\id(v)$; otherwise it sends $\Prevote$ message with the special $\nil$ value.
Every process maintains the following variables in the Algorithm
\ref{alg:tendermint}: $step$, $lockedValue$, $lockedRound$, $validValue$ and
$validRound$. The $step$ denotes the current state of the internal Tendermint
state machine, i.e., it reflects the stage of the algorithm execution in the
current round. The $lockedValue$ stores the most recent value (with respect to
a round number) for which a $\Precommit$ message has been sent. The
$lockedRound$ is the last round in which the process sent a $\Precommit$
message that is not $\nil$. We also say that a correct process locks a value
$v$ in a round $r$ by setting $lockedValue = v$ and $lockedRound = r$ before
sending $\Precommit$ message for $\id(v)$. As a correct process can decide a
value $v$ only if $2f+1$ $\Precommit$ messages for $\id(v)$ are received, this
implies that a possible decision value is a value that is locked by at least
$f+1$ voting power equivalent of correct processes. Therefore, any value $v$
for which $\Proposal$ and $2f+1$ of the corresponding $\Prevote$ messages are
received in some round $r$ is a \emph{possible decision} value. The role of the
$validValue$ variable is to store the most recent possible decision value; the
$validRound$ is the last round in which $validValue$ is updated. Apart from
those variables, a process also stores the current consensus instance ($h_p$,
called \emph{height} in Tendermint), and the current round number ($round_p$)
and attaches them to every message. Finally, a process also stores an array of
decisions, $decision_p$ (Tendermint assumes a sequence of consensus instances,
one for each height).
Every round starts by a proposer suggesting a value with the $\Proposal$
message (see line \ref{line:tab:send-proposal}). In the initial round of each
height, the proposer is free to chose the value to suggest. In the
Algorithm~\ref{alg:tendermint}, a correct process obtains a value to propose
using an external function $getValue()$ that returns a valid value to
propose. In the following rounds, a correct proposer will suggest a new value
only if $validValue = \nil$; otherwise $validValue$ is proposed (see
lines~\ref{line:tab:isThereLockedValue}-\ref{line:tab:getValidValue}).
In addition to the value proposed, the $\Proposal$ message also
contains the $validRound$ so other processes are informed about the last round
in which the proposer observed $validValue$ as a possible decision value.
Note that if a correct proposer $p$ sends $validValue$ with the $validRound$ in the
$\Proposal$, this implies that the process $p$ received $\Proposal$ and the
corresponding $2f+1$ $\Prevote$ messages for $validValue$ in the round
$validRound$.
If a correct process sends $\Proposal$ message with $validValue$ ($validRound > -1$)
at time $t > GST$, by the \emph{Gossip communication} property, the
corresponding $\Proposal$ and the $\Prevote$ messages will be received by all
correct processes before time $t+\Delta$. Therefore, all correct processes will
be able to verify the correctness of the suggested value as it is supported by
the $\Proposal$ and the corresponding $2f+1$ voting power equivalent $\Prevote$
messages.
A correct process $p$ accepts the proposal for a value $v$ (send $\Prevote$
for $id(v)$) if an external \emph{valid} function returns $true$ for the value
$v$, and if $p$ hasn't locked any value ($lockedRound = -1$) or $p$ has locked
the value $v$ ($lockedValue = v$); see the line
\ref{line:tab:accept-proposal-2}. In case the proposed pair is $(v,vr \ge 0)$ and a
correct process $p$ has locked some value, it will accept
$v$ if it is a more recent possible decision value\footnote{As
explained above, the possible decision value in a round $r$ is the one for
which $\Proposal$ and the corresponding $2f+1$ $\Prevote$ messages are received
for the round $r$.}, $vr > lockedRound_p$, or if $lockedValue = v$
(see line~\ref{line:tab:cond-prevote-higher-proposal}). Otherwise, a correct
process will reject the proposal by sending $\Prevote$ message with $\nil$
value. A correct process will send $\Prevote$ message with $\nil$ value also in
case $\timeoutPropose$ expired (it is triggered when a correct process starts a
new round) and a process has not sent $\Prevote$ message in the current round
yet (see the line \ref{line:tab:onTimeoutPropose}).
If a correct process receives $\Proposal$ message for some value $v$ and $2f+1$
$\Prevote$ messages for $\id(v)$, then it sends $\Precommit$ message with
$\id(v)$. Otherwise, it sends $\Precommit$ $\nil$. A correct process will send
$\Precommit$ message with $\nil$ value also in case $\timeoutPrevote$ expired
(it is started when a correct process sent $\Prevote$ message and received any
$2f+1$ $\Prevote$ messages) and a process has not sent $\Precommit$ message in
the current round yet (see the line \ref{line:tab:onTimeoutPrecommit}). A
correct process decides on some value $v$ if it receives in some round $r$
$\Proposal$ message for $v$ and $2f+1$ $\Precommit$ messages with $\id(v)$ (see
the line \ref{line:tab:decide}). To prevent the algorithm from blocking and
waiting forever for this condition to be true, the Algorithm
\ref{alg:tendermint} relies on $\timeoutPrecommit$. It is triggered after a
process receives any set of $2f+1$ $\Precommit$ messages for the current round.
If the $\timeoutPrecommit$ expires and a process has not decided yet, the
process starts the next round (see the line \ref{line:tab:onTimeoutPrecommit}).
When a correct process $p$ decides, it starts the next consensus instance
(for the next height). The \emph{Gossip communication} property ensures
that $\Proposal$ and $2f+1$ $\Prevote$ messages that led $p$ to decide
are eventually received by all correct processes, so they will also decide.
\subsection{Termination mechanism}
Tendermint ensures termination by a novel mechanism that benefits from the
gossip based nature of communication (see \emph{Gossip communication}
property). It requires managing two additional variables, $validValue$ and
$validRound$ that are then used by the proposer during the propose step as
explained above. The $validValue$ and $validRound$ are updated to $v$ and $r$
by a correct process in a round $r$ when the process receives valid $\Proposal$
message for the value $v$ and the corresponding $2f+1$ $\Prevote$ messages for
$id(v)$ in the round $r$ (see the rule at line~\ref{line:tab:recvPrevote}).
We now give briefly the intuition how managing and proposing $validValue$
and $validRound$ ensures termination. Formal treatment is left for
Section~\ref{sec:proof}.
The first thing to note is that during good period, because of the
\emph{Gossip communication} property, if a correct process $p$ locks a value
$v$ in some round $r$, all correct processes will update $validValue$ to $v$
and $validRound$ to $r$ before the end of the round $r$ (we prove this formally
in the Section~\ref{sec:proof}). The intuition is that messages that led to $p$
locking a value $v$ in the round $r$ will be gossiped to all correct processes
before the end of the round $r$, so it will update $validValue$ and
$validRound$ (the line~\ref{line:tab:recvPrevote}). Therefore, if a correct
process locks some value during good period, $validValue$ and $validRound$ are
updated by all correct processes so that the value proposed in the following
rounds will be acceptable by all correct processes. Note
that it could happen that during good period, no correct process locks a value,
but some correct process $q$ updates $validValue$ and $validRound$ during some
round. As no correct process locks a value in this case, $validValue_q$ and
$validRound_q$ will also be acceptable by all correct processes as
$validRound_q > lockedRound_c$ for every correct process $c$ and as the
\emph{Gossip communication} property ensures that the corresponding $\Prevote$
messages that $q$ received in the round $validRound_q$ are received by all
correct processes $\Delta$ time later.
Finally, it could happen that after GST, there is a long sequence of rounds in which
no correct process neither locks a value nor update $validValue$ and $validRound$.
In this case, during this sequence of rounds, the proposed value suggested by correct
processes was not accepted by all correct processes. Note that this sequence of rounds
is always finite as at the beginning of every
round there is at least a single correct process $c$ such that $validValue_c$
and $validRound_c$ are acceptable by every correct process. This is true as
there exists a correct process $c$ such that for every other correct process
$p$, $validRound_c > lockedRound_p$ or $validValue_c = lockedValue_p$. This is
true as $c$ is the process that has locked a value in the most recent round
among all correct processes (or no correct process locked any value). Therefore,
eventually $c$ will be the proper in some round and the proposed value will be accepted
by all correct processes, terminating therefore this sequence of
rounds.
Therefore, updating $validValue$ and $validRound$ variables, and the
\emph{Gossip communication} property, together ensures that eventually, during
the good period, there exists a round with a correct proposer whose proposed
value will be accepted by all correct processes, and all correct processes will
terminate in that round. Note that this mechanism, contrary to the common
termination mechanism illustrated in the
Figure~\ref{ch3:fig:coordinator-change}, does not require exchanging any
additional information in addition to messages already sent as part of what is
normally being called "normal" case.

126
spec/consensus/consensus-paper/definitions.tex Normal file
View File

@@ -0,0 +1,126 @@
\section{Definitions} \label{sec:definitions}
\subsection{Model}
We consider a system of processes that communicate by exchanging messages.
Processes can be correct or faulty, where a faulty process can behave in an
arbitrary way, i.e., we consider Byzantine faults. We assume that each process
has some amount of voting power (voting power of a process can be $0$).
Processes in our model are not part of a single administrative domain;
therefore we cannot enforce a direct network connectivity between all
processes. Instead, we assume that each process is connected to a subset of
processes called peers, such that there is an indirect communication channel
between all correct processes. Communication between processes is established
using a gossip protocol \cite{Dem1987:gossip}.
Formally, we model the network communication using a variant of the \emph{partially
synchronous system model}~\cite{DLS88:jacm}: in all executions of the system
there is a bound $\Delta$ and an instant GST (Global Stabilization Time) such
that all communication among correct processes after GST is reliable and
$\Delta$-timely, i.e., if a correct process $p$ sends message $m$ at time $t
\ge GST$ to a correct process $q$, then $q$ will receive $m$ before $t +
\Delta$\footnote{Note that as we do not assume direct communication channels
among all correct processes, this implies that before the message $m$
reaches $q$, it might pass through a number of correct processes that will
forward the message $m$ using gossip protocol towards $q$.}.
In addition to the standard \emph{partially
synchronous system model}~\cite{DLS88:jacm}, we assume an auxiliary property
that captures gossip-based nature of communication\footnote{The details of the Tendermint gossip protocol will be discussed in a separate
technical report. }:
\begin{itemize} \item \emph{Gossip communication:} If a correct process $p$
sends some message $m$ at time $t$, all correct processes will receive
$m$ before $max\{t, GST\} + \Delta$. Furthermore, if a correct process $p$
receives some message $m$ at time $t$, all correct processes will receive
$m$ before $max\{t, GST\} + \Delta$. \end{itemize}
The bound $\Delta$ and GST are system
parameters whose values are not required to be known for the safety of our
algorithm. Termination of the algorithm is guaranteed within a bounded duration
after GST. In practice, the algorithm will work correctly in the slightly
weaker variant of the model where the system alternates between (long enough)
good periods (corresponds to the \emph{after} GST period where system is
reliable and $\Delta$-timely) and bad periods (corresponds to the period
\emph{before} GST during which the system is asynchronous and messages can be
lost), but consideration of the GST model simplifies the discussion.
We assume that process steps (which might include sending and receiving
messages) take zero time. Processes are equipped with clocks so they can
measure local timeouts.
Spoofing/impersonation attacks are assumed to be impossible at all times due to
the use of public-key cryptography, i.e., we assume that all protocol messages contains a digital signature.
Therefore, when a correct
process $q$ receives a signed message $m$ from its peer, the process $q$ can
verify who was the original sender of the message $m$ and if the message signature is valid.
We do not explicitly state a signature verification step in the pseudo-code of the algorithm to improve readability;
we assume that only messages with the valid signature are considered at that level (and messages with invalid signatures
are dropped).
%Messages that are being gossiped are created by the consensus layer. We can
%think about consensus protocol as a content creator, which %defines what
%messages should be disseminated using the gossip protocol. A correct
%process creates the message for dissemination either i) %explicitly, by
%invoking \emph{send} function as part of the consensus protocol or ii)
%implicitly, by receiving a message from some other %process. Note that in
%the case ii) gossiping of messages is implicit, i.e., it happens without
%explicit send clause in the consensus algorithm %whenever a correct
%process receives some messages in the consensus algorithm\footnote{If a
%message is received by a correct process at %the consensus level then it
%is considered valid from the protocol point of view, i.e., it has a
%correct signature, a proper message structure %and a valid height and
%round number.}.
%\item Processes keep resending messages (in case of failures or message loss)
%until all its peers get them. This ensures that every message %sent or
%received by a correct process is eventually received by all correct
%processes.
\subsection{State Machine Replication}
State machine replication (SMR) is a general approach for replicating services
modeled as a deterministic state machine~\cite{Lam78:cacm,Sch90:survey}. The
key idea of this approach is to guarantee that all replicas start in the same
state and then apply requests from clients in the same order, thereby
guaranteeing that the replicas' states will not diverge. Following
Schneider~\cite{Sch90:survey}, we note that the following is key for
implementing a replicated state machine tolerant to (Byzantine) faults:
\begin{itemize} \item \emph{Replica Coordination.} All [non-faulty] replicas
receive and process the same sequence of requests. \end{itemize}
Moreover, as Schneider also notes, this property can be decomposed into two
parts, \emph{Agreement} and \emph{Order}: Agreement requires all (non-faulty)
replicas to receive all requests, and Order requires that the order of received
requests is the same at all replicas.
There is an additional requirement that needs to be ensured by Byzantine
tolerant state machine replication: only requests (called transactions in the
Tendermint terminology) proposed by clients are executed. In Tendermint,
transaction verification is the responsibility of the service that is being
replicated; upon receiving a transaction from the client, the Tendermint
process will ask the service if the request is valid, and only valid requests
will be processed.
\subsection{Consensus} \label{sec:consensus}
Tendermint solves state machine replication by sequentially executing consensus
instances to agree on each block of transactions that are
then executed by the service being replicated. We consider a variant of the
Byzantine consensus problem called Validity Predicate-based Byzantine consensus
that is motivated by blockchain systems~\cite{GLR17:red-belly-bc}. The problem
is defined by an agreement, a termination, and a validity property.
\begin{itemize} \item \emph{Agreement:} No two correct processes decide on
different values. \item \emph{Termination:} All correct processes
eventually decide on a value. \item \emph{Validity:} A decided value
is valid, i.e., it satisfies the predefined predicate denoted
\emph{valid()}. \end{itemize}
This variant of the Byzantine consensus problem has an application-specific
\emph{valid()} predicate to indicate whether a value is valid. In the context
of blockchain systems, for example, a value is not valid if it does not
contain an appropriate hash of the last value (block) added to the blockchain.

32
spec/consensus/consensus-paper/homodel.sty Normal file
View File

@@ -0,0 +1,32 @@
\newcommand{\NC}{\mbox{\it NC}}
\newcommand{\HO}{\mbox{\it HO}}
\newcommand{\AS}{\mbox{\it AS}}
\newcommand{\SK}{\mbox{\it SK}}
\newcommand{\SHO}{\mbox{\it SHO}}
\newcommand{\AHO}{\mbox{\it AHO}}
\newcommand{\CONS}{\mbox{\it CONS}}
\newcommand{\K}{\mbox{\it K}}
\newcommand{\Alg}{\mathcal{A}}
\newcommand{\Pred}{\mathcal{P}}
\newcommand{\Spr}{S_p^r}
\newcommand{\Tpr}{T_p^r}
\newcommand{\mupr}{\vec{\mu}_p^{\,r}}
\newcommand{\MSpr}{S_p^{\rho}}
\newcommand{\MTpr}{T_p^{\rho}}
\newconstruct{\SEND}{$\Spr$:}{}{\ENDSEND}{}
\newconstruct{\TRAN}{$\Tpr$:}{}{\ENDTRAN}{}
\newconstruct{\ROUND}{\textbf{Round}}{\!\textbf{:}}{\ENDROUND}{}
\newconstruct{\VARIABLES}{\textbf{Variables:}}{}{\ENDVARIABLES}{}
\newconstruct{\INIT}{\textbf{Initialization:}}{}{\ENDINIT}{}
\newconstruct{\MSEND}{$\MSpr$:}{}{\ENDMSEND}{}
\newconstruct{\MTRAN}{$\MTpr$:}{}{\ENDMTRAN}{}
\newconstruct{\SROUND}{\textbf{Selection Round}}{\!\textbf{:}}{\ENDSROUND}{}
\newconstruct{\VROUND}{\textbf{Validation Round}}{\!\textbf{:}}{\ENDVROUND}{}
\newconstruct{\DROUND}{\textbf{Decision Round}}{\!\textbf{:}}{\ENDDROUND}{}

138
spec/consensus/consensus-paper/intro.tex Normal file
View File

@@ -0,0 +1,138 @@
\section{Introduction} \label{sec:tendermint}
Consensus is a fundamental problem in distributed computing. It
is important because of it's role in State Machine Replication (SMR), a generic
approach for replicating services that can be modeled as a deterministic state
machine~\cite{Lam78:cacm, Sch90:survey}. The key idea of this approach is that
service replicas start in the same initial state, and then execute requests
(also called transactions) in the same order; thereby guaranteeing that
replicas stay in sync with each other. The role of consensus in the SMR
approach is ensuring that all replicas receive transactions in the same order.
Traditionally, deployments of SMR based systems are in data-center settings
(local area network), have a small number of replicas (three to seven) and are
typically part of a single administration domain (e.g., Chubby
\cite{Bur:osdi06}); therefore they handle benign (crash) failures only, as more
general forms of failure (in particular, malicious or Byzantine faults) are
considered to occur with only negligible probability.
The success of cryptocurrencies and blockchain systems in recent years (e.g.,
\cite{Nak2012:bitcoin, But2014:ethereum}) pose a whole new set of challenges on
the design and deployment of SMR based systems: reaching agreement over wide
area network, among large number of nodes (hundreds or thousands) that are not
part of the same administrative domain, and where a subset of nodes can behave
maliciously (Byzantine faults). Furthermore, contrary to the previous
data-center deployments where nodes are fully connected to each other, in
blockchain systems, a node is only connected to a subset of other nodes, so
communication is achieved by gossip-based peer-to-peer protocols.
The new requirements demand designs and algorithms that are not necessarily
present in the classical academic literature on Byzantine fault tolerant
consensus (or SMR) systems (e.g., \cite{DLS88:jacm, CL02:tcs}) as the primary
focus was different setup.
In this paper we describe a novel Byzantine-fault tolerant consensus algorithm
that is the core of the BFT SMR platform called Tendermint\footnote{The
Tendermint platform is available open source at
https://github.com/tendermint/tendermint.}. The Tendermint platform consists of
a high-performance BFT SMR implementation written in Go, a flexible interface
for
building arbitrary deterministic applications above the consensus, and a suite
of tools for deployment and management.
The Tendermint consensus algorithm is inspired by the PBFT SMR
algorithm~\cite{CL99:osdi} and the DLS algorithm for authenticated faults (the
Algorithm 2 from \cite{DLS88:jacm}). Similar to DLS algorithm, Tendermint
proceeds in
rounds\footnote{Tendermint is not presented in the basic round model of
\cite{DLS88:jacm}. Furthermore, we use the term round differently than in
\cite{DLS88:jacm}; in Tendermint a round denotes a sequence of communication
steps instead of a single communication step in \cite{DLS88:jacm}.}, where each
round has a dedicated proposer (also called coordinator or
leader) and a process proceeds to a new round as part of normal
processing (not only in case the proposer is faulty or suspected as being faulty
by enough processes as in PBFT).
The communication pattern of each round is very similar to the "normal" case
of PBFT. Therefore, in preferable conditions (correct proposer, timely and
reliable communication between correct processes), Tendermint decides in three
communication steps (the same as PBFT).
The major novelty and contribution of the Tendermint consensus algorithm is a
new termination mechanism. As explained in \cite{MHS09:opodis, RMS10:dsn}, the
existing BFT consensus (and SMR) algorithms for the partially synchronous
system model (for example PBFT~\cite{CL99:osdi}, \cite{DLS88:jacm},
\cite{MA06:tdsc}) typically relies on the communication pattern illustrated in
Figure~\ref{ch3:fig:coordinator-change} for termination. The
Figure~\ref{ch3:fig:coordinator-change} illustrates messages exchanged during
the proposer change when processes start a new round\footnote{There is no
consistent terminology in the distributed computing terminology on naming
sequence of communication steps that corresponds to a logical unit. It is
sometimes called a round, phase or a view.}. It guarantees that eventually (ie.
after some Global Stabilization Time, GST), there exists a round with a correct
proposer that will bring the system into a univalent configuration.
Intuitively, in a round in which the proposed value is accepted
by all correct processes, and communication between correct processes is
timely and reliable, all correct processes decide.
\begin{figure}[tbh!] \def\rdstretch{5} \def\ystretch{3} \centering
\begin{rounddiag}{4}{2} \round{1}{~} \rdmessage{1}{1}{$v_1$}
\rdmessage{2}{1}{$v_2$} \rdmessage{3}{1}{$v_3$} \rdmessage{4}{1}{$v_4$}
\round{2}{~} \rdmessage{1}{1}{$x, [v_{1..4}]$}
\rdmessage{1}{2}{$~~~~~~x, [v_{1..4}]$} \rdmessage{1}{3}{$~~~~~~~~x,
[v_{1..4}]$} \rdmessage{1}{4}{$~~~~~~~x, [v_{1..4}]$} \end{rounddiag}
\vspace{-5mm} \caption{\boldmath Proposer (coordinator) change: $p_1$ is the
new proposer.} \label{ch3:fig:coordinator-change} \end{figure}
To ensure that a proposed value is accepted by all correct
processes\footnote{The proposed value is not blindly accepted by correct
processes in BFT algorithms. A correct process always verifies if the proposed
value is safe to be accepted so that safety properties of consensus are not
violated.}
a proposer will 1) build the global state by receiving messages from other
processes, 2) select the safe value to propose and 3) send the selected value
together with the signed messages
received in the first step to support it. The
value $v_i$ that a correct process sends to the next proposer normally
corresponds to a value the process considers as acceptable for a decision:
\begin{itemize} \item in PBFT~\cite{CL99:osdi} and DLS~\cite{DLS88:jacm} it is
not the value itself but a set of $2f+1$ signed messages with the same
value id, \item in Fast Byzantine Paxos~\cite{MA06:tdsc} the value
itself is being sent. \end{itemize}
In both cases, using this mechanism in our system model (ie. high
number of nodes over gossip based network) would have high communication
complexity that increases with the number of processes: in the first case as
the message sent depends on the total number of processes, and in the second
case as the value (block of transactions) is sent by each process. The set of
messages received in the first step are normally piggybacked on the proposal
message (in the Figure~\ref{ch3:fig:coordinator-change} denoted with
$[v_{1..4}]$) to justify the choice of the selected value $x$. Note that
sending this message also does not scale with the number of processes in the
system.
We designed a novel termination mechanism for Tendermint that better suits the
system model we consider. It does not require additional communication (neither
sending new messages nor piggybacking information on the existing messages) and
it is fully based on the communication pattern that is very similar to the
normal case in PBFT \cite{CL99:osdi}. Therefore, there is only a single mode of
execution in Tendermint, i.e., there is no separation between the normal and
the recovery mode, which is the case in other PBFT-like protocols (e.g.,
\cite{CL99:osdi}, \cite{Ver09:spinning} or \cite{Cle09:aardvark}). We believe
this makes Tendermint simpler to understand and implement correctly.
Note that the orthogonal approach for reducing message complexity in order to
improve
scalability and decentralization (number of processes) of BFT consensus
algorithms is using advanced cryptography (for example Boneh-Lynn-Shacham (BLS)
signatures \cite{BLS2001:crypto}) as done for example in SBFT
\cite{Gue2018:sbft}.
The remainder of the paper is as follows: Section~\ref{sec:definitions} defines
the system model and gives the problem definitions. Tendermint
consensus algorithm is presented in Section~\ref{sec:tendermint} and the
proofs are given in Section~\ref{sec:proof}. We conclude in
Section~\ref{sec:conclusion}.

1124
spec/consensus/consensus-paper/latex8.bst Normal file
View File

File diff suppressed because it is too large Load Diff

168
spec/consensus/consensus-paper/latex8.sty Normal file
View File

@@ -0,0 +1,168 @@
% ---------------------------------------------------------------
%
% $Id: latex8.sty,v 1.2 1995/09/15 15:31:13 ienne Exp $
%
% by Paolo.Ienne@di.epfl.ch
%
% ---------------------------------------------------------------
%
% no guarantee is given that the format corresponds perfectly to
% IEEE 8.5" x 11" Proceedings, but most features should be ok.
%
% ---------------------------------------------------------------
% with LaTeX2e:
% =============
%
% use as
% \documentclass[times,10pt,twocolumn]{article}
% \usepackage{latex8}
% \usepackage{times}
%
% ---------------------------------------------------------------
% with LaTeX 2.09:
% ================
%
% use as
% \documentstyle[times,art10,twocolumn,latex8]{article}
%
% ---------------------------------------------------------------
% with both versions:
% ===================
%
% specify \pagestyle{empty} to omit page numbers in the final
% version
%
% specify references as
% \bibliographystyle{latex8}
% \bibliography{...your files...}
%
% use Section{} and SubSection{} instead of standard section{}
% and subsection{} to obtain headings in the form
% "1.3. My heading"
%
% ---------------------------------------------------------------
\typeout{IEEE 8.5 x 11-Inch Proceedings Style `latex8.sty'.}
% ten point helvetica bold required for captions
% in some sites the name of the helvetica bold font may differ,
% change the name here:
\font\tenhv = phvb at 10pt
%\font\tenhv = phvb7t at 10pt
% eleven point times bold required for second-order headings
% \font\elvbf = cmbx10 scaled 1100
\font\elvbf = ptmb scaled 1100
% set dimensions of columns, gap between columns, and paragraph indent
\setlength{\textheight}{8.875in}
\setlength{\textwidth}{6.875in}
\setlength{\columnsep}{0.3125in}
\setlength{\topmargin}{0in}
\setlength{\headheight}{0in}
\setlength{\headsep}{0in}
\setlength{\parindent}{1pc}
\setlength{\oddsidemargin}{-.304in}
\setlength{\evensidemargin}{-.304in}
% memento from size10.clo
% \normalsize{\@setfontsize\normalsize\@xpt\@xiipt}
% \small{\@setfontsize\small\@ixpt{11}}
% \footnotesize{\@setfontsize\footnotesize\@viiipt{9.5}}
% \scriptsize{\@setfontsize\scriptsize\@viipt\@viiipt}
% \tiny{\@setfontsize\tiny\@vpt\@vipt}
% \large{\@setfontsize\large\@xiipt{14}}
% \Large{\@setfontsize\Large\@xivpt{18}}
% \LARGE{\@setfontsize\LARGE\@xviipt{22}}
% \huge{\@setfontsize\huge\@xxpt{25}}
% \Huge{\@setfontsize\Huge\@xxvpt{30}}
\def\@maketitle
{
\newpage
\null
\vskip .375in
\begin{center}
{\Large \bf \@title \par}
% additional two empty lines at the end of the title
\vspace*{24pt}
{
\large
\lineskip .5em
\begin{tabular}[t]{c}
\@author
\end{tabular}
\par
}
% additional small space at the end of the author name
\vskip .5em
{
\large
\begin{tabular}[t]{c}
\@affiliation
\end{tabular}
\par
\ifx \@empty \@email
\else
\begin{tabular}{r@{~}l}
E-mail: & {\tt \@email}
\end{tabular}
\par
\fi
}
% additional empty line at the end of the title block
\vspace*{12pt}
\end{center}
}
\def\abstract
{%
\centerline{\large\bf Abstract}%
\vspace*{12pt}%
\it%
}
\def\endabstract
{
% additional empty line at the end of the abstract
\vspace*{12pt}
}
\def\affiliation#1{\gdef\@affiliation{#1}} \gdef\@affiliation{}
\def\email#1{\gdef\@email{#1}}
\gdef\@email{}
\newlength{\@ctmp}
\newlength{\@figindent}
\setlength{\@figindent}{1pc}
\long\def\@makecaption#1#2{
\vskip 10pt
\setbox\@tempboxa\hbox{\tenhv\noindent #1.~#2}
\setlength{\@ctmp}{\hsize}
\addtolength{\@ctmp}{-\@figindent}\addtolength{\@ctmp}{-\@figindent}
% IF longer than one indented paragraph line
\ifdim \wd\@tempboxa >\@ctmp
% THEN set as an indented paragraph
\begin{list}{}{\leftmargin\@figindent \rightmargin\leftmargin}
\item[]\tenhv #1.~#2\par
\end{list}
\else
% ELSE center
\hbox to\hsize{\hfil\box\@tempboxa\hfil}
\fi}
% correct heading spacing and type
\def\section{\@startsection {section}{1}{\z@}
{14pt plus 2pt minus 2pt}{14pt plus 2pt minus 2pt} {\large\bf}}
\def\subsection{\@startsection {subsection}{2}{\z@}
{13pt plus 2pt minus 2pt}{13pt plus 2pt minus 2pt} {\elvbf}}
% add the period after section numbers
\newcommand{\Section}[1]{\section{\hskip -1em.~#1}}
\newcommand{\SubSection}[1]{\subsection{\hskip -1em.~#1}}
% end of file latex8.sty
% ---------------------------------------------------------------

1659
spec/consensus/consensus-paper/lit.bib Normal file
View File

File diff suppressed because it is too large Load Diff

153
spec/consensus/consensus-paper/paper.tex Normal file
View File

@@ -0,0 +1,153 @@
%\documentclass[conference]{IEEEtran}
\documentclass[conference,onecolumn,draft,a4paper]{IEEEtran}
% Add the compsoc option for Computer Society conferences.
%
% If IEEEtran.cls has not been installed into the LaTeX system files,
% manually specify the path to it like:
% \documentclass[conference]{../sty/IEEEtran}
% *** GRAPHICS RELATED PACKAGES ***
%
\ifCLASSINFOpdf
\else
\fi
% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}
%\usepackage[caption=false,font=footnotesize]{subfig}
\usepackage{tikz}
\usetikzlibrary{decorations,shapes,backgrounds,calc}
\tikzstyle{msg}=[->,black,>=latex]
\tikzstyle{rubber}=[|<->|]
\tikzstyle{announce}=[draw=blue,fill=blue,shape=diamond,right,minimum
height=2mm,minimum width=1.6667mm,inner sep=0pt]
\tikzstyle{decide}=[draw=red,fill=red,shape=isosceles triangle,right,minimum
height=2mm,minimum width=1.6667mm,inner sep=0pt,shape border rotate=90]
\tikzstyle{cast}=[draw=green!50!black,fill=green!50!black,shape=circle,left,minimum
height=2mm,minimum width=1.6667mm,inner sep=0pt]
\usepackage{multirow}
\usepackage{graphicx}
\usepackage{epstopdf}
\usepackage{amssymb}
\usepackage{rounddiag}
\graphicspath{{../}}
\usepackage{technote}
\usepackage{homodel}
\usepackage{enumerate}
%%\usepackage{ulem}\normalem
% to center caption
\usepackage{caption}
\newcommand{\textstretch}{1.4}
\newcommand{\algostretch}{1}
\newcommand{\eqnstretch}{0.5}
\newconstruct{\FOREACH}{\textbf{for each}}{\textbf{do}}{\ENDFOREACH}{}
%\newconstruct{\ON}{\textbf{on}}{\textbf{do}}{\ENDON}{\textbf{end on}}
\newcommand\With{\textbf{while}}
\newcommand\From{\textbf{from}}
\newcommand\Broadcast{\textbf{broadcast}}
\newcommand\PBroadcast{send}
\newcommand\UpCall{\textbf{UpCall}}
\newcommand\DownCall{\textbf{DownCall}}
\newcommand \Call{\textbf{Call}}
\newident{noop}
\newconstruct{\UPON}{\textbf{upon}}{\textbf{do}}{\ENDUPON}{}
\newcommand{\abcast}{\mathsf{to\mbox{\sf-}broadcast}}
\newcommand{\adeliver}{\mathsf{to\mbox{\sf-}deliver}}
\newcommand{\ABCAgreement}{\emph{TO-Agreement}}
\newcommand{\ABCIntegrity}{\emph{TO-Integrity}}
\newcommand{\ABCValidity}{\emph{TO-Validity}}
\newcommand{\ABCTotalOrder}{\emph{TO-Order}}
\newcommand{\ABCBoundedDelivery}{\emph{TO-Bounded Delivery}}
\newcommand{\tabc}{\mathit{atab\mbox{\sf-}cast}}
\newcommand{\anno}{\mathit{atab\mbox{\sf-}announce}}
\newcommand{\abort}{\mathit{atab\mbox{\sf-}abort}}
\newcommand{\tadel}{\mathit{atab\mbox{\sf-}deliver}}
\newcommand{\ATABAgreement}{\emph{ATAB-Agreement}}
\newcommand{\ATABAbort}{\emph{ATAB-Abort}}
\newcommand{\ATABIntegrity}{\emph{ATAB-Integrity}}
\newcommand{\ATABValidity}{\emph{ATAB-Validity}}
\newcommand{\ATABAnnounce}{\emph{ATAB-Announcement}}
\newcommand{\ATABTermination}{\emph{ATAB-Termination}}
%\newcommand{\ATABFastAnnounce}{\emph{ATAB-Fast-Announcement}}
%% Command for observations.
\newtheorem{observation}{Observation}
%% HO ALGORITHM DEFINITIONS
\newconstruct{\FUNCTION}{\textbf{Function}}{\textbf{:}}{\ENDFUNCTION}{}
%% Uncomment the following four lines to remove remarks and visible traces of
%% modifications in the document
%%\renewcommand{\sout}[1]{\relaxx}
%%\renewcommand{\uline}[1]{#1}
%% \renewcommand{\uwave}[1]{#1}
\renewcommand{\note}[2][default]{\relax}
%% The following commands can be used to generate TR or Conference version of the paper
\newcommand{\tr}[1]{}
\renewcommand{\tr}[1]{#1}
\newcommand{\onlypaper}[1]{#1}
%\renewcommand{\onlypaper}[1]{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\pagestyle{plain}
%\pagestyle{empty}
%% IEEE tweaks
%\setlength{\IEEEilabelindent}{.5\parindent}
%\setlength{\IEEEiednormlabelsep}{.5\parindent}
\begin{document}
%
% paper title
% can use linebreaks \\ within to get better formatting as desired
\title{The latest gossip on BFT consensus\vspace{-0.7\baselineskip}}
\author{\IEEEauthorblockN{\large Ethan Buchman, Jae Kwon and Zarko Milosevic\\}
\IEEEauthorblockN{\large Tendermint}\\
%\\\vspace{-0.5\baselineskip}
\IEEEauthorblockN{September 24, 2018}
}
% make the title area
\maketitle
\vspace*{0.5em}
\begin{abstract}
This paper presents Tendermint, a new protocol for ordering events in a distributed network under adversarial conditions. More commonly known as Byzantine Fault Tolerant (BFT) consensus or atomic broadcast, the problem has attracted significant attention in recent years due to the widespread success of blockchain-based digital currencies, such as Bitcoin and Ethereum, which successfully solved the problem in a public setting without a central authority. Tendermint modernizes classic academic work on the subject and simplifies the design of the BFT algorithm by relying on a peer-to-peer gossip protocol among nodes.
\end{abstract}
%\noindent \textbf{Keywords:} Blockchain, Byzantine Fault Tolerance, State Machine %Replication
\input{intro}
\input{definitions}
\input{consensus}
\input{proof}
\input{conclusion}
\bibliographystyle{IEEEtran}
\bibliography{lit}
%\appendix
\end{document}

280
spec/consensus/consensus-paper/proof.tex Normal file
View File

@@ -0,0 +1,280 @@
\section{Proof of Tendermint consensus algorithm} \label{sec:proof}
\begin{lemma} \label{lemma:majority-intersection} For all $f\geq 0$, any two
sets of processes with voting power at least equal to $2f+1$ have at least one
correct process in common. \end{lemma}
\begin{proof} As the total voting power is equal to $n=3f+1$, we have $2(2f+1)
= n+f+1$. This means that the intersection of two sets with the voting
power equal to $2f+1$ contains at least $f+1$ voting power in common, \ie,
at least one correct process (as the total voting power of faulty processes
is $f$). The result follows directly from this. \end{proof}
\begin{lemma} \label{lemma:locked-decision_value-prevote-v} If $f+1$ correct
processes lock value $v$ in round $r_0$ ($lockedValue = v$ and $lockedRound =
r_0$), then in all rounds $r > r_0$, they send $\Prevote$ for $id(v)$ or
$\nil$. \end{lemma}
\begin{proof} We prove the result by induction on $r$.
\emph{Base step $r = r_0 + 1:$} Let's denote with $C$ the set of correct
processes with voting power equal to $f+1$. By the rules at
line~\ref{line:tab:recvProposal} and line~\ref{line:tab:acceptProposal}, the
processes from the set $C$ can't accept $\Proposal$ for any value different
from $v$ in round $r$, and therefore can't send a $\li{\Prevote,height_p,
r,id(v')}$ message, if $v' \neq v$. Therefore, the Lemma holds for the base
step.
\emph{Induction step from $r_1$ to $r_1+1$:} We assume that no process from the
set $C$ has sent $\Prevote$ for values different than $id(v)$ or $\nil$ until
round $r_1 + 1$. We now prove that the Lemma also holds for round $r_1 + 1$. As
processes from the set $C$ send $\Prevote$ for $id(v)$ or $\nil$ in rounds $r_0
\le r \le r_1$, by Lemma~\ref{lemma:majority-intersection} there is no value
$v' \neq v$ for which it is possible to receive $2f+1$ $\Prevote$ messages in
those rounds (i). Therefore, we have for all processes from the set $C$,
$lockedValue = v$ and $lockedRound \ge r_0$. Let's assume by a contradiction
that a process $q$ from the set $C$ sends $\Prevote$ in round $r_1 + 1$ for
value $id(v')$, where $v' \neq v$. This is possible only by
line~\ref{line:tab:prevote-higher-proposal}. Note that this implies that $q$
received $2f+1$ $\li{\Prevote,h_q, r,id(v')}$ messages, where $r > r_0$ and $r
< r_1 +1$ (see line~\ref{line:tab:cond-prevote-higher-proposal}). A
contradiction with (i) and Lemma~\ref{lemma:majority-intersection}.
\end{proof}
\begin{lemma} \label{lemma:agreement} Algorithm~\ref{alg:tendermint} satisfies
Agreement. \end{lemma}
\begin{proof} Let round $r_0$ be the first round of height $h$ such that some
correct process $p$ decides $v$. We now prove that if some correct process
$q$ decides $v'$ in some round $r \ge r_0$, then $v = v'$.
In case $r = r_0$, $q$ has received at least $2f+1$
$\li{\Precommit,h_p,r_0,id(v')}$ messages at line~\ref{line:tab:onDecideRule},
while $p$ has received at least $2f+1$ $\li{\Precommit,h_p,r_0,id(v)}$
messages. By Lemma~\ref{lemma:majority-intersection} two sets of messages of
voting power $2f+1$ intersect in at least one correct process. As a correct
process sends a single $\Precommit$ message in a round, then $v=v'$.
We prove the case $r > r_0$ by contradiction. By the
rule~\ref{line:tab:onDecideRule}, $p$ has received at least $2f+1$ voting-power
equivalent of $\li{\Precommit,h_p,r_0,id(v)}$ messages, i.e., at least $f+1$
voting-power equivalent correct processes have locked value $v$ in round $r_0$ and have
sent those messages (i). Let denote this set of messages with $C$. On the
other side, $q$ has received at least $2f+1$ voting power equivalent of
$\li{\Precommit,h_q, r,id(v')}$ messages. As the voting power of all faulty
processes is at most $f$, some correct process $c$ has sent one of those
messages. By the rule at line~\ref{line:tab:recvPrevote}, $c$ has locked value
$v'$ in round $r$ before sending $\li{\Precommit,h_q, r,id(v')}$. Therefore $c$
has received $2f+1$ $\Prevote$ messages for $id(v')$ in round $r > r_0$ (see
line~\ref{line:tab:recvPrevote}). By Lemma~\ref{lemma:majority-intersection}, a
process from the set $C$ has sent $\Prevote$ message for $id(v')$ in round $r$.
A contradiction with (i) and Lemma~\ref{lemma:locked-decision_value-prevote-v}.
\end{proof}
\begin{lemma} \label{lemma:agreement} Algorithm~\ref{alg:tendermint} satisfies
Validity. \end{lemma}
\begin{proof} Trivially follows from the rule at line
\ref{line:tab:validDecisionValue} which ensures that only valid values can be
decided. \end{proof}
\begin{lemma} \label{lemma:round-synchronisation} If we assume that:
\begin{enumerate}
\item a correct process $p$ is the first correct process to
enter a round $r>0$ at time $t > GST$ (for every correct process
$c$, $round_c \le r$ at time $t$)
\item the proposer of round $r$ is
a correct process $q$
\item for every correct process $c$,
$lockedRound_c \le validRound_q$ at time $t$
\item $\timeoutPropose(r)
> 2\Delta + \timeoutPrecommit(r-1)$, $\timeoutPrevote(r) > 2\Delta$ and
$\timeoutPrecommit(r) > 2\Delta$,
\end{enumerate}
then all correct processes decide in round $r$ before $t + 4\Delta +
\timeoutPrecommit(r-1)$.
\end{lemma}
\begin{proof} As $p$ is the first correct process to enter round $r$, it
executed the line~\ref{line:tab:nextRound} after $\timeoutPrecommit(r-1)$
expired. Therefore, $p$ received $2f+1$ $\Precommit$ messages in the round
$r-1$ before time $t$. By the \emph{Gossip communication} property, all
correct processes will receive those messages the latest at time $t +
\Delta$. Correct processes that are in rounds $< r-1$ at time $t$ will
enter round $r-1$ (see the rule at line~\ref{line:tab:nextRound2}) and
trigger $\timeoutPrecommit(r-1)$ (see rule~\ref{line:tab:startTimeoutPrecommit})
by time $t+\Delta$. Therefore, all correct processes will start round $r$
by time $t+\Delta+\timeoutPrecommit(r-1)$ (i).
In the worst case, the process $q$ is the last correct process to enter round
$r$, so $q$ starts round $r$ and sends $\Proposal$ message for some value $v$
at time $t + \Delta + \timeoutPrecommit(r-1)$. Therefore, all correct processes
receive the $\Proposal$ message from $q$ the latest by time $t + 2\Delta +
\timeoutPrecommit(r-1)$. Therefore, if $\timeoutPropose(r) > 2\Delta +
\timeoutPrecommit(r-1)$, all correct processes will receive $\Proposal$ message
before $\timeoutPropose(r)$ expires.
By (3) and the rules at line~\ref{line:tab:recvProposal} and
\ref{line:tab:acceptProposal}, all correct processes will accept the
$\Proposal$ message for value $v$ and will send a $\Prevote$ message for
$id(v)$ by time $t + 2\Delta + \timeoutPrecommit(r-1)$. Note that by the
\emph{Gossip communication} property, the $\Prevote$ messages needed to trigger
the rule at line~\ref{line:tab:acceptProposal} are received before time $t +
\Delta$.
By time $t + 3\Delta + \timeoutPrecommit(r-1)$, all correct processes will receive
$\Proposal$ for $v$ and $2f+1$ corresponding $\Prevote$ messages for $id(v)$.
By the rule at line~\ref{line:tab:recvPrevote}, all correct processes will send
a $\Precommit$ message (see line~\ref{line:tab:precommit-v}) for $id(v)$ by
time $t + 3\Delta + \timeoutPrecommit(r-1)$. Therefore, by time $t + 4\Delta +
\timeoutPrecommit(r-1)$, all correct processes will have received the $\Proposal$
for $v$ and $2f+1$ $\Precommit$ messages for $id(v)$, so they decide at
line~\ref{line:tab:decide} on $v$.
This scenario holds if every correct process $q$ sends a $\Precommit$ message
before $\timeoutPrevote(r)$ expires, and if $\timeoutPrecommit(r)$ does not expire
before $t + 4\Delta + \timeoutPrecommit(r-1)$. Let's assume that a correct process
$c_1$ is the first correct process to trigger $\timeoutPrevote(r)$ (see the rule
at line~\ref{line:tab:recvAny2/3Prevote}) at time $t_1 > t$. This implies that
before time $t_1$, $c_1$ received a $\Proposal$ ($step_{c_1}$ must be
$\prevote$ by the rule at line~\ref{line:tab:recvAny2/3Prevote}) and a set of
$2f+1$ $\Prevote$ messages. By time $t_1 + \Delta$, all correct processes will
receive those messages. Note that even if some correct process was in the
smaller round before time $t_1$, at time $t_1 + \Delta$ it will start round $r$
after receiving those messages (see the rule at
line~\ref{line:tab:skipRounds}). Therefore, all correct processes will send
their $\Prevote$ message for $id(v)$ by time $t_1 + \Delta$, and all correct
processes will receive those messages the by time $t_1 + 2\Delta$. Therefore,
as $\timeoutPrevote(r) > 2\Delta$, this ensures that all correct processes receive
$\Prevote$ messages from all correct processes before their respective local
$\timeoutPrevote(r)$ expire.
On the other hand, $\timeoutPrecommit(r)$ is triggered in a correct process $c_2$
after it receives any set of $2f+1$ $\Precommit$ messages for the first time.
Let's denote with $t_2 > t$ the earliest point in time $\timeoutPrecommit(r)$ is
triggered in some correct process $c_2$. This implies that $c_2$ has received
at least $f+1$ $\Precommit$ messages for $id(v)$ from correct processes, i.e.,
those processes have received $\Proposal$ for $v$ and $2f+1$ $\Prevote$
messages for $id(v)$ before time $t_2$. By the \emph{Gossip communication}
property, all correct processes will receive those messages by time $t_2 +
\Delta$, and will send $\Precommit$ messages for $id(v)$. Note that even if
some correct processes were at time $t_2$ in a round smaller than $r$, by the
rule at line~\ref{line:tab:skipRounds} they will enter round $r$ by time $t_2 +
\Delta$. Therefore, by time $t_2 + 2\Delta$, all correct processes will
receive $\Proposal$ for $v$ and $2f+1$ $\Precommit$ messages for $id(v)$. So if
$\timeoutPrecommit(r) > 2\Delta$, all correct processes will decide before the
timeout expires. \end{proof}
\begin{lemma} \label{lemma:validValue} If a correct process $p$ locks a value
$v$ at time $t_0 > GST$ in some round $r$ ($lockedValue = v$ and
$lockedRound = r$) and $\timeoutPrecommit(r) > 2\Delta$, then all correct
processes set $validValue$ to $v$ and $validRound$ to $r$ before starting
round $r+1$. \end{lemma}
\begin{proof} In order to prove this Lemma, we need to prove that if the
process $p$ locks a value $v$ at time $t_0$, then no correct process will
leave round $r$ before time $t_0 + \Delta$ (unless it has already set
$validValue$ to $v$ and $validRound$ to $r$). It is sufficient to prove
this, since by the \emph{Gossip communication} property the messages that
$p$ received at time $t_0$ and that triggered rule at
line~\ref{line:tab:recvPrevote} will be received by time $t_0 + \Delta$ by
all correct processes, so all correct processes that are still in round $r$
will set $validValue$ to $v$ and $validRound$ to $r$ (by the rule at
line~\ref{line:tab:recvPrevote}). To prove this, we need to compute the
earliest point in time a correct process could leave round $r$ without
updating $validValue$ to $v$ and $validRound$ to $r$ (we denote this time
with $t_1$). The Lemma is correct if $t_0 + \Delta < t_1$.
If the process $p$ locks a value $v$ at time $t_0$, this implies that $p$
received the valid $\Proposal$ message for $v$ and $2f+1$
$\li{\Prevote,h,r,id(v)}$ at time $t_0$. At least $f+1$ of those messages are
sent by correct processes. Let's denote this set of correct processes as $C$. By
Lemma~\ref{lemma:majority-intersection} any set of $2f+1$ $\Prevote$ messages
in round $r$ contains at least a single message from the set $C$.
Let's denote as time $t$ the earliest point in time a correct process, $c_1$, triggered
$\timeoutPrevote(r)$. This implies that $c_1$ received $2f+1$ $\Prevote$ messages
(see the rule at line \ref{line:tab:recvAny2/3Prevote}), where at least one of
those messages was sent by a process $c_2$ from the set $C$. Therefore, process
$c_2$ had received $\Proposal$ message before time $t$. By the \emph{Gossip
communication} property, all correct processes will receive $\Proposal$ and
$2f+1$ $\Prevote$ messages for round $r$ by time $t+\Delta$. The latest point
in time $p$ will trigger $\timeoutPrevote(r)$ is $t+\Delta$\footnote{Note that
even if $p$ was in smaller round at time $t$ it will start round $r$ by time
$t+\Delta$.}. So the latest point in time $p$ can lock the value $v$ in
round $r$ is $t_0 = t+\Delta+\timeoutPrevote(r)$ (as at this point
$\timeoutPrevote(r)$ expires, so a process sends $\Precommit$ $\nil$ and updates
$step$ to $\precommit$, see line \ref{line:tab:onTimeoutPrevote}).
Note that according to the Algorithm \ref{alg:tendermint}, a correct process
can not send a $\Precommit$ message before receiving $2f+1$ $\Prevote$
messages. Therefore, no correct process can send a $\Precommit$ message in
round $r$ before time $t$. If a correct process sends a $\Precommit$ message
for $\nil$, it implies that it has waited for the full duration of
$\timeoutPrevote(r)$ (see line
\ref{line:tab:precommit-nil-onTimeout})\footnote{The other case in which a
correct process $\Precommit$ for $\nil$ is after receiving $2f+1$ $Prevote$ for
$\nil$ messages, see the line \ref{line:tab:precommit-v-1}. By
Lemma~\ref{lemma:majority-intersection}, this is not possible in round $r$.}.
Therefore, no correct process can send $\Precommit$ for $\nil$ before time $t +
\timeoutPrevote(r)$ (*).
A correct process $q$ that enters round $r+1$ must wait (i) $\timeoutPrecommit(r)$
(see line \ref{line:tab:nextRound}) or (ii) receiving $f+1$ messages from the
round $r+1$ (see the line \ref{line:tab:skipRounds}). In the former case, $q$
receives $2f+1$ $\Precommit$ messages before starting $\timeoutPrecommit(r)$. If
at least a single $\Precommit$ message from a correct process (at least $f+1$
voting power equivalent of those messages is sent by correct processes) is for
$\nil$, then $q$ cannot start round $r+1$ before time $t_1 = t +
\timeoutPrevote(r) + \timeoutPrecommit(r)$ (see (*)). Therefore in this case we have:
$t_0 + \Delta < t_1$, i.e., $t+2\Delta+\timeoutPrevote(r) < t + \timeoutPrevote(r) +
\timeoutPrecommit(r)$, and this is true whenever $\timeoutPrecommit(r) > 2\Delta$, so
Lemma holds in this case.
If in the set of $2f+1$ $\Precommit$ messages $q$ receives, there is at least a
single $\Precommit$ for $id(v)$ message from a correct process $c$, then $q$
can start the round $r+1$ the earliest at time $t_1 = t+\timeoutPrecommit(r)$. In
this case, by the \emph{Gossip communication} property, all correct processes
will receive $\Proposal$ and $2f+1$ $\Prevote$ messages (that $c$ received
before time $t$) the latest at time $t+\Delta$. Therefore, $q$ will set
$validValue$ to $v$ and $validRound$ to $r$ the latest at time $t+\Delta$. As
$t+\Delta < t+\timeoutPrecommit(r)$, whenever $\timeoutPrecommit(r) > \Delta$, the
Lemma holds also in this case.
In case (ii), $q$ received at least a single message from a correct process $c$
from the round $r+1$. The earliest point in time $c$ could have started round
$r+1$ is $t+\timeoutPrecommit(r)$ in case it received a $\Precommit$ message for
$v$ from some correct process in the set of $2f+1$ $\Precommit$ messages it
received. The same reasoning as above holds also in this case, so $q$ set
$validValue$ to $v$ and $validRound$ to $r$ the latest by time $t+\Delta$. As
$t+\Delta < t+\timeoutPrecommit(r)$, whenever $\timeoutPrecommit(r) > \Delta$, the
Lemma holds also in this case. \end{proof}
\begin{lemma} \label{lemma:agreement} Algorithm~\ref{alg:tendermint} satisfies
Termination. \end{lemma}
\begin{proof} Lemma~\ref{lemma:round-synchronisation} defines a scenario in
which all correct processes decide. We now prove that within a bounded
duration after GST such a scenario will unfold. Let's assume that at time
$GST$ the highest round started by a correct process is $r_0$, and that
there exists a correct process $p$ such that the following holds: for every
correct process $c$, $lockedRound_c \le validRound_p$. Furthermore, we
assume that $p$ will be the proposer in some round $r_1 > r$ (this is
ensured by the $\coord$ function).
We have two cases to consider. In the first case, for all rounds $r \ge r_0$
and $r < r_1$, no correct process locks a value (set $lockedRound$ to $r$). So
in round $r_1$ we have the scenario from the
Lemma~\ref{lemma:round-synchronisation}, so all correct processes decides in
round $r_1$.
In the second case, a correct process locks a value $v$ in round $r_2$, where
$r_2 \ge r_0$ and $r_2 < r_1$. Let's assume that $r_2$ is the highest round
before $r_1$ in which some correct process $q$ locks a value. By Lemma
\ref{lemma:validValue} at the end of round $r_2$ the following holds for all
correct processes $c$: $validValue_c = lockedValue_q$ and $validRound_c = r_2$.
Then in round $r_1$, the conditions for the
Lemma~\ref{lemma:round-synchronisation} holds, so all correct processes decide.
\end{proof}

62
spec/consensus/consensus-paper/rounddiag.sty Normal file
View File

@@ -0,0 +1,62 @@
% ROUNDDIAG STYLE
% for LaTeX version 2e
% by -- 2008 Martin Hutle <martin.hutle@epfl.ch>
%
% This style file is free software; you can redistribute it and/or
% modify it under the terms of the GNU Lesser General Public
% License as published by the Free Software Foundation; either
% version 2 of the License, or (at your option) any later version.
%
% This style file is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
% Lesser General Public License for more details.
%
% You should have received a copy of the GNU Lesser General Public
% License along with this style file; if not, write to the
% Free Software Foundation, Inc., 59 Temple Place - Suite 330,
% Boston, MA 02111-1307, USA.
%
\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{rounddiag}
\typeout{Document Style `rounddiag' - provides simple round diagrams}
%
\RequirePackage{ifthen}
\RequirePackage{calc}
\RequirePackage{tikz}
\def\rdstretch{3}
\tikzstyle{msg}=[->,thick,>=latex]
\tikzstyle{rndline}=[dotted]
\tikzstyle{procline}=[dotted]
\newenvironment{rounddiag}[2]{
\begin{center}
\begin{tikzpicture}
\foreach \i in {1,...,#1}{
\draw[procline] (0,#1-\i) node[xshift=-1em]{$p_{\i}$} -- (#2*\rdstretch+1,#1-\i);
}
\foreach \i in {0,...,#2}{
\draw[rndline] (\i*\rdstretch+0.5,0) -- (\i*\rdstretch+0.5,#1-1);
}
\newcommand{\rdat}[2]{
(##2*\rdstretch+0.5,#1-##1)
}%
\newcommand{\round}[2]{%
\def\rdround{##1}
\ifthenelse{\equal{##2}{}}{}{
\node[yshift=-1em] at ({##1*\rdstretch+0.5-0.5*\rdstretch},0) {##2};
}
}%
\newcommand{\rdmessage}[3]{\draw[msg]
(\rdround*\rdstretch-\rdstretch+0.5,#1-##1) -- node[yshift=1.2ex]{##3}
(\rdround*\rdstretch+0.5,#1-##2);}%
\newcommand{\rdalltoall}{%
\foreach \i in {1,...,#1}{
\foreach \j in {1,...,#1}{
{ \rdmessage{\i}{\j}{}}}}}%
}{%
\end{tikzpicture}
\end{center}
}

118
spec/consensus/consensus-paper/technote.sty Normal file
View File

@@ -0,0 +1,118 @@
\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{technote}[2007/11/09]
\typeout{Template for quick notes with some useful definitions}
\RequirePackage{ifthen}
\RequirePackage{calc}
\RequirePackage{amsmath,amssymb,amsthm}
\RequirePackage{epsfig}
\RequirePackage{algorithm}
\RequirePackage[noend]{algorithmicplus}
\newboolean{technote@noedit}
\setboolean{technote@noedit}{false}
\DeclareOption{noedit}{\setboolean{technote@noedit}{true}}
\newcounter{technote@lang}
\setcounter{technote@lang}{0}
\DeclareOption{german}{\setcounter{technote@lang}{1}}
\DeclareOption{french}{\setcounter{technote@lang}{2}}
\DeclareOption{fullpage}{
\oddsidemargin -10mm % Margin on odd side pages (default=0mm)
\evensidemargin -10mm % Margin on even side pages (default=0mm)
\topmargin -10mm % Top margin space (default=16mm)
\headheight \baselineskip % Height of headers (default=0mm)
\headsep \baselineskip % Separation spc btw header and text (d=0mm)
\footskip 30pt % Separation spc btw text and footer (d=30pt)
\textheight 230mm % Total text height (default=200mm)
\textwidth 180mm % Total text width (default=160mm)
}
\renewcommand{\algorithmiccomment}[1]{\hfill/* #1 */}
\renewcommand{\algorithmiclnosize}{\scriptsize}
\newboolean{technote@truenumbers}
\setboolean{technote@truenumbers}{false}
\DeclareOption{truenumbers}{\setboolean{technote@truenumbers}{true}}
\ProcessOptions
\newcommand{\N}{\ifthenelse{\boolean{technote@truenumbers}}%
{\mbox{\rm I\hspace{-.5em}N}}%
{\mathbb{N}}}
\newcommand{\R}{\ifthenelse{\boolean{technote@truenumbers}}%
{\mbox{\rm I\hspace{-.2em}R}}%
{\mathbb{R}}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\set}[1]{\left\{#1\right\}}
\newcommand{\mathsc}[1]{\mbox{\sc #1}}
\newcommand{\li}[1]{\langle#1\rangle}
\newcommand{\st}{\;s.t.\;}
\newcommand{\Real}{\R}
\newcommand{\Natural}{\N}
\newcommand{\Integer}{\Z}
% edit commands
\newcommand{\newedit}[2]{
\newcommand{#1}[2][default]{%
\ifthenelse{\boolean{technote@noedit}}{}{
\par\vspace{2mm}
\noindent
\begin{tabular}{|l|}\hline
\parbox{\linewidth-\tabcolsep*2}{{\bf #2:}\hfill\ifthenelse{\equal{##1}{default}}{}{##1}}\\\hline
\parbox{\linewidth-\tabcolsep*2}{\rule{0pt}{5mm}##2\rule[-2mm]{0pt}{2mm}}\\\hline
\end{tabular}
\par\vspace{2mm}
}
}
}
\newedit{\note}{Note}
\newedit{\comment}{Comment}
\newedit{\question}{Question}
\newedit{\content}{Content}
\newedit{\problem}{Problem}
\newcommand{\mnote}[1]{\marginpar{\scriptsize\it
\begin{minipage}[t]{0.8 in}
\raggedright #1
\end{minipage}}}
\newcommand{\Insert}[1]{\underline{#1}\marginpar{$|$}}
\newcommand{\Delete}[1]{\marginpar{$|$}
}
% lemma, theorem, etc.
\newtheorem{lemma}{Lemma}
\newtheorem{proposition}{Proposition}
\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}
\newtheorem{assumption}{Assumption}
\newtheorem{definition}{Definition}
\gdef\op|{\,|\;}
\gdef\op:{\,:\;}
\newcommand{\assign}{\leftarrow}
\newcommand{\inc}[1]{#1 \assign #1 + 1}
\newcommand{\isdef}{:=}
\newcommand{\ident}[1]{\mathit{#1}}
\def\newident#1{\expandafter\def\csname #1\endcsname{\ident{#1}}}
\newcommand{\eg}{{\it e.g.}}
\newcommand{\ie}{{\it i.e.}}
\newcommand{\apriori}{{\it apriori}}
\newcommand{\etal}{{\it et al.}}
\newcommand\ps@technote{%
\renewcommand\@oddhead{\theheader}%
\let\@evenhead\@oddhead
\renewcommand\@evenfoot
{\hfil\normalfont\textrm{\thepage}\hfil}%
\let\@oddfoot\@evenfoot
}

352
spec/consensus/consensus.md Normal file
View File

@@ -0,0 +1,352 @@
---
order: 1
---
# Byzantine Consensus Algorithm
## Terms
- The network is composed of optionally connected _nodes_. Nodes
directly connected to a particular node are called _peers_.
- The consensus process in deciding the next block (at some _height_
`H`) is composed of one or many _rounds_.
- `NewHeight`, `Propose`, `Prevote`, `Precommit`, and `Commit`
represent state machine states of a round. (aka `RoundStep` or
just "step").
- A node is said to be _at_ a given height, round, and step, or at
`(H,R,S)`, or at `(H,R)` in short to omit the step.
- To _prevote_ or _precommit_ something means to broadcast a [prevote
vote](https://godoc.org/github.com/tendermint/tendermint/types#Vote)
or [first precommit
vote](https://godoc.org/github.com/tendermint/tendermint/types#FirstPrecommit)
for something.
- A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R`
included in its [sign-bytes](../core/data_structures.md#vote).
- _+2/3_ is short for "more than 2/3"
- _1/3+_ is short for "1/3 or more"
- A set of +2/3 of prevotes for a particular block or `<nil>` at
`(H,R)` is called a _proof-of-lock-change_ or _PoLC_ for short.
## State Machine Overview
At each height of the blockchain a round-based protocol is run to
determine the next block. Each round is composed of three _steps_
(`Propose`, `Prevote`, and `Precommit`), along with two special steps
`Commit` and `NewHeight`.
In the optimal scenario, the order of steps is:
```md
NewHeight -> (Propose -> Prevote -> Precommit)+ -> Commit -> NewHeight ->...
```
The sequence `(Propose -> Prevote -> Precommit)` is called a _round_.
There may be more than one round required to commit a block at a given
height. Examples for why more rounds may be required include:
- The designated proposer was not online.
- The block proposed by the designated proposer was not valid.
- The block proposed by the designated proposer did not propagate
in time.
- The block proposed was valid, but +2/3 of prevotes for the proposed
block were not received in time for enough validator nodes by the
time they reached the `Precommit` step. Even though +2/3 of prevotes
are necessary to progress to the next step, at least one validator
may have voted `<nil>` or maliciously voted for something else.
- The block proposed was valid, and +2/3 of prevotes were received for
enough nodes, but +2/3 of precommits for the proposed block were not
received for enough validator nodes.
Some of these problems are resolved by moving onto the next round &
proposer. Others are resolved by increasing certain round timeout
parameters over each successive round.
## State Machine Diagram
```md
+-------------------------------------+
v |(Wait til `CommmitTime+timeoutCommit`)
+-----------+ +-----+-----+
+----------> | Propose +--------------+ | NewHeight |
| +-----------+ | +-----------+
| | ^
|(Else, after timeoutPrecommit) v |
+-----+-----+ +-----------+ |
| Precommit | <------------------------+ Prevote | |
+-----+-----+ +-----------+ |
|(When +2/3 Precommits for block found) |
v |
+--------------------------------------------------------------------+
| Commit |
| |
| * Set CommitTime = now; |
| * Wait for block, then stage/save/commit block; |
+--------------------------------------------------------------------+
```
# Background Gossip
A node may not have a corresponding validator private key, but it
nevertheless plays an active role in the consensus process by relaying
relevant meta-data, proposals, blocks, and votes to its peers. A node
that has the private keys of an active validator and is engaged in
signing votes is called a _validator-node_. All nodes (not just
validator-nodes) have an associated state (the current height, round,
and step) and work to make progress.
Between two nodes there exists a `Connection`, and multiplexed on top of
this connection are fairly throttled `Channel`s of information. An
epidemic gossip protocol is implemented among some of these channels to
bring peers up to speed on the most recent state of consensus. For
example,
- Nodes gossip `PartSet` parts of the current round's proposer's
proposed block. A LibSwift inspired algorithm is used to quickly
broadcast blocks across the gossip network.
- Nodes gossip prevote/precommit votes. A node `NODE_A` that is ahead
of `NODE_B` can send `NODE_B` prevotes or precommits for `NODE_B`'s
current (or future) round to enable it to progress forward.
- Nodes gossip prevotes for the proposed PoLC (proof-of-lock-change)
round if one is proposed.
- Nodes gossip to nodes lagging in blockchain height with block
[commits](https://godoc.org/github.com/tendermint/tendermint/types#Commit)
for older blocks.
- Nodes opportunistically gossip `ReceivedVote` messages to hint peers what
votes it already has.
- Nodes broadcast their current state to all neighboring peers. (but
is not gossiped further)
There's more, but let's not get ahead of ourselves here.
## Proposals
A proposal is signed and published by the designated proposer at each
round. The proposer is chosen by a deterministic and non-choking round
robin selection algorithm that selects proposers in proportion to their
voting power (see
[implementation](https://github.com/tendermint/tendermint/blob/master/types/validator_set.go)).
A proposal at `(H,R)` is composed of a block and an optional latest
`PoLC-Round < R` which is included iff the proposer knows of one. This
hints the network to allow nodes to unlock (when safe) to ensure the
liveness property.
## State Machine Spec
### Propose Step (height:H,round:R)
Upon entering `Propose`:
- The designated proposer proposes a block at `(H,R)`.
The `Propose` step ends:
- After `timeoutProposeR` after entering `Propose`. --> goto
`Prevote(H,R)`
- After receiving proposal block and all prevotes at `PoLC-Round`. -->
goto `Prevote(H,R)`
- After [common exit conditions](#common-exit-conditions)
### Prevote Step (height:H,round:R)
Upon entering `Prevote`, each validator broadcasts its prevote vote.
- First, if the validator is locked on a block since `LastLockRound`
but now has a PoLC for something else at round `PoLC-Round` where
`LastLockRound < PoLC-Round < R`, then it unlocks.
- If the validator is still locked on a block, it prevotes that.
- Else, if the proposed block from `Propose(H,R)` is good, it
prevotes that.
- Else, if the proposal is invalid or wasn't received on time, it
prevotes `<nil>`.
The `Prevote` step ends:
- After +2/3 prevotes for a particular block or `<nil>`. -->; goto
`Precommit(H,R)`
- After `timeoutPrevote` after receiving any +2/3 prevotes. --> goto
`Precommit(H,R)`
- After [common exit conditions](#common-exit-conditions)
### Precommit Step (height:H,round:R)
Upon entering `Precommit`, each validator broadcasts its precommit vote.
- If the validator has a PoLC at `(H,R)` for a particular block `B`, it
(re)locks (or changes lock to) and precommits `B` and sets
`LastLockRound = R`.
- Else, if the validator has a PoLC at `(H,R)` for `<nil>`, it unlocks
and precommits `<nil>`.
- Else, it keeps the lock unchanged and precommits `<nil>`.
A precommit for `<nil>` means "I didn’t see a PoLC for this round, but I
did get +2/3 prevotes and waited a bit".
The Precommit step ends:
- After +2/3 precommits for `<nil>`. --> goto `Propose(H,R+1)`
- After `timeoutPrecommit` after receiving any +2/3 precommits. --> goto
`Propose(H,R+1)`
- After [common exit conditions](#common-exit-conditions)
### Common exit conditions
- After +2/3 precommits for a particular block. --> goto
`Commit(H)`
- After any +2/3 prevotes received at `(H,R+x)`. --> goto
`Prevote(H,R+x)`
- After any +2/3 precommits received at `(H,R+x)`. --> goto
`Precommit(H,R+x)`
### Commit Step (height:H)
- Set `CommitTime = now()`
- Wait until block is received. --> goto `NewHeight(H+1)`
### NewHeight Step (height:H)
- Move `Precommits` to `LastCommit` and increment height.
- Set `StartTime = CommitTime+timeoutCommit`
- Wait until `StartTime` to receive straggler commits. --> goto
`Propose(H,0)`
## Proofs
### Proof of Safety
Assume that at most -1/3 of the voting power of validators is byzantine.
If a validator commits block `B` at round `R`, it's because it saw +2/3
of precommits at round `R`. This implies that 1/3+ of honest nodes are
still locked at round `R' > R`. These locked validators will remain
locked until they see a PoLC at `R' > R`, but this won't happen because
1/3+ are locked and honest, so at most -2/3 are available to vote for
anything other than `B`.
### Proof of Liveness
If 1/3+ honest validators are locked on two different blocks from
different rounds, a proposers' `PoLC-Round` will eventually cause nodes
locked from the earlier round to unlock. Eventually, the designated
proposer will be one that is aware of a PoLC at the later round. Also,
`timeoutProposalR` increments with round `R`, while the size of a
proposal are capped, so eventually the network is able to "fully gossip"
the whole proposal (e.g. the block & PoLC).
### Proof of Fork Accountability
Define the JSet (justification-vote-set) at height `H` of a validator
`V1` to be all the votes signed by the validator at `H` along with
justification PoLC prevotes for each lock change. For example, if `V1`
signed the following precommits: `Precommit(B1 @ round 0)`,
`Precommit(<nil> @ round 1)`, `Precommit(B2 @ round 4)` (note that no
precommits were signed for rounds 2 and 3, and that's ok),
`Precommit(B1 @ round 0)` must be justified by a PoLC at round 0, and
`Precommit(B2 @ round 4)` must be justified by a PoLC at round 4; but
the precommit for `<nil>` at round 1 is not a lock-change by definition
so the JSet for `V1` need not include any prevotes at round 1, 2, or 3
(unless `V1` happened to have prevoted for those rounds).
Further, define the JSet at height `H` of a set of validators `VSet` to
be the union of the JSets for each validator in `VSet`. For a given
commit by honest validators at round `R` for block `B` we can construct
a JSet to justify the commit for `B` at `R`. We say that a JSet
_justifies_ a commit at `(H,R)` if all the committers (validators in the
commit-set) are each justified in the JSet with no duplicitous vote
signatures (by the committers).
- **Lemma**: When a fork is detected by the existence of two
conflicting [commits](../core/data_structures.md#commit), the
union of the JSets for both commits (if they can be compiled) must
include double-signing by at least 1/3+ of the validator set.
**Proof**: The commit cannot be at the same round, because that
would immediately imply double-signing by 1/3+. Take the union of
the JSets of both commits. If there is no double-signing by at least
1/3+ of the validator set in the union, then no honest validator
could have precommitted any different block after the first commit.
Yet, +2/3 did. Reductio ad absurdum.
As a corollary, when there is a fork, an external process can determine
the blame by requiring each validator to justify all of its round votes.
Either we will find 1/3+ who cannot justify at least one of their votes,
and/or, we will find 1/3+ who had double-signed.
### Alternative algorithm
Alternatively, we can take the JSet of a commit to be the "full commit".
That is, if light clients and validators do not consider a block to be
committed unless the JSet of the commit is also known, then we get the
desirable property that if there ever is a fork (e.g. there are two
conflicting "full commits"), then 1/3+ of the validators are immediately
punishable for double-signing.
There are many ways to ensure that the gossip network efficiently share
the JSet of a commit. One solution is to add a new message type that
tells peers that this node has (or does not have) a +2/3 majority for B
(or) at (H,R), and a bitarray of which votes contributed towards that
majority. Peers can react by responding with appropriate votes.
We will implement such an algorithm for the next iteration of the
Tendermint consensus protocol.
Other potential improvements include adding more data in votes such as
the last known PoLC round that caused a lock change, and the last voted
round/step (or, we may require that validators not skip any votes). This
may make JSet verification/gossip logic easier to implement.
### Censorship Attacks
Due to the definition of a block
[commit](https://github.com/tendermint/tendermint/blob/master/docs/tendermint-core/validators.md), any 1/3+ coalition of
validators can halt the blockchain by not broadcasting their votes. Such
a coalition can also censor particular transactions by rejecting blocks
that include these transactions, though this would result in a
significant proportion of block proposals to be rejected, which would
slow down the rate of block commits of the blockchain, reducing its
utility and value. The malicious coalition might also broadcast votes in
a trickle so as to grind blockchain block commits to a near halt, or
engage in any combination of these attacks.
If a global active adversary were also involved, it can partition the
network in such a way that it may appear that the wrong subset of
validators were responsible for the slowdown. This is not just a
limitation of Tendermint, but rather a limitation of all consensus
protocols whose network is potentially controlled by an active
adversary.
### Overcoming Forks and Censorship Attacks
For these types of attacks, a subset of the validators through external
means should coordinate to sign a reorg-proposal that chooses a fork
(and any evidence thereof) and the initial subset of validators with
their signatures. Validators who sign such a reorg-proposal forego its
collateral on all other forks. Clients should verify the signatures on
the reorg-proposal, verify any evidence, and make a judgement or prompt
the end-user for a decision. For example, a phone wallet app may prompt
the user with a security warning, while a refrigerator may accept any
reorg-proposal signed by +1/2 of the original validators.
No non-synchronous Byzantine fault-tolerant algorithm can come to
consensus when 1/3+ of validators are dishonest, yet a fork assumes that
1/3+ of validators have already been dishonest by double-signing or
lock-changing without justification. So, signing the reorg-proposal is a
coordination problem that cannot be solved by any non-synchronous
protocol (i.e. automatically, and without making assumptions about the
reliability of the underlying network). It must be provided by means
external to the weakly-synchronous Tendermint consensus algorithm. For
now, we leave the problem of reorg-proposal coordination to human
coordination via internet media. Validators must take care to ensure
that there are no significant network partitions, to avoid situations
where two conflicting reorg-proposals are signed.
Assuming that the external coordination medium and protocol is robust,
it follows that forks are less of a concern than [censorship
attacks](#censorship-attacks).
### Canonical vs subjective commit
We distinguish between "canonical" and "subjective" commits. A subjective commit is what
each validator sees locally when they decide to commit a block. The canonical commit is
what is included by the proposer of the next block in the `LastCommit` field of
the block. This is what makes it canonical and ensures every validator agrees on the canonical commit,
even if it is different from the +2/3 votes a validator has seen, which caused the validator to
commit the respective block. Each block contains a canonical +2/3 commit for the previous
block.

43
spec/consensus/creating-proposal.md Normal file
View File

@@ -0,0 +1,43 @@
---
order: 2
---
# Creating a proposal
A block consists of a header, transactions, votes (the commit),
and a list of evidence of malfeasance (ie. signing conflicting votes).
We include no more than 1/10th of the maximum block size
(`ConsensusParams.Block.MaxBytes`) of evidence with each block.
## Reaping transactions from the mempool
When we reap transactions from the mempool, we calculate maximum data
size by subtracting maximum header size (`MaxHeaderBytes`), the maximum
amino overhead for a block (`MaxAminoOverheadForBlock`), the size of
the last commit (if present) and evidence (if present). While reaping
we account for amino overhead for each transaction.
```go
func MaxDataBytes(maxBytes int64, valsCount, evidenceCount int) int64 {
return maxBytes -
MaxOverheadForBlock -
MaxHeaderBytes -
int64(valsCount)*MaxVoteBytes -
int64(evidenceCount)*MaxEvidenceBytes
}
```
## Validating transactions in the mempool
Before we accept a transaction in the mempool, we check if it's size is no more
than {MaxDataSize}. {MaxDataSize} is calculated using the same formula as
above, except we subtract the max number of evidence, {MaxNum} by the maximum size of evidence
```go
func MaxDataBytesUnknownEvidence(maxBytes int64, valsCount int) int64 {
return maxBytes -
MaxOverheadForBlock -
MaxHeaderBytes -
(maxNumEvidence * MaxEvidenceBytes)
}
```

199
spec/consensus/evidence.md Normal file
View File

@@ -0,0 +1,199 @@
# Evidence
Evidence is an important component of Tendermint's security model. Whilst the core
consensus protocol provides correctness gaurantees for state machine replication
that can tolerate less than 1/3 failures, the evidence system looks to detect and
gossip byzantine faults whose combined power is greater than or equal to 1/3. It is worth noting that
the evidence system is designed purely to detect possible attacks, gossip them,
commit them on chain and inform the application running on top of Tendermint.
Evidence in itself does not punish "bad actors", this is left to the discretion
of the application. A common form of punishment is slashing where the validators
that were caught violating the protocol have all or a portion of their voting
power removed. Evidence, given the assumption that 1/3+ of the network is still
byzantine, is susceptible to censorship and should therefore be considered added
security on a "best effort" basis.
This document walks through the various forms of evidence, how they are detected,
gossiped, verified and committed.
> NOTE: Evidence here is internal to tendermint and should not be confused with
> application evidence
## Detection
### Equivocation
Equivocation is the most fundamental of byzantine faults. Simply put, to prevent
replication of state across all nodes, a validator tries to convince some subset
of nodes to commit one block whilst convincing another subset to commit a
different block. This is achieved by double voting (hence
`DuplicateVoteEvidence`). A successful duplicate vote attack requires greater
than 1/3 voting power and a (temporary) network partition between the aforementioned
subsets. This is because in consensus, votes are gossiped around. When a node
observes two conflicting votes from the same peer, it will use the two votes of
evidence and begin gossiping this evidence to other nodes. [Verification](#duplicatevoteevidence) is addressed further down.
```go
type DuplicateVoteEvidence struct {
VoteA Vote
VoteB Vote
// and abci specific fields
}
```
### Light Client Attacks
Light clients also comply with the 1/3+ security model, however, by using a
different, more lightweight verification method they are subject to a
different kind of 1/3+ attack whereby the byzantine validators could sign an
alternative light block that the light client will think is valid. Detection,
explained in greater detail
[here](../light-client/detection/detection_003_reviewed.md), involves comparison
with multiple other nodes in the hope that at least one is "honest". An "honest"
node will return a challenging light block for the light client to validate. If
this challenging light block also meets the
[validation criteria](../light-client/verification/verification_001_published.md)
then the light client sends the "forged" light block to the node.
[Verification](#lightclientattackevidence) is addressed further down.
```go
type LightClientAttackEvidence struct {
ConflictingBlock LightBlock
CommonHeight int64
// and abci specific fields
}
```
## Verification
If a node receives evidence, it will first try to verify it, then persist it.
Evidence of byzantine behavior should only be committed once (uniqueness) and
should be committed within a certain period from the point that it occurred
(timely). Timelines is defined by the `EvidenceParams`: `MaxAgeNumBlocks` and
`MaxAgeDuration`. In Proof of Stake chains where validators are bonded, evidence
age should be less than the unbonding period so validators still can be
punished. Given these two propoerties the following initial checks are made.
1. Has the evidence expired? This is done by taking the height of the `Vote`
within `DuplicateVoteEvidence` or `CommonHeight` within
`LightClientAttakEvidence`. The evidence height is then used to retrieve the
header and thus the time of the block that corresponds to the evidence. If
`CurrentHeight - MaxAgeNumBlocks > EvidenceHeight` && `CurrentTime -
MaxAgeDuration > EvidenceTime`, the evidence is considered expired and
ignored.
2. Has the evidence already been committed? The evidence pool tracks the hash of
all committed evidence and uses this to determine uniqueness. If a new
evidence has the same hash as a committed one, the new evidence will be
ignored.
### DuplicateVoteEvidence
Valid `DuplicateVoteEvidence` must adhere to the following rules:
- Validator Address, Height, Round and Type must be the same for both votes
- BlockID must be different for both votes (BlockID can be for a nil block)
- Validator must have been in the validator set at that height
- Vote signature must be correctly signed. This also uses `ChainID` so we know
that the fault occurred on this chain
### LightClientAttackEvidence
Valid Light Client Attack Evidence must adhere to the following rules:
- If the header of the light block is invalid, thus indicating a lunatic attack,
the node must check that they can use `verifySkipping` from their header at
the common height to the conflicting header
- If the header is valid, then the validator sets are the same and this is
either a form of equivocation or amnesia. We therefore check that 2/3 of the
validator set also signed the conflicting header.
- The nodes own header at the same height as the conflicting header must have a
different hash to the conflicting header.
- If the nodes latest header is less in height to the conflicting header, then
the node must check that the conflicting block has a time that is less than
this latest header (This is a forward lunatic attack).
## Gossiping
If a node verifies evidence it then broadcasts it to all peers, continously sending
the same evidence once every 10 seconds until the evidence is seen on chain or
expires.
## Commiting on Chain
Evidence takes strict priority over regular transactions, thus a block is filled
with evidence first and transactions take up the remainder of the space. To
mitigate the threat of an already punished node from spamming the network with
more evidence, the size of the evidence in a block can be capped by
`EvidenceParams.MaxBytes`. Nodes receiving blocks with evidence will validate
the evidence before sending `Prevote` and `Precommit` votes. The evidence pool
will usually cache verifications so that this process is much quicker.
## Sending Evidence to the Application
After evidence is committed, the block is then processed by the block executor
which delivers the evidence to the application via `EndBlock`. Evidence is
stripped of the actual proof, split up per faulty validator and only the
validator, height, time and evidence type is sent.
```proto
enum EvidenceType {
UNKNOWN = 0;
DUPLICATE_VOTE = 1;
LIGHT_CLIENT_ATTACK = 2;
}
message Evidence {
EvidenceType type = 1;
// The offending validator
Validator validator = 2 [(gogoproto.nullable) = false];
// The height when the offense occurred
int64 height = 3;
// The corresponding time where the offense occurred
google.protobuf.Timestamp time = 4 [
(gogoproto.nullable) = false, (gogoproto.stdtime) = true];
// Total voting power of the validator set in case the ABCI application does
// not store historical validators.
// https://github.com/tendermint/tendermint/issues/4581
int64 total_voting_power = 5;
}
```
`DuplicateVoteEvidence` and `LightClientAttackEvidence` are self-contained in
the sense that the evidence can be used to derive the `abci.Evidence` that is
sent to the application. Because of this, extra fields are necessary:
```go
type DuplicateVoteEvidence struct {
VoteA *Vote
VoteB *Vote
// abci specific information
TotalVotingPower int64
ValidatorPower int64
Timestamp time.Time
}
type LightClientAttackEvidence struct {
ConflictingBlock *LightBlock
CommonHeight int64
// abci specific information
ByzantineValidators []*Validator
TotalVotingPower int64
Timestamp time.Time
}
```
These ABCI specific fields don't affect validity of the evidence itself but must
be consistent amongst nodes and agreed upon on chain. If evidence with the
incorrect abci information is sent, a node will create new evidence from it and
replace the ABCI fields with the correct information.

9
spec/consensus/light-client/README.md Normal file
View File

@@ -0,0 +1,9 @@
---
order: 1
parent:
title: Light Client
order: false
---
# Tendermint Light Client Protocol
Deprecated, please see [light-client](../../light-client/README.md).

3
spec/consensus/light-client/accountability.md Normal file
View File

@@ -0,0 +1,3 @@
# Fork accountability
Deprecated, please see [light-client/accountability](../../light-client/accountability.md).

BIN
spec/consensus/light-client/assets/light-node-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 119 KiB

3
spec/consensus/light-client/detection.md Normal file
View File

@@ -0,0 +1,3 @@
# Detection
Deprecated, please see [light-client/detection](../../light-client/detection.md).

3
spec/consensus/light-client/verification.md Normal file
View File

@@ -0,0 +1,3 @@
# Core Verification
Deprecated, please see [light-client/accountability](../../light-client/verification.md).

20
spec/consensus/proposer-based-timestamp/README.md Normal file
View File

@@ -0,0 +1,20 @@
# Proposer-Based Timestamps
This section describes a version of the Tendermint consensus protocol,
which uses proposer-based timestamps.
## Contents
- [Proposer-Based Time][main] (entry point)
- [Part I - System Model and Properties][sysmodel]
- [Part II - Protocol Specification][algorithm]
- [TLA+ Specification][proposertla]
[algorithm]: ./pbts-algorithm_001_draft.md
[sysmodel]: ./pbts-sysmodel_001_draft.md
[main]: ./pbts_001_draft.md
[proposertla]: ./tla/TendermintPBT_001_draft.tla

163
spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md Normal file
View File

@@ -0,0 +1,163 @@
# Proposer-Based Time - Part II
## Updated Consensus Algorithm
### Outline
The algorithm in the [arXiv paper][arXiv] evaluates rules of the received messages without making explicit how these messages are received. In our solution, we will make some message filtering explicit. We will assume that there are message reception steps (where messages are received and possibly stored locally for later evaluation of rules) and processing steps (the latter roughly as described in a way similar to the pseudo code of the arXiv paper).
In contrast to the original algorithm the field `proposal` in the `PROPOSE` message is a pair `(v, time)`, of the proposed consensus value `v` and the proposed time `time`.
#### **[PBTS-RECEPTION-STEP.0]**
In the reception step at process `p` at local time `now_p`, upon receiving a message `m`:
- if the message `m` is of type `PROPOSE` and satisfies `now_p - PRECISION < m.time < now_p + PRECISION + MSGDELAY`, then mark the message as `timely`
> if `m` does not satisfy the constraint consider it `untimely`
#### **[PBTS-PROCESSING-STEP.0]**
In the processing step, based on the messages stored, the rules of the algorithms are
executed. Note that the processing step only operates on messages
for the current height. The consensus algorithm rules are defined by the following updates to arXiv paper.
#### New `StartRound`
There are two additions
- in case the proposer's local time is smaller than the time of the previous block, the proposer waits until this is not the case anymore (to ensure the block time is monotonically increasing)
- the proposer sends its time `now_p` as part of its proposal
We update the timeout for the `PROPOSE` step according to the following reasoning:
- If a correct proposer needs to wait to make sure its proposed time is larger than the `blockTime` of the previous block, then it sends by realtime `blockTime + ACCURACY` (By this time, its local clock must exceed `blockTime`)
- the receiver will receive a `PROPOSE` message by `blockTime + ACCURACY + MSGDELAY`
- the receiver's local clock will be `<= blockTime + 2 * ACCURACY + MSGDELAY`
- thus when the receiver `p` enters this round it can set its timeout to a value `waitingTime => blockTime + 2 * ACCURACY + MSGDELAY - now_p`
So we should set the timeout to `max(timeoutPropose(round_p), waitingTime)`.
> If, in the future, a block delay parameter `BLOCKDELAY` is introduced, this means
that the proposer should wait for `now_p > blockTime + BLOCKDELAY` before sending a `PROPOSE` message.
Also, `BLOCKDELAY` needs to be added to `waitingTime`.
#### **[PBTS-ALG-STARTROUND.0]**
```go
function StartRound(round) {
blockTime ← block time of block h_p - 1
waitingTime ← blockTime + 2 * ACCURACY + MSGDELAY - now_p
round_p ← round
step_p ← propose
if proposer(h_p, round_p) = p {
wait until now_p > blockTime // new wait condition
if validValue_p != nil {
proposal ← (validValue_p, now_p) // added "now_p"
}
else {
proposal ← (getValue(), now_p) // added "now_p"
}
broadcast ⟨PROPOSAL, h_p, round_p, proposal, validRound_p⟩
}
else {
schedule OnTimeoutPropose(h_p,round_p) to be executed after max(timeoutPropose(round_p), waitingTime)
}
}
```
#### New Rule Replacing Lines 22 - 27
- a validator prevotes for the consensus value `v` **and** the time `t`
- the code changes as the `PROPOSAL` message carries time (while `lockedValue` does not)
#### **[PBTS-ALG-UPON-PROP.0]**
```go
upon timely(⟨PROPOSAL, h_p, round_p, (v,t), −1⟩) from proposer(h_p, round_p) while step_p = propose do {
if valid(v) ∧ (lockedRound_p = −1 ∨ lockedValue_p = v) {
broadcast ⟨PREVOTE, h_p, round_p, id(v,t)⟩
}
else {
broadcast ⟨PREVOTE, h_p, round_p, nil⟩
}
step_p ← prevote
}
```
#### New Rule Replacing Lines 28 - 33
In case consensus is not reached in round 1, in `StartRound` the proposer of future rounds may propose the same value but with a different time.
Thus, the time `tprop` in the `PROPOSAL` message need not match the time `tvote` in the (old) `PREVOTE` messages.
A validator may send `PREVOTE` for the current round as long as the value `v` matches.
This gives the following rule:
#### **[PBTS-ALG-OLD-PREVOTE.0]**
```go
upon timely(⟨PROPOSAL, h_p, round_p, (v, tprop), vr⟩) from proposer(h_p, round_p) AND 2f + 1 ⟨PREVOTE, h_p, vr, id((v, tvote)⟩
while step_p = propose ∧ (vr ≥ 0 ∧ vr < round_p) do {
if valid(v) ∧ (lockedRound_p ≤ vr ∨ lockedValue_p = v) {
broadcast ⟨PREVOTE, h_p, roundp, id(v, tprop)⟩
}
else {
broadcast ⟨PREVOTE, hp, roundp, nil⟩
}
step_p ← prevote
}
```
#### New Rule Replacing Lines 36 - 43
- As above, in the following `(v,t)` is part of the message rather than `v`
- the stored values (i.e., `lockedValue`, `validValue`) do not contain the time
#### **[PBTS-ALG-NEW-PREVOTE.0]**
```go
upon timely(⟨PROPOSAL, h_p, round_p, (v,t), ∗⟩) from proposer(h_p, round_p) AND 2f + 1 ⟨PREVOTE, h_p, round_p, id(v,t)⟩ while valid(v) ∧ step_p ≥ prevote for the first time do {
if step_p = prevote {
lockedValue_p ← v
lockedRound_p ← round_p
broadcast ⟨PRECOMMIT, h_p, round_p, id(v,t))⟩
step_p ← precommit
}
validValue_p ← v
validRound_p ← round_p
}
```
#### New Rule Replacing Lines 49 - 54
- we decide on `v` as well as on the time from the proposal message
- here we do not care whether the proposal was received timely.
> In particular we need to take care of the case where the proposer is untimely to one correct validator only. We need to ensure that this validator decides if all decide.
#### **[PBTS-ALG-DECIDE.0]**
```go
upon ⟨PROPOSAL, h_p, r, (v,t), ∗⟩ from proposer(h_p, r) AND 2f + 1 ⟨PRECOMMIT, h_p, r, id(v,t)⟩ while decisionp[h_p] = nil do {
if valid(v) {
decision_p [h_p] = (v,t) // decide on time too
h_p ← h_p + 1
reset lockedRound_p , lockedValue_p, validRound_p and validValue_p to initial values and empty message log
StartRound(0)
}
}
```
**All other rules remains unchanged.**
Back to [main document][main].
[main]: ./pbts_001_draft.md
[arXiv]: https://arxiv.org/abs/1807.04938
[tlatender]: https://github.com/tendermint/spec/blob/master/rust-spec/tendermint-accountability/README.md
[bfttime]: https://github.com/tendermint/spec/blob/439a5bcacb5ef6ef1118566d7b0cd68fff3553d4/spec/consensus/bft-time.md
[lcspec]: https://github.com/tendermint/spec/blob/439a5bcacb5ef6ef1118566d7b0cd68fff3553d4/rust-spec/lightclient/README.md

198
spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md Normal file
View File

@@ -0,0 +1,198 @@
# Proposer-Based Time - Part I
## System Model
### Time and Clocks
#### **[PBTS-CLOCK-NEWTON.0]**
There is a reference Newtonian real-time `t` (UTC).
Every correct validator `V` maintains a synchronized clock `C_V` that ensures:
#### **[PBTS-CLOCK-PRECISION.0]**
There exists a system parameter `PRECISION` such that for any two correct validators `V` and `W`, and at any real-time `t`,
`|C_V(t) - C_W(t)| < PRECISION`
### Message Delays
We do not want to interfere with the Tendermint timing assumptions. We will postulate a timing restriction, which, if satisfied, ensures that liveness is preserved.
In general the local clock may drift from the global time. (It may progress faster, e.g., one second of clock time might take 1.005 seconds of real-time). As a result the local clock and the global clock may be measured in different time units. Usually, the message delay is measured in global clock time units. To estimate the correct local timeout precisely, we would need to estimate the clock time duration of a message delay taking into account the clock drift. For simplicity we ignore this, and directly postulate the message delay assumption in terms of local time.
#### **[PBTS-MSG-D.0]**
There exists a system parameter `MSGDELAY` for message end-to-end delays **counted in clock-time**.
> Observe that [PBTS-MSG-D.0] imposes constraints on message delays as well as on the clock.
#### **[PBTS-MSG-FAIR.0]**
The message end-to-end delay between a correct proposer and a correct validator (for `PROPOSE` messages) is less than `MSGDELAY`.
## Problem Statement
In this section we define the properties of Tendermint consensus (cf. the [arXiv paper][arXiv]) in this new system model.
#### **[PBTS-PROPOSE.0]**
A proposer proposes a pair `(v,t)` of consensus value `v` and time `t`.
> We then restrict the allowed decisions along the following lines:
#### **[PBTS-INV-AGREEMENT.0]**
[Agreement] No two correct validators decide on different values `v`.
#### **[PBTS-INV-TIME-VAL.0]**
[Time-Validity] If a correct validator decides on `t` then `t` is "OK" (we will formalize this below), even if up to `2f` validators are faulty.
However, the properties of Tendermint consensus are of more interest with respect to the blocks, that is, what is written into a block and when. We therefore, in the following, will give the safety and liveness properties from this block-centric viewpoint.
For this, observe that the time `t` decided at consensus height `k` will be written in the block of height `k+1`, and will be supported by `2f + 1` `PRECOMMIT` messages of the same consensus round `r`. The time written in the block, we will denote by `b.time` (to distinguish it from the term `bfttime` used for median-based time). For this, it is important to have the following consensus algorithm property:
#### **[PBTS-INV-TIME-AGR.0]**
[Time-Agreement] If two correct validators decide in the same round, then they decide on the same `t`.
#### **[PBTS-DECISION-ROUND.0]**
Note that the relation between consensus decisions, on the one hand, and blocks, on the other hand, is not immediate; in particular if we consider time: In the proposed solution,
as validators may decide in different rounds, they may decide on different times.
The proposer of the next block, may pick a commit (at least `2f + 1` `PRECOMMIT` messages from one round), and thus it picks a decision round that is going to become "canonic".
As a result, the proposer implicitly has a choice of one of the times that belong to rounds in which validators decided. Observe that this choice was implicitly the case already in the median-based `bfttime`.
However, as most consensus instances terminate within one round on the Cosmos hub, this is hardly ever observed in practice.
Finally, observe that the agreement ([Agreement] and [Time-Agreement]) properties are based on the Tendermint security model [TMBC-FM-2THIRDS.0] of more than 2/3 correct validators, while [Time-Validity] is based on more than 1/3 correct validators.
### SAFETY
Here we will provide specifications that relate local time to block time. However, since we do not assume (by now) that local time is linked to real-time, these specifications also do not provide a relation between block time and real-time. Such properties are given [later](#REAL-TIME-SAFETY).
For a correct validator `V`, let `beginConsensus(V,k)` be the local time when it sets its height to `k`, and let `endConsensus(V,k)` be the time when it sets its height to `k + 1`.
Let
- `beginConsensus(k)` be the minimum over `beginConsensus(V,k)`, and
- `last-beginConsensus(k)` be the maximum over `beginConsensus(V,k)`, and
- `endConsensus(k)` the maximum over `endConsensus(V,k)`
for all correct validators `V`.
> Observe that `beginConsensus(k) <= last-beginConsensus(k)` and if local clocks are monotonic, then `last-beginConsensus(k) <= endConsensus(k)`.
#### **[PBTS-CLOCK-GROW.0]**
We assume that during one consensus instance, local clocks are not set back, in particular for each correct validator `V` and each height `k`, we have `beginConsensus(V,k) < endConsensus(V,k)`.
#### **[PBTS-CONSENSUS-TIME-VALID.0]**
If
- there is a valid commit `c` for height `k`, and
- `c` contains a `PRECOMMIT` message by at least one correct validator,
then the time `b.time` in the block `b` that is signed by `c` satisfies
- `beginConsensus(k) - PRECISION <= b.time < endConsensus(k) + PRECISION + MSGDELAY`.
> [PBTS-CONSENSUS-TIME-VALID.0] is based on an analysis where the proposer is faulty (and does does not count towards `beginConsensus(k)` and `endConsensus(k)`), and we estimate the times at which correct validators receive and `accept` the `propose` message. If the proposer is correct we obtain
#### **[PBTS-CONSENSUS-LIVE-VALID-CORR-PROP.0]**
If the proposer of round 1 is correct, and
- [TMBC-FM-2THIRDS.0] holds for a block of height `k - 1`, and
- [PBTS-MSG-FAIR.0], and
- [PBTS-CLOCK-PRECISION.0], and
- [PBTS-CLOCK-GROW.0] (**TODO:** is that enough?)
then eventually (within bounded time) every correct validator decides in round 1.
#### **[PBTS-CONSENSUS-SAFE-VALID-CORR-PROP.0]**
If the proposer of round 1 is correct, and
- [TMBC-FM-2THIRDS.0] holds for a block of height `k - 1`, and
- [PBTS-MSG-FAIR.0], and
- [PBTS-CLOCK-PRECISION.0], and
- [PBTS-CLOCK-GROW.0] (**TODO:** is that enough?)
then `beginConsensus_k <= b.time <= last-beginConsensus_k`.
> For the above two properties we will assume that a correct proposer `v` sends its `PROPOSAL` at its local time `beginConsensus(v,k)`.
### LIVENESS
If
- [TMBC-FM-2THIRDS.0] holds for a block of height `k - 1`, and
- [PBTS-MSG-FAIR.0],
- [PBTS-CLOCK.0], and
- [PBTS-CLOCK-GROW.0] (**TODO:** is that enough?)
then eventually there is a valid commit `c` for height `k`.
### REAL-TIME SAFETY
> We want to give a property that can be exploited from the outside, that is, given a block with some time stored in it, what is the estimate at which real-time the block was generated. To do so, we need to link clock-time to real-time; which is not the case with [PBTS-CLOCK.0]. For this, we introduce the following assumption on the clocks:
#### **[PBTS-CLOCKSYNC-EXTERNAL.0]**
There is a system parameter `ACCURACY`, such that for all real-times `t` and all correct validators `V`,
- `| C_V(t) - t | < ACCURACY`.
> `ACCURACY` is not necessarily visible at the code level. The properties below just show that the smaller
its value, the closer the block time will be to real-time
#### **[PBTS-CONSENSUS-PTIME.0]**
LET `m` be a propose message. We consider the following two real-times `proposalTime(m)` and `propRecvTime(m)`:
- if the proposer is correct and sends `m` at time `t`, we write `proposalTime(m)` for real-time `t`.
- if first correct validator receives `m` at time `t`, we write `propRecvTime(m)` for real-time `t`.
#### **[PBTS-CONSENSUS-REALTIME-VALID.0]**
Let `b` be a block with a valid commit that contains at least one `precommit` message by a correct validator (and `proposalTime` is the time for the height/round `propose` message `m` that triggered the `precommit`). Then:
`propRecvTime(m) - ACCURACY - PRECISION < b.time < propRecvTime(m) + ACCURACY + PRECISION + MSGDELAY`
#### **[PBTS-CONSENSUS-REALTIME-VALID-CORR.0]**
Let `b` be a block with a valid commit that contains at least one `precommit` message by a correct validator (and `proposalTime` is the time for the height/round `propose` message `m` that triggered the `precommit`). Then, if the proposer is correct:
`proposalTime(m) - ACCURACY < b.time < proposalTime(m) + ACCURACY`
> by the algorithm at time `proposalTime(m)` the proposer fixes `m.time <- now_p(proposalTime(m))`
> "triggered the `PRECOMMIT`" implies that the data in `m` and `b` are "matching", that is, `m` proposed the values that are actually stored in `b`.
Back to [main document][main].
[main]: ./pbts_001_draft.md
[arXiv]: https://arxiv.org/abs/1807.04938
[tlatender]: https://github.com/tendermint/spec/blob/master/rust-spec/tendermint-accountability/README.md
[bfttime]: https://github.com/tendermint/spec/blob/439a5bcacb5ef6ef1118566d7b0cd68fff3553d4/spec/consensus/bft-time.md
[lcspec]: https://github.com/tendermint/spec/blob/439a5bcacb5ef6ef1118566d7b0cd68fff3553d4/rust-spec/lightclient/README.md
[algorithm]: ./pbts-algorithm_001_draft.md
[sysmodel]: ./pbts-sysmodel_001_draft.md

270
spec/consensus/proposer-based-timestamp/pbts_001_draft.md Normal file
View File

@@ -0,0 +1,270 @@
# Proposer-Based Time
## Current BFTTime
### Description
In Tendermint consensus, the first version of how time is computed and stored in a block works as follows:
- validators send their current local time as part of `precommit` messages
- upon collecting the `precommit` messages that the proposer uses to build a commit to be put in the next block, the proposer computes the `time` of the next block as the median (weighted over voting power) of the times in the `precommit` messages.
### Analysis
1. **Fault tolerance.** The computed median time is called [`bfttime`][bfttime] as it is indeed fault-tolerant: if **less than a third** of the validators is faulty (counted in voting power), it is guaranteed that the computed time lies between the minimum and the maximum times sent by correct validators.
1. **Effect of faulty validators.** If more than `1/2` of the voting power (which is in fact more than one third and less than two thirds of the voting power) is held by faulty validators, then the time is under total control of the faulty validators. (This is particularly challenging in the context of [lightclient][lcspec] security.)
1. **Proposer influence on block time.** The proposer of the next block has a degree of freedom in choosing the `bfttime`, since it computes the median time based on the timestamps from `precommit` messages sent by
`2f + 1` correct validators.
1. If there are `n` different timestamps in the `precommit` messages, the proposer can use any subset of timestamps that add up to `2f + 1`
of the voting power in order to compute the median.
1. If the validators decide in different rounds, the proposer can decide on which round the median computation is based.
1. **Liveness.** The liveness of the protocol:
1. does not depend on clock synchronization,
1. depends on bounded message delays.
1. **Relation to real time.** There is no clock synchronizaton, which implies that there is **no relation** between the computed block `time` and real time.
1. **Aggregate signatures.** As the `precommit` messages contain the local times, all these `precommit` messages typically differ in the time field, which **prevents** the use of aggregate signatures.
## Suggested Proposer-Based Time
### Outline
An alternative approach to time has been discussed: Rather than having the validators send the time in the `precommit` messages, the proposer in the consensus algorithm sends its time in the `propose` message, and the validators locally check whether the time is OK (by comparing to their local clock).
This proposed solution adds the requirement of having synchronized clocks, and other implicit assumptions.
### Comparison of the Suggested Method to the Old One
1. **Fault tolerance.** Maintained in the suggested protocol.
1. **Effect of faulty validators.** Eliminated in the suggested protocol,
that is, the block `time` can be corrupted only in the extreme case when
`>2/3` of the validators are faulty.
1. **Proposer influence on block time.** The proposer of the next block
has less freedom when choosing the block time.
1. This scenario is eliminated in the suggested protocol, provided that there are `<1/3` faulty validators.
1. This scenario is still there.
1. **Liveness.** The liveness of the suggested protocol:
1. depends on the introduced assumptions on synchronized clocks (see below),
1. still depends on the message delays (unavoidable).
1. **Relation to real time.** We formalize clock synchronization, and obtain a **well-defined relation** between the block `time` and real time.
1. **Aggregate signatures.** The `precommit` messages free of time, which **allows** for aggregate signatures.
### Protocol Overview
#### Proposed Time
We assume that the field `proposal` in the `PROPOSE` message is a pair `(v, time)`, of the proposed consensus value `v` and the proposed time `time`.
#### Reception Step
In the reception step at node `p` at local time `now_p`, upon receiving a message `m`:
- **if** the message `m` is of type `PROPOSE` and satisfies `now_p - PRECISION < m.time < now_p + PRECISION + MSGDELAY`, then mark the message as `timely`.
(`PRECISION` and `MSGDELAY` being system parameters, see [below](#safety-and-liveness))
> after the presentation in the dev session, we realized that different semantics for the reception step is closer aligned to the implementation. Instead of dropping propose messages, we keep all of them, and mark timely ones.
#### Processing Step
- Start round
<table>
<tr>
<th>arXiv paper</th>
<th>Proposer-based time</th>
</tr>
<tr>
<td>
```go
function StartRound(round) {
round_p ← round
step_p ← propose
if proposer(h_p, round_p) = p {
if validValue_p != nil {
proposal ← validValue_p
} else {
proposal ← getValue()
}
broadcast ⟨PROPOSAL, h_p, round_p, proposal, validRound_p⟩
} else {
schedule OnTimeoutPropose(h_p,round_p) to
be executed after timeoutPropose(round_p)
}
}
```
</td>
<td>
```go
function StartRound(round) {
round_p ← round
step_p ← propose
if proposer(h_p, round_p) = p {
// new wait condition
wait until now_p > block time of block h_p - 1
if validValue_p != nil {
// add "now_p"
proposal ← (validValue_p, now_p)
} else {
// add "now_p"
proposal ← (getValue(), now_p)
}
broadcast ⟨PROPOSAL, h_p, round_p, proposal, validRound_p⟩
} else {
schedule OnTimeoutPropose(h_p,round_p) to
be executed after timeoutPropose(round_p)
}
}
```
</td>
</tr>
</table>
- Rule on lines 28-35
<table>
<tr>
<th>arXiv paper</th>
<th>Proposer-based time</th>
</tr>
<tr>
<td>
```go
upon timely(⟨PROPOSAL, h_p, round_p, v, vr⟩)
from proposer(h_p, round_p)
AND 2f + 1 ⟨PREVOTE, h_p, vr, id(v)⟩
while step_p = propose ∧ (vr ≥ 0 ∧ vr < round_p) do {
if valid(v) ∧ (lockedRound_p ≤ vr ∨ lockedValue_p = v) {
broadcast ⟨PREVOTE, h_p, round_p, id(v)⟩
} else {
broadcast ⟨PREVOTE, hp, round_p, nil⟩
}
}
```
</td>
<td>
```go
upon timely(⟨PROPOSAL, h_p, round_p, (v, tprop), vr⟩)
from proposer(h_p, round_p)
AND 2f + 1 ⟨PREVOTE, h_p, vr, id(v, tvote)⟩
while step_p = propose ∧ (vr ≥ 0 ∧ vr < round_p) do {
if valid(v) ∧ (lockedRound_p ≤ vr ∨ lockedValue_p = v) {
// send hash of v and tprop in PREVOTE message
broadcast ⟨PREVOTE, h_p, round_p, id(v, tprop)⟩
} else {
broadcast ⟨PREVOTE, hp, round_p, nil⟩
}
}
```
</td>
</tr>
</table>
- Rule on lines 49-54
<table>
<tr>
<th>arXiv paper</th>
<th>Proposer-based time</th>
</tr>
<tr>
<td>
```go
upon ⟨PROPOSAL, h_p, r, v, ∗⟩ from proposer(h_p, r)
AND 2f + 1 ⟨PRECOMMIT, h_p, r, id(v)⟩
while decisionp[h_p] = nil do {
if valid(v) {
decision_p [h_p] = v
h_p ← h_p + 1
reset lockedRound_p , lockedValue_p, validRound_p and
validValue_p to initial values and empty message log
StartRound(0)
}
}
```
</td>
<td>
```go
upon ⟨PROPOSAL, h_p, r, (v,t), ∗⟩ from proposer(h_p, r)
AND 2f + 1 ⟨PRECOMMIT, h_p, r, id(v,t)⟩
while decisionp[h_p] = nil do {
if valid(v) {
// decide on time too
decision_p [h_p] = (v,t)
h_p ← h_p + 1
reset lockedRound_p , lockedValue_p, validRound_p and
validValue_p to initial values and empty message log
StartRound(0)
}
}
```
</td>
</tr>
</table>
- Other rules are extended in a similar way, or remain unchanged
### Property Overview
#### Safety and Liveness
For safety (Point 1, Point 2, Point 3i) and liveness (Point 4) we need
the following assumptions:
- There exists a system parameter `PRECISION` such that for any two correct validators `V` and `W`, and at any real-time `t`, their local times `C_V(t)` and `C_W(t)` differ by less than `PRECISION` time units,
i.e., `|C_V(t) - C_W(t)| < PRECISION`
- The message end-to-end delay between a correct proposer and a correct validator (for `PROPOSE` messages) is less than `MSGDELAY`.
#### Relation to Real-Time
For analyzing real-time safety (Point 5), we use a system parameter `ACCURACY`, such that for all real-times `t` and all correct validators `V`, we have `| C_V(t) - t | < ACCURACY`.
> `ACCURACY` is not necessarily visible at the code level. We might even view `ACCURACY` as variable over time. The smaller it is during a consensus instance, the closer the block time will be to real-time.
>
> Note that `PRECISION` and `MSGDELAY` show up in the code.
### Detailed Specification
This specification describes the changes needed to be done to the Tendermint consensus algorithm as described in the [arXiv paper][arXiv] and the simplified specification in [TLA+][tlatender], and makes precise the underlying assumptions and the required properties.
- [Part I - System Model and Properties][sysmodel]
- [Part II - Protocol specification][algorithm]
- [TLA+ Specification][proposertla]
[arXiv]: https://arxiv.org/abs/1807.04938
[tlatender]: https://github.com/tendermint/spec/blob/master/rust-spec/tendermint-accountability/README.md
[bfttime]: https://github.com/tendermint/spec/blob/439a5bcacb5ef6ef1118566d7b0cd68fff3553d4/spec/consensus/bft-time.md
[lcspec]: https://github.com/tendermint/spec/blob/439a5bcacb5ef6ef1118566d7b0cd68fff3553d4/rust-spec/lightclient/README.md
[algorithm]: ./pbts-algorithm_001_draft.md
[sysmodel]: ./pbts-sysmodel_001_draft.md
[main]: ./pbts_001_draft.md
[proposertla]: ./tla/TendermintPBT_001_draft.tla

597
spec/consensus/proposer-based-timestamp/tla/TendermintPBT_001_draft.tla Normal file
View File

@@ -0,0 +1,597 @@
-------------------- MODULE TendermintPBT_001_draft ---------------------------
(*
A TLA+ specification of a simplified Tendermint consensus, with added clocks
and proposer-based timestamps. This TLA+ specification extends and modifies
the Tendermint TLA+ specification for fork accountability:
https://github.com/tendermint/spec/blob/master/spec/light-client/accountability/TendermintAcc_004_draft.tla
* Version 1. A preliminary specification.
Zarko Milosevic, Igor Konnov, Informal Systems, 2019-2020.
Ilina Stoilkovska, Josef Widder, Informal Systems, 2021.
*)
EXTENDS Integers, FiniteSets
(********************* PROTOCOL PARAMETERS **********************************)
CONSTANTS
Corr, \* the set of correct processes
Faulty, \* the set of Byzantine processes, may be empty
N, \* the total number of processes: correct, defective, and Byzantine
T, \* an upper bound on the number of Byzantine processes
ValidValues, \* the set of valid values, proposed both by correct and faulty
InvalidValues, \* the set of invalid values, never proposed by the correct ones
MaxRound, \* the maximal round number
MaxTimestamp, \* the maximal value of the clock tick
Delay, \* message delay
Precision, \* clock precision: the maximal difference between two local clocks
Accuracy, \* clock accuracy: the maximal difference between a local clock and the real time
Proposer, \* the proposer function from 0..NRounds to 1..N
ClockDrift \* is there clock drift between the local clocks and the global clock
ASSUME(N = Cardinality(Corr \union Faulty))
(*************************** DEFINITIONS ************************************)
AllProcs == Corr \union Faulty \* the set of all processes
Rounds == 0..MaxRound \* the set of potential rounds
Timestamps == 0..MaxTimestamp \* the set of clock ticks
NilRound == -1 \* a special value to denote a nil round, outside of Rounds
NilTimestamp == -1 \* a special value to denote a nil timestamp, outside of Ticks
RoundsOrNil == Rounds \union {NilRound}
Values == ValidValues \union InvalidValues \* the set of all values
NilValue == "None" \* a special value for a nil round, outside of Values
Proposals == Values \X Timestamps
NilProposal == <<NilValue, NilTimestamp>>
ValuesOrNil == Values \union {NilValue}
Decisions == Values \X Timestamps \X Rounds
NilDecision == <<NilValue, NilTimestamp, NilRound>>
\* a value hash is modeled as identity
Id(v) == v
\* The validity predicate
IsValid(v) == v \in ValidValues
\* the two thresholds that are used in the algorithm
THRESHOLD1 == T + 1 \* at least one process is not faulty
THRESHOLD2 == 2 * T + 1 \* a quorum when having N > 3 * T
Min(S) == CHOOSE x \in S : \A y \in S : x <= y
Max(S) == CHOOSE x \in S : \A y \in S : y <= x
(********************* TYPE ANNOTATIONS FOR APALACHE **************************)
\* the operator for type annotations
a <: b == a
\* the type of message records
MT == [type |-> STRING, src |-> STRING, round |-> Int,
proposal |-> <<STRING, Int>>, validRound |-> Int, id |-> <<STRING, Int>>]
RP == <<STRING, MT>>
\* a type annotation for a message
AsMsg(m) == m <: MT
\* a type annotation for a set of messages
SetOfMsgs(S) == S <: {MT}
\* a type annotation for an empty set of messages
EmptyMsgSet == SetOfMsgs({})
SetOfRcvProp(S) == S <: {RP}
EmptyRcvProp == SetOfRcvProp({})
SetOfProc(S) == S <: {STRING}
EmptyProcSet == SetOfProc({})
(********************* PROTOCOL STATE VARIABLES ******************************)
VARIABLES
round, \* a process round number: Corr -> Rounds
localClock, \* a process local clock: Corr -> Ticks
realTime, \* a reference Newtonian real time
step, \* a process step: Corr -> { "PROPOSE", "PREVOTE", "PRECOMMIT", "DECIDED" }
decision, \* process decision: Corr -> ValuesOrNil
lockedValue, \* a locked value: Corr -> ValuesOrNil
lockedRound, \* a locked round: Corr -> RoundsOrNil
validValue, \* a valid value: Corr -> ValuesOrNil
validRound \* a valid round: Corr -> RoundsOrNil
\* book-keeping variables
VARIABLES
msgsPropose, \* PROPOSE messages broadcast in the system, Rounds -> Messages
msgsPrevote, \* PREVOTE messages broadcast in the system, Rounds -> Messages
msgsPrecommit, \* PRECOMMIT messages broadcast in the system, Rounds -> Messages
receivedTimelyProposal, \* used to keep track when a process receives a timely PROPOSAL message, {<<Corr, Messages>>}
inspectedProposal, \* used to keep track when a process tries to receive a message, [Rounds -> <<Corr, Messages>>]
evidence, \* the messages that were used by the correct processes to make transitions
action, \* we use this variable to see which action was taken
beginConsensus, \* the minimum of the local clocks in the initial state, Int
endConsensus, \* the local time when a decision is made, [Corr -> Int]
lastBeginConsensus, \* the maximum of the local clocks in the initial state, Int
proposalTime, \* the real time when a proposer proposes in a round, [Rounds -> Int]
proposalReceivedTime \* the real time when a correct process first receives a proposal message in a round, [Rounds -> Int]
(* to see a type invariant, check TendermintAccInv3 *)
\* a handy definition used in UNCHANGED
vars == <<round, step, decision, lockedValue, lockedRound,
validValue, validRound, evidence, msgsPropose, msgsPrevote, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal, action,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
(********************* PROTOCOL INITIALIZATION ******************************)
FaultyProposals(r) ==
SetOfMsgs([type: {"PROPOSAL"}, src: Faulty,
round: {r}, proposal: Proposals, validRound: RoundsOrNil])
AllFaultyProposals ==
SetOfMsgs([type: {"PROPOSAL"}, src: Faulty,
round: Rounds, proposal: Proposals, validRound: RoundsOrNil])
FaultyPrevotes(r) ==
SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: {r}, id: Proposals])
AllFaultyPrevotes ==
SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: Rounds, id: Proposals])
FaultyPrecommits(r) ==
SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: {r}, id: Proposals])
AllFaultyPrecommits ==
SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: Rounds, id: Proposals])
AllProposals ==
SetOfMsgs([type: {"PROPOSAL"}, src: AllProcs,
round: Rounds, proposal: Proposals, validRound: RoundsOrNil])
RoundProposals(r) ==
SetOfMsgs([type: {"PROPOSAL"}, src: AllProcs,
round: {r}, proposal: Proposals, validRound: RoundsOrNil])
BenignRoundsInMessages(msgfun) ==
\* the message function never contains a message for a wrong round
\A r \in Rounds:
\A m \in msgfun[r]:
r = m.round
\* The initial states of the protocol. Some faults can be in the system already.
Init ==
/\ round = [p \in Corr |-> 0]
/\ \/ /\ ~ClockDrift
/\ localClock \in [Corr -> 0..Accuracy]
\/ /\ ClockDrift
/\ localClock = [p \in Corr |-> 0]
/\ realTime = 0
/\ step = [p \in Corr |-> "PROPOSE"]
/\ decision = [p \in Corr |-> NilDecision]
/\ lockedValue = [p \in Corr |-> NilValue]
/\ lockedRound = [p \in Corr |-> NilRound]
/\ validValue = [p \in Corr |-> NilValue]
/\ validRound = [p \in Corr |-> NilRound]
/\ msgsPropose \in [Rounds -> SUBSET AllFaultyProposals]
/\ msgsPrevote \in [Rounds -> SUBSET AllFaultyPrevotes]
/\ msgsPrecommit \in [Rounds -> SUBSET AllFaultyPrecommits]
/\ receivedTimelyProposal = EmptyRcvProp
/\ inspectedProposal = [r \in Rounds |-> EmptyProcSet]
/\ BenignRoundsInMessages(msgsPropose)
/\ BenignRoundsInMessages(msgsPrevote)
/\ BenignRoundsInMessages(msgsPrecommit)
/\ evidence = EmptyMsgSet
/\ action' = "Init"
/\ beginConsensus = Min({localClock[p] : p \in Corr})
/\ endConsensus = [p \in Corr |-> NilTimestamp]
/\ lastBeginConsensus = Max({localClock[p] : p \in Corr})
/\ proposalTime = [r \in Rounds |-> NilTimestamp]
/\ proposalReceivedTime = [r \in Rounds |-> NilTimestamp]
(************************ MESSAGE PASSING ********************************)
BroadcastProposal(pSrc, pRound, pProposal, pValidRound) ==
LET newMsg ==
AsMsg([type |-> "PROPOSAL", src |-> pSrc, round |-> pRound,
proposal |-> pProposal, validRound |-> pValidRound])
IN
msgsPropose' = [msgsPropose EXCEPT ![pRound] = msgsPropose[pRound] \union {newMsg}]
BroadcastPrevote(pSrc, pRound, pId) ==
LET newMsg == AsMsg([type |-> "PREVOTE",
src |-> pSrc, round |-> pRound, id |-> pId])
IN
msgsPrevote' = [msgsPrevote EXCEPT ![pRound] = msgsPrevote[pRound] \union {newMsg}]
BroadcastPrecommit(pSrc, pRound, pId) ==
LET newMsg == AsMsg([type |-> "PRECOMMIT",
src |-> pSrc, round |-> pRound, id |-> pId])
IN
msgsPrecommit' = [msgsPrecommit EXCEPT ![pRound] = msgsPrecommit[pRound] \union {newMsg}]
(***************************** TIME **************************************)
\* [PBTS-CLOCK-PRECISION.0]
SynchronizedLocalClocks ==
\A p \in Corr : \A q \in Corr :
p /= q =>
\/ /\ localClock[p] >= localClock[q]
/\ localClock[p] - localClock[q] < Precision
\/ /\ localClock[p] < localClock[q]
/\ localClock[q] - localClock[p] < Precision
\* [PBTS-PROPOSE.0]
Proposal(v, t) ==
<<v, t>>
\* [PBTS-DECISION-ROUND.0]
Decision(v, t, r) ==
<<v, t, r>>
(**************** MESSAGE PROCESSING TRANSITIONS *************************)
\* lines 12-13
StartRound(p, r) ==
/\ step[p] /= "DECIDED" \* a decided process does not participate in consensus
/\ round' = [round EXCEPT ![p] = r]
/\ step' = [step EXCEPT ![p] = "PROPOSE"]
\* lines 14-19, a proposal may be sent later
InsertProposal(p) ==
LET r == round[p] IN
/\ p = Proposer[r]
/\ step[p] = "PROPOSE"
\* if the proposer is sending a proposal, then there are no other proposals
\* by the correct processes for the same round
/\ \A m \in msgsPropose[r]: m.src /= p
/\ \E v \in ValidValues:
LET proposal == IF validValue[p] /= NilValue
THEN Proposal(validValue[p], localClock[p])
ELSE Proposal(v, localClock[p]) IN
/\ BroadcastProposal(p, round[p], proposal, validRound[p])
/\ proposalTime' = [proposalTime EXCEPT ![r] = realTime]
/\ UNCHANGED <<evidence, round, decision, lockedValue, lockedRound,
validValue, step, validRound, msgsPrevote, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalReceivedTime>>
/\ action' = "InsertProposal"
\* a new action used to filter messages that are not on time
\* [PBTS-RECEPTION-STEP.0]
ReceiveProposal(p) ==
\E v \in Values, t \in Timestamps:
/\ LET r == round[p] IN
LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> Proposal(v, t), validRound |-> NilRound]) IN
/\ msg \in msgsPropose[round[p]]
/\ p \notin inspectedProposal[r]
/\ <<p, msg>> \notin receivedTimelyProposal
/\ inspectedProposal' = [inspectedProposal EXCEPT ![r] = @ \union {p}]
/\ \/ /\ localClock[p] - Precision < t
/\ t < localClock[p] + Precision + Delay
/\ receivedTimelyProposal' = receivedTimelyProposal \union {<<p, msg>>}
/\ \/ /\ proposalReceivedTime[r] = NilTimestamp
/\ proposalReceivedTime' = [proposalReceivedTime EXCEPT ![r] = realTime]
\/ /\ proposalReceivedTime[r] /= NilTimestamp
/\ UNCHANGED proposalReceivedTime
\/ /\ \/ localClock[p] - Precision >= t
\/ t >= localClock[p] + Precision + Delay
/\ UNCHANGED <<receivedTimelyProposal, proposalReceivedTime>>
/\ UNCHANGED <<round, step, decision, lockedValue, lockedRound,
validValue, validRound, evidence, msgsPropose, msgsPrevote, msgsPrecommit,
localClock, realTime, beginConsensus, endConsensus, lastBeginConsensus, proposalTime>>
/\ action' = "ReceiveProposal"
\* lines 22-27
UponProposalInPropose(p) ==
\E v \in Values, t \in Timestamps:
/\ step[p] = "PROPOSE" (* line 22 *)
/\ LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> Proposal(v, t), validRound |-> NilRound]) IN
/\ <<p, msg>> \in receivedTimelyProposal \* updated line 22
/\ evidence' = {msg} \union evidence
/\ LET mid == (* line 23 *)
IF IsValid(v) /\ (lockedRound[p] = NilRound \/ lockedValue[p] = v)
THEN Id(Proposal(v, t))
ELSE NilProposal
IN
BroadcastPrevote(p, round[p], mid) \* lines 24-26
/\ step' = [step EXCEPT ![p] = "PREVOTE"]
/\ UNCHANGED <<round, decision, lockedValue, lockedRound,
validValue, validRound, msgsPropose, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "UponProposalInPropose"
\* lines 28-33
\* [PBTS-ALG-OLD-PREVOTE.0]
UponProposalInProposeAndPrevote(p) ==
\E v \in Values, t1 \in Timestamps, t2 \in Timestamps, vr \in Rounds:
/\ step[p] = "PROPOSE" /\ 0 <= vr /\ vr < round[p] \* line 28, the while part
/\ LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> Proposal(v, t1), validRound |-> vr])
IN
/\ <<p, msg>> \in receivedTimelyProposal \* updated line 28
/\ LET PV == { m \in msgsPrevote[vr]: m.id = Id(Proposal(v, t2)) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 28
/\ evidence' = PV \union {msg} \union evidence
/\ LET mid == (* line 29 *)
IF IsValid(v) /\ (lockedRound[p] <= vr \/ lockedValue[p] = v)
THEN Id(Proposal(v, t1))
ELSE NilProposal
IN
BroadcastPrevote(p, round[p], mid) \* lines 24-26
/\ step' = [step EXCEPT ![p] = "PREVOTE"]
/\ UNCHANGED <<round, decision, lockedValue, lockedRound,
validValue, validRound, msgsPropose, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "UponProposalInProposeAndPrevote"
\* lines 34-35 + lines 61-64 (onTimeoutPrevote)
UponQuorumOfPrevotesAny(p) ==
/\ step[p] = "PREVOTE" \* line 34 and 61
/\ \E MyEvidence \in SUBSET msgsPrevote[round[p]]:
\* find the unique voters in the evidence
LET Voters == { m.src: m \in MyEvidence } IN
\* compare the number of the unique voters against the threshold
/\ Cardinality(Voters) >= THRESHOLD2 \* line 34
/\ evidence' = MyEvidence \union evidence
/\ BroadcastPrecommit(p, round[p], NilProposal)
/\ step' = [step EXCEPT ![p] = "PRECOMMIT"]
/\ UNCHANGED <<round, decision, lockedValue, lockedRound,
validValue, validRound, msgsPropose, msgsPrevote,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "UponQuorumOfPrevotesAny"
\* lines 36-46
\* [PBTS-ALG-NEW-PREVOTE.0]
UponProposalInPrevoteOrCommitAndPrevote(p) ==
\E v \in ValidValues, t \in Timestamps, vr \in RoundsOrNil:
/\ step[p] \in {"PREVOTE", "PRECOMMIT"} \* line 36
/\ LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> Proposal(v, t), validRound |-> vr]) IN
/\ <<p, msg>> \in receivedTimelyProposal \* updated line 36
/\ LET PV == { m \in msgsPrevote[round[p]]: m.id = Id(Proposal(v, t)) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 36
/\ evidence' = PV \union {msg} \union evidence
/\ IF step[p] = "PREVOTE"
THEN \* lines 38-41:
/\ lockedValue' = [lockedValue EXCEPT ![p] = v]
/\ lockedRound' = [lockedRound EXCEPT ![p] = round[p]]
/\ BroadcastPrecommit(p, round[p], Id(Proposal(v, t)))
/\ step' = [step EXCEPT ![p] = "PRECOMMIT"]
ELSE
UNCHANGED <<lockedValue, lockedRound, msgsPrecommit, step>>
\* lines 42-43
/\ validValue' = [validValue EXCEPT ![p] = v]
/\ validRound' = [validRound EXCEPT ![p] = round[p]]
/\ UNCHANGED <<round, decision, msgsPropose, msgsPrevote,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "UponProposalInPrevoteOrCommitAndPrevote"
\* lines 47-48 + 65-67 (onTimeoutPrecommit)
UponQuorumOfPrecommitsAny(p) ==
/\ \E MyEvidence \in SUBSET msgsPrecommit[round[p]]:
\* find the unique committers in the evidence
LET Committers == { m.src: m \in MyEvidence } IN
\* compare the number of the unique committers against the threshold
/\ Cardinality(Committers) >= THRESHOLD2 \* line 47
/\ evidence' = MyEvidence \union evidence
/\ round[p] + 1 \in Rounds
/\ StartRound(p, round[p] + 1)
/\ UNCHANGED <<decision, lockedValue, lockedRound, validValue,
validRound, msgsPropose, msgsPrevote, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "UponQuorumOfPrecommitsAny"
\* lines 49-54
\* [PBTS-ALG-DECIDE.0]
UponProposalInPrecommitNoDecision(p) ==
/\ decision[p] = NilDecision \* line 49
/\ \E v \in ValidValues, t \in Timestamps (* line 50*) , r \in Rounds, vr \in RoundsOrNil:
/\ LET msg == AsMsg([type |-> "PROPOSAL", src |-> Proposer[r],
round |-> r, proposal |-> Proposal(v, t), validRound |-> vr]) IN
/\ msg \in msgsPropose[r] \* line 49
/\ p \in inspectedProposal[r]
/\ LET PV == { m \in msgsPrecommit[r]: m.id = Id(Proposal(v, t)) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 49
/\ evidence' = PV \union {msg} \union evidence
/\ decision' = [decision EXCEPT ![p] = Decision(v, t, round[p])] \* update the decision, line 51
\* The original algorithm does not have 'DECIDED', but it increments the height.
\* We introduced 'DECIDED' here to prevent the process from changing its decision.
/\ endConsensus' = [endConsensus EXCEPT ![p] = localClock[p]]
/\ step' = [step EXCEPT ![p] = "DECIDED"]
/\ UNCHANGED <<round, lockedValue, lockedRound, validValue,
validRound, msgsPropose, msgsPrevote, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "UponProposalInPrecommitNoDecision"
\* the actions below are not essential for safety, but added for completeness
\* lines 20-21 + 57-60
OnTimeoutPropose(p) ==
/\ step[p] = "PROPOSE"
/\ p /= Proposer[round[p]]
/\ BroadcastPrevote(p, round[p], NilProposal)
/\ step' = [step EXCEPT ![p] = "PREVOTE"]
/\ UNCHANGED <<round, lockedValue, lockedRound, validValue,
validRound, decision, evidence, msgsPropose, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "OnTimeoutPropose"
\* lines 44-46
OnQuorumOfNilPrevotes(p) ==
/\ step[p] = "PREVOTE"
/\ LET PV == { m \in msgsPrevote[round[p]]: m.id = Id(NilProposal) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 36
/\ evidence' = PV \union evidence
/\ BroadcastPrecommit(p, round[p], Id(NilProposal))
/\ step' = [step EXCEPT ![p] = "PRECOMMIT"]
/\ UNCHANGED <<round, lockedValue, lockedRound, validValue,
validRound, decision, msgsPropose, msgsPrevote,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "OnQuorumOfNilPrevotes"
\* lines 55-56
OnRoundCatchup(p) ==
\E r \in {rr \in Rounds: rr > round[p]}:
LET RoundMsgs == msgsPropose[r] \union msgsPrevote[r] \union msgsPrecommit[r] IN
\E MyEvidence \in SUBSET RoundMsgs:
LET Faster == { m.src: m \in MyEvidence } IN
/\ Cardinality(Faster) >= THRESHOLD1
/\ evidence' = MyEvidence \union evidence
/\ StartRound(p, r)
/\ UNCHANGED <<decision, lockedValue, lockedRound, validValue,
validRound, msgsPropose, msgsPrevote, msgsPrecommit,
localClock, realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "OnRoundCatchup"
(********************* PROTOCOL TRANSITIONS ******************************)
\* advance the global clock
AdvanceRealTime ==
/\ realTime < MaxTimestamp
/\ realTime' = realTime + 1
/\ \/ /\ ~ClockDrift
/\ localClock' = [p \in Corr |-> localClock[p] + 1]
\/ /\ ClockDrift
/\ UNCHANGED localClock
/\ UNCHANGED <<round, step, decision, lockedValue, lockedRound,
validValue, validRound, evidence, msgsPropose, msgsPrevote, msgsPrecommit,
localClock, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "AdvanceRealTime"
\* advance the local clock of node p
AdvanceLocalClock(p) ==
/\ localClock[p] < MaxTimestamp
/\ localClock' = [localClock EXCEPT ![p] = @ + 1]
/\ UNCHANGED <<round, step, decision, lockedValue, lockedRound,
validValue, validRound, evidence, msgsPropose, msgsPrevote, msgsPrecommit,
realTime, receivedTimelyProposal, inspectedProposal,
beginConsensus, endConsensus, lastBeginConsensus, proposalTime, proposalReceivedTime>>
/\ action' = "AdvanceLocalClock"
\* process timely messages
MessageProcessing(p) ==
\* start round
\/ InsertProposal(p)
\* reception step
\/ ReceiveProposal(p)
\* processing step
\/ UponProposalInPropose(p)
\/ UponProposalInProposeAndPrevote(p)
\/ UponQuorumOfPrevotesAny(p)
\/ UponProposalInPrevoteOrCommitAndPrevote(p)
\/ UponQuorumOfPrecommitsAny(p)
\/ UponProposalInPrecommitNoDecision(p)
\* the actions below are not essential for safety, but added for completeness
\/ OnTimeoutPropose(p)
\/ OnQuorumOfNilPrevotes(p)
\/ OnRoundCatchup(p)
(*
* A system transition. In this specificatiom, the system may eventually deadlock,
* e.g., when all processes decide. This is expected behavior, as we focus on safety.
*)
Next ==
\/ AdvanceRealTime
\/ /\ ClockDrift
/\ \E p \in Corr: AdvanceLocalClock(p)
\/ /\ SynchronizedLocalClocks
/\ \E p \in Corr: MessageProcessing(p)
-----------------------------------------------------------------------------
(*************************** INVARIANTS *************************************)
\* [PBTS-INV-AGREEMENT.0]
AgreementOnValue ==
\A p, q \in Corr:
/\ decision[p] /= NilDecision
/\ decision[q] /= NilDecision
=> \E v \in ValidValues, t1 \in Timestamps, t2 \in Timestamps, r1 \in Rounds, r2 \in Rounds :
/\ decision[p] = Decision(v, t1, r1)
/\ decision[q] = Decision(v, t2, r2)
\* [PBTS-INV-TIME-AGR.0]
AgreementOnTime ==
\A p, q \in Corr:
\A v1 \in ValidValues, v2 \in ValidValues, t1 \in Timestamps, t2 \in Timestamps, r \in Rounds :
/\ decision[p] = Decision(v1, t1, r)
/\ decision[q] = Decision(v2, t2, r)
=> t1 = t2
\* [PBTS-CONSENSUS-TIME-VALID.0]
ConsensusTimeValid ==
\A p \in Corr, t \in Timestamps :
\* if a process decides on v and t
(\E v \in ValidValues, r \in Rounds : decision[p] = Decision(v, t, r))
\* then
=> /\ beginConsensus - Precision <= t
/\ t < endConsensus[p] + Precision + Delay
\* [PBTS-CONSENSUS-SAFE-VALID-CORR-PROP.0]
ConsensusSafeValidCorrProp ==
\A v \in ValidValues, t \in Timestamps :
\* if the proposer in the first round is correct
(/\ Proposer[0] \in Corr
\* and there exists a process that decided on v, t
/\ \E p \in Corr, r \in Rounds : decision[p] = Decision(v, t, r))
\* then t is between the minimal and maximal initial local time
=> /\ beginConsensus <= t
/\ t <= lastBeginConsensus
\* [PBTS-CONSENSUS-REALTIME-VALID-CORR.0]
ConsensusRealTimeValidCorr ==
\A t \in Timestamps, r \in Rounds :
(/\ \E p \in Corr, v \in ValidValues : decision[p] = Decision(v, t, r)
/\ proposalTime[r] /= NilTimestamp)
=> /\ proposalTime[r] - Accuracy < t
/\ t < proposalTime[r] + Accuracy
\* [PBTS-CONSENSUS-REALTIME-VALID.0]
ConsensusRealTimeValid ==
\A t \in Timestamps, r \in Rounds :
(\E p \in Corr, v \in ValidValues : decision[p] = Decision(v, t, r))
=> /\ proposalReceivedTime[r] - Accuracy - Precision < t
/\ t < proposalReceivedTime[r] + Accuracy + Precision + Delay
\* [PBTS-MSG-FAIR.0]
BoundedDelay ==
\A r \in Rounds :
(/\ proposalTime[r] /= NilTimestamp
/\ proposalTime[r] + Delay < realTime)
=> inspectedProposal[r] = Corr
\* [PBTS-CONSENSUS-TIME-LIVE.0]
ConsensusTimeLive ==
\A r \in Rounds, p \in Corr :
(/\ proposalTime[r] /= NilTimestamp
/\ proposalTime[r] + Delay < realTime
/\ Proposer[r] \in Corr
/\ round[p] <= r)
=> \E msg \in RoundProposals(r) : <<p, msg>> \in receivedTimelyProposal
\* a conjunction of all invariants
Inv ==
/\ AgreementOnValue
/\ AgreementOnTime
/\ ConsensusTimeValid
/\ ConsensusSafeValidCorrProp
/\ ConsensusRealTimeValid
/\ ConsensusRealTimeValidCorr
/\ BoundedDelay
Liveness ==
ConsensusTimeLive
=============================================================================

323
spec/consensus/proposer-selection.md Normal file
View File

@@ -0,0 +1,323 @@
---
order: 3
---
# Proposer Selection Procedure
This document specifies the Proposer Selection Procedure that is used in Tendermint to choose a round proposer.
As Tendermint is “leader-based protocol”, the proposer selection is critical for its correct functioning.
At a given block height, the proposer selection algorithm runs with the same validator set at each round .
Between heights, an updated validator set may be specified by the application as part of the ABCIResponses' EndBlock.
## Requirements for Proposer Selection
This sections covers the requirements with Rx being mandatory and Ox optional requirements.
The following requirements must be met by the Proposer Selection procedure:
### R1: Determinism
Given a validator set `V`, and two honest validators `p` and `q`, for each height `h` and each round `r` the following must hold:
`proposer_p(h,r) = proposer_q(h,r)`
where `proposer_p(h,r)` is the proposer returned by the Proposer Selection Procedure at process `p`, at height `h` and round `r`.
### R2: Fairness
Given a validator set with total voting power P and a sequence S of elections. In any sub-sequence of S with length C*P, a validator v must be elected as proposer P/VP(v) times, i.e. with frequency:
f(v) ~ VP(v) / P
where C is a tolerance factor for validator set changes with following values:
- C == 1 if there are no validator set changes
- C ~ k when there are validator changes
*[this needs more work]*
## Basic Algorithm
At its core, the proposer selection procedure uses a weighted round-robin algorithm.
A model that gives a good intuition on how/ why the selection algorithm works and it is fair is that of a priority queue. The validators move ahead in this queue according to their voting power (the higher the voting power the faster a validator moves towards the head of the queue). When the algorithm runs the following happens:
- all validators move "ahead" according to their powers: for each validator, increase the priority by the voting power
- first in the queue becomes the proposer: select the validator with highest priority
- move the proposer back in the queue: decrease the proposer's priority by the total voting power
Notation:
- vset - the validator set
- n - the number of validators
- VP(i) - voting power of validator i
- A(i) - accumulated priority for validator i
- P - total voting power of set
- avg - average of all validator priorities
- prop - proposer
Simple view at the Selection Algorithm:
```md
def ProposerSelection (vset):
// compute priorities and elect proposer
for each validator i in vset:
A(i) += VP(i)
prop = max(A)
A(prop) -= P
```
## Stable Set
Consider the validator set:
Validator | p1 | p2
----------|----|---
VP | 1 | 3
Assuming no validator changes, the following table shows the proposer priority computation over a few runs. Four runs of the selection procedure are shown, starting with the 5th the same values are computed.
Each row shows the priority queue and the process place in it. The proposer is the closest to the head, the rightmost validator. As priorities are updated, the validators move right in the queue. The proposer moves left as its priority is reduced after election.
| Priority Run | -2 | -1 | 0 | 1 | 2 | 3 | 4 | 5 | Alg step |
|----------------|----|----|-------|----|-------|----|----|----|------------------|
| | | | p1,p2 | | | | | | Initialized to 0 |
| run 1 | | | | p1 | | p2 | | | A(i)+=VP(i) |
| | | p2 | | p1 | | | | | A(p2)-= P |
| run 2 | | | | | p1,p2 | | | | A(i)+=VP(i) |
| | p1 | | | | p2 | | | | A(p1)-= P |
| run 3 | | p1 | | | | | | p2 | A(i)+=VP(i) |
| | | p1 | | p2 | | | | | A(p2)-= P |
| run 4 | | | p1 | | | | p2 | | A(i)+=VP(i) |
| | | | p1,p2 | | | | | | A(p2)-= P |
It can be shown that:
- At the end of each run k+1 the sum of the priorities is the same as at end of run k. If a new set's priorities are initialized to 0 then the sum of priorities will be 0 at each run while there are no changes.
- The max distance between priorites is (n-1) *P.*[formal proof not finished]*
## Validator Set Changes
Between proposer selection runs the validator set may change. Some changes have implications on the proposer election.
### Voting Power Change
Consider again the earlier example and assume that the voting power of p1 is changed to 4:
Validator | p1 | p2
----------|----|---
VP | 4 | 3
Let's also assume that before this change the proposer priorites were as shown in first row (last run). As it can be seen, the selection could run again, without changes, as before.
| Priority Run | -2 | -1 | 0 | 1 | 2 | Comment |
|----------------|----|----|---|----|----|-------------------|
| last run | | p2 | | p1 | | __update VP(p1)__ |
| next run | | | | | p2 | A(i)+=VP(i) |
| | p1 | | | | p2 | A(p1)-= P |
However, when a validator changes power from a high to a low value, some other validator remain far back in the queue for a long time. This scenario is considered again in the Proposer Priority Range section.
As before:
- At the end of each run k+1 the sum of the priorities is the same as at run k.
- The max distance between priorites is (n-1) * P.
### Validator Removal
Consider a new example with set:
Validator | p1 | p2 | p3
----------|----|----|---
VP | 1 | 2 | 3
Let's assume that after the last run the proposer priorities were as shown in first row with their sum being 0. After p2 is removed, at the end of next proposer selection run (penultimate row) the sum of priorities is -2 (minus the priority of the removed process).
The procedure could continue without modifications. However, after a sufficiently large number of modifications in validator set, the priority values would migrate towards maximum or minimum allowed values causing truncations due to overflow detection.
For this reason, the selection procedure adds another __new step__ that centers the current priority values such that the priority sum remains close to 0.
| Priority Run | -3 | -2 | -1 | 0 | 1 | 2 | 4 | Comment |
|----------------|----|----|----|---|----|----|---|-----------------------|
| last run | p3 | | | | p1 | p2 | | __remove p2__ |
| nextrun | | | | | | | | |
| __new step__ | | p3 | | | | p1 | | A(i) -= avg, avg = -1 |
| | | | | | p3 | p1 | | A(i)+=VP(i) |
| | | | p1 | | p3 | | | A(p1)-= P |
The modified selection algorithm is:
```md
def ProposerSelection (vset):
// center priorities around zero
avg = sum(A(i) for i in vset)/len(vset)
for each validator i in vset:
A(i) -= avg
// compute priorities and elect proposer
for each validator i in vset:
A(i) += VP(i)
prop = max(A)
A(prop) -= P
```
Observations:
- The sum of priorities is now close to 0. Due to integer division the sum is an integer in (-n, n), where n is the number of validators.
### New Validator
When a new validator is added, same problem as the one described for removal appears, the sum of priorities in the new set is not zero. This is fixed with the centering step introduced above.
One other issue that needs to be addressed is the following. A validator V that has just been elected is moved to the end of the queue. If the validator set is large and/ or other validators have significantly higher power, V will have to wait many runs to be elected. If V removes and re-adds itself to the set, it would make a significant (albeit unfair) "jump" ahead in the queue.
In order to prevent this, when a new validator is added, its initial priority is set to:
```md
A(V) = -1.125 * P
```
where P is the total voting power of the set including V.
Curent implementation uses the penalty factor of 1.125 because it provides a small punishment that is efficient to calculate. See [here](https://github.com/tendermint/tendermint/pull/2785#discussion_r235038971) for more details.
If we consider the validator set where p3 has just been added:
Validator | p1 | p2 | p3
----------|----|----|---
VP | 1 | 3 | 8
then p3 will start with proposer priority:
```md
A(p3) = -1.125 * (1 + 3 + 8) ~ -13
```
Note that since current computation uses integer division there is penalty loss when sum of the voting power is less than 8.
In the next run, p3 will still be ahead in the queue, elected as proposer and moved back in the queue.
| Priority Run | -13 | -9 | -5 | -2 | -1 | 0 | 1 | 2 | 5 | 6 | 7 | Alg step |
|----------------|-----|----|----|----|----|---|---|----|----|----|----|-----------------------|
| last run | | | | p2 | | | | p1 | | | | __add p3__ |
| | p3 | | | p2 | | | | p1 | | | | A(p3) = -4 |
| next run | | p3 | | | | | | p2 | | p1 | | A(i) -= avg, avg = -4 |
| | | | | | p3 | | | | p2 | | p1 | A(i)+=VP(i) |
| | | | p1 | | p3 | | | | p2 | | | A(p1)-=P |
## Proposer Priority Range
With the introduction of centering, some interesting cases occur. Low power validators that bind early in a set that includes high power validator(s) benefit from subsequent additions to the set. This is because these early validators run through more right shift operations during centering, operations that increase their priority.
As an example, consider the set where p2 is added after p1, with priority -1.125 * 80k = -90k. After the selection procedure runs once:
Validator | p1 | p2 | Comment
----------|------|------|------------------
VP | 80k | 10 |
A | 0 | -90k | __added p2__
A | -45k | 45k | __run selection__
Then execute the following steps:
1. Add a new validator p3:
Validator | p1 | p2 | p3
----------|-----|----|---
VP | 80k | 10 | 10
2. Run selection once. The notation '..p'/'p..' means very small deviations compared to column priority.
| Priority Run | -90k.. | -60k | -45k | -15k | 0 | 45k | 75k | 155k | Comment |
|---------------|--------|------|------|------|---|-----|-----|------|--------------|
| last run | p3 | | p2 | | | p1 | | | __added p3__ |
| next run
| *right_shift*| | p3 | | p2 | | | p1 | | A(i) -= avg,avg=-30k
| | | ..p3| | ..p2| | | | p1 | A(i)+=VP(i)
| | | ..p3| | ..p2| | | p1.. | | A(p1)-=P, P=80k+20
3. Remove p1 and run selection once:
Validator | p3 | p2 | Comment
----------|--------|-------|------------------
VP | 10 | 10 |
A | -60k | -15k |
A | -22.5k | 22.5k | __run selection__
At this point, while the total voting power is 20, the distance between priorities is 45k. It will take 4500 runs for p3 to catch up with p2.
In order to prevent these types of scenarios, the selection algorithm performs scaling of priorities such that the difference between min and max values is smaller than two times the total voting power.
The modified selection algorithm is:
```md
def ProposerSelection (vset):
// scale the priority values
diff = max(A)-min(A)
threshold = 2 * P
if diff > threshold:
scale = diff/threshold
for each validator i in vset:
A(i) = A(i)/scale
// center priorities around zero
avg = sum(A(i) for i in vset)/len(vset)
for each validator i in vset:
A(i) -= avg
// compute priorities and elect proposer
for each validator i in vset:
A(i) += VP(i)
prop = max(A)
A(prop) -= P
```
Observations:
- With this modification, the maximum distance between priorites becomes 2 * P.
Note also that even during steady state the priority range may increase beyond 2 * P. The scaling introduced here helps to keep the range bounded.
## Wrinkles
### Validator Power Overflow Conditions
The validator voting power is a positive number stored as an int64. When a validator is added the `1.125 * P` computation must not overflow. As a consequence the code handling validator updates (add and update) checks for overflow conditions making sure the total voting power is never larger than the largest int64 `MAX`, with the property that `1.125 * MAX` is still in the bounds of int64. Fatal error is return when overflow condition is detected.
### Proposer Priority Overflow/ Underflow Handling
The proposer priority is stored as an int64. The selection algorithm performs additions and subtractions to these values and in the case of overflows and underflows it limits the values to:
```go
MaxInt64 = 1 << 63 - 1
MinInt64 = -1 << 63
```
## Requirement Fulfillment Claims
__[R1]__
The proposer algorithm is deterministic giving consistent results across executions with same transactions and validator set modifications.
[WIP - needs more detail]
__[R2]__
Given a set of processes with the total voting power P, during a sequence of elections of length P, the number of times any process is selected as proposer is equal to its voting power. The sequence of the P proposers then repeats. If we consider the validator set:
Validator | p1 | p2
----------|----|---
VP | 1 | 3
With no other changes to the validator set, the current implementation of proposer selection generates the sequence:
`p2, p1, p2, p2, p2, p1, p2, p2,...` or [`p2, p1, p2, p2`]*
A sequence that starts with any circular permutation of the [`p2, p1, p2, p2`] sub-sequence would also provide the same degree of fairness. In fact these circular permutations show in the sliding window (over the generated sequence) of size equal to the length of the sub-sequence.
Assigning priorities to each validator based on the voting power and updating them at each run ensures the fairness of the proposer selection. In addition, every time a validator is elected as proposer its priority is decreased with the total voting power.
Intuitively, a process v jumps ahead in the queue at most (max(A) - min(A))/VP(v) times until it reaches the head and is elected. The frequency is then:
```md
f(v) ~ VP(v)/(max(A)-min(A)) = 1/k * VP(v)/P
```
For current implementation, this means v should be proposer at least VP(v) times out of k * P runs, with scaling factor k=2.

32
spec/consensus/readme.md Normal file
View File

@@ -0,0 +1,32 @@
---
order: 1
parent:
title: Consensus
order: 4
---
# Consensus
Specification of the Tendermint consensus protocol.
## Contents
- [Consensus Paper](./consensus-paper) - Latex paper on
[arxiv](https://arxiv.org/abs/1807.04938) describing the
core Tendermint consensus state machine with proofs of safety and termination.
- [BFT Time](./bft-time.md) - How the timestamp in a Tendermint
block header is computed in a Byzantine Fault Tolerant manner
- [Creating Proposal](./creating-proposal.md) - How a proposer
creates a block proposal for consensus
- [Light Client Protocol](./light-client) - A protocol for light weight consensus
verification and syncing to the latest state
- [Signing](./signing.md) - Rules for cryptographic signatures
produced by validators.
- [Write Ahead Log](./wal.md) - Write ahead log used by the
consensus state machine to recover from crashes.
The protocol used to gossip consensus messages between peers, which is critical
for liveness, is described in the [reactors section](../reactors/consensus/consensus.md).
There is also a [stale markdown description](consensus.md) of the consensus state machine
(TODO update this).

229
spec/consensus/signing.md Normal file
View File

@@ -0,0 +1,229 @@
# Validator Signing
Here we specify the rules for validating a proposal and vote before signing.
First we include some general notes on validating data structures common to both types.
We then provide specific validation rules for each. Finally, we include validation rules to prevent double-sigining.
## SignedMsgType
The `SignedMsgType` is a single byte that refers to the type of the message
being signed. It is defined in Go as follows:
```go
// SignedMsgType is a type of signed message in the consensus.
type SignedMsgType byte
const (
// Votes
PrevoteType SignedMsgType = 0x01
PrecommitType SignedMsgType = 0x02
// Proposals
ProposalType SignedMsgType = 0x20
)
```
All signed messages must correspond to one of these types.
## Timestamp
Timestamp validation is subtle and there are currently no bounds placed on the
timestamp included in a proposal or vote. It is expected that validators will honestly
report their local clock time. The median of all timestamps
included in a commit is used as the timestamp for the next block height.
Timestamps are expected to be strictly monotonic for a given validator, though
this is not currently enforced.
## ChainID
ChainID is an unstructured string with a max length of 50-bytes.
In the future, the ChainID may become structured, and may take on longer lengths.
For now, it is recommended that signers be configured for a particular ChainID,
and to only sign votes and proposals corresponding to that ChainID.
## BlockID
BlockID is the structure used to represent the block:
```go
type BlockID struct {
Hash []byte
PartsHeader PartSetHeader
}
type PartSetHeader struct {
Hash []byte
Total int
}
```
To be included in a valid vote or proposal, BlockID must either represent a `nil` block, or a complete one.
We introduce two methods, `BlockID.IsZero()` and `BlockID.IsComplete()` for these cases, respectively.
`BlockID.IsZero()` returns true for BlockID `b` if each of the following
are true:
```go
b.Hash == nil
b.PartsHeader.Total == 0
b.PartsHeader.Hash == nil
```
`BlockID.IsComplete()` returns true for BlockID `b` if each of the following
are true:
```go
len(b.Hash) == 32
b.PartsHeader.Total > 0
len(b.PartsHeader.Hash) == 32
```
## Proposals
The structure of a proposal for signing looks like:
```go
type CanonicalProposal struct {
Type SignedMsgType // type alias for byte
Height int64 `binary:"fixed64"`
Round int64 `binary:"fixed64"`
POLRound int64 `binary:"fixed64"`
BlockID BlockID
Timestamp time.Time
ChainID string
}
```
A proposal is valid if each of the following lines evaluates to true for proposal `p`:
```go
p.Type == 0x20
p.Height > 0
p.Round >= 0
p.POLRound >= -1
p.BlockID.IsComplete()
```
In other words, a proposal is valid for signing if it contains the type of a Proposal
(0x20), has a positive, non-zero height, a
non-negative round, a POLRound not less than -1, and a complete BlockID.
## Votes
The structure of a vote for signing looks like:
```go
type CanonicalVote struct {
Type SignedMsgType // type alias for byte
Height int64 `binary:"fixed64"`
Round int64 `binary:"fixed64"`
BlockID BlockID
Timestamp time.Time
ChainID string
}
```
A vote is valid if each of the following lines evaluates to true for vote `v`:
```go
v.Type == 0x1 || v.Type == 0x2
v.Height > 0
v.Round >= 0
v.BlockID.IsZero() || v.BlockID.IsComplete()
```
In other words, a vote is valid for signing if it contains the type of a Prevote
or Precommit (0x1 or 0x2, respectively), has a positive, non-zero height, a
non-negative round, and an empty or valid BlockID.
## Invalid Votes and Proposals
Votes and proposals which do not satisfy the above rules are considered invalid.
Peers gossipping invalid votes and proposals may be disconnected from other peers on the network.
Note, however, that there is not currently any explicit mechanism to punish validators signing votes or proposals that fail
these basic validation rules.
## Double Signing
Signers must be careful not to sign conflicting messages, also known as "double signing" or "equivocating".
Tendermint has mechanisms to publish evidence of validators that signed conflicting votes, so they can be punished
by the application. Note Tendermint does not currently handle evidence of conflciting proposals, though it may in the future.
### State
To prevent such double signing, signers must track the height, round, and type of the last message signed.
Assume the signer keeps the following state, `s`:
```go
type LastSigned struct {
Height int64
Round int64
Type SignedMsgType // byte
}
```
After signing a vote or proposal `m`, the signer sets:
```go
s.Height = m.Height
s.Round = m.Round
s.Type = m.Type
```
### Proposals
A signer should only sign a proposal `p` if any of the following lines are true:
```go
p.Height > s.Height
p.Height == s.Height && p.Round > s.Round
```
In other words, a proposal should only be signed if it's at a higher height, or a higher round for the same height.
Once a proposal or vote has been signed for a given height and round, a proposal should never be signed for the same height and round.
### Votes
A signer should only sign a vote `v` if any of the following lines are true:
```go
v.Height > s.Height
v.Height == s.Height && v.Round > s.Round
v.Height == s.Height && v.Round == s.Round && v.Step == 0x1 && s.Step == 0x20
v.Height == s.Height && v.Round == s.Round && v.Step == 0x2 && s.Step != 0x2
```
In other words, a vote should only be signed if it's:
- at a higher height
- at a higher round for the same height
- a prevote for the same height and round where we haven't signed a prevote or precommit (but have signed a proposal)
- a precommit for the same height and round where we haven't signed a precommit (but have signed a proposal and/or a prevote)
This means that once a validator signs a prevote for a given height and round, the only other message it can sign for that height and round is a precommit.
And once a validator signs a precommit for a given height and round, it must not sign any other message for that same height and round.
Note this includes votes for `nil`, ie. where `BlockID.IsZero()` is true. If a
signer has already signed a vote where `BlockID.IsZero()` is true, it cannot
sign another vote with the same type for the same height and round where
`BlockID.IsComplete()` is true. Thus only a single vote of a particular type
(ie. 0x01 or 0x02) can be signed for the same height and round.
### Other Rules
According to the rules of Tendermint consensus, once a validator precommits for
a block, they become "locked" on that block, which means they can't prevote for
another block unless they see sufficient justification (ie. a polka from a
higher round). For more details, see the [consensus
spec](https://arxiv.org/abs/1807.04938).
Violating this rule is known as "amnesia". In contrast to equivocation,
which is easy to detect, amnesia is difficult to detect without access to votes
from all the validators, as this is what constitutes the justification for
"unlocking". Hence, amnesia is not punished within the protocol, and cannot
easily be prevented by a signer. If enough validators simultaneously commit an
amnesia attack, they may cause a fork of the blockchain, at which point an
off-chain protocol must be engaged to collect votes from all the validators and
determine who misbehaved. For more details, see [fork
detection](https://github.com/tendermint/tendermint/pull/3978).

32
spec/consensus/wal.md Normal file
View File

@@ -0,0 +1,32 @@
# WAL
Consensus module writes every message to the WAL (write-ahead log).
It also issues fsync syscall through
[File#Sync](https://golang.org/pkg/os/#File.Sync) for messages signed by this
node (to prevent double signing).
Under the hood, it uses
[autofile.Group](https://godoc.org/github.com/tendermint/tmlibs/autofile#Group),
which rotates files when those get too big (> 10MB).
The total maximum size is 1GB. We only need the latest block and the block before it,
but if the former is dragging on across many rounds, we want all those rounds.
## Replay
Consensus module will replay all the messages of the last height written to WAL
before a crash (if such occurs).
The private validator may try to sign messages during replay because it runs
somewhat autonomously and does not know about replay process.
For example, if we got all the way to precommit in the WAL and then crash,
after we replay the proposal message, the private validator will try to sign a
prevote. But it will fail. That's ok because we’ll see the prevote later in the
WAL. Then it will go to precommit, and that time it will work because the
private validator contains the `LastSignBytes` and then we’ll replay the
precommit from the WAL.
Make sure to read about [WAL corruption](https://github.com/tendermint/tendermint/blob/master/docs/tendermint-core/running-in-production.md#wal-corruption)
and recovery strategies.

456
spec/core/data_structures.md Normal file
View File

@@ -0,0 +1,456 @@
# Data Structures
Here we describe the data structures in the Tendermint blockchain and the rules for validating them.
The Tendermint blockchains consists of a short list of data types:
- [Data Structures](#data-structures)
- [Block](#block)
- [Execution](#execution)
- [Header](#header)
- [Version](#version)
- [BlockID](#blockid)
- [PartSetHeader](#partsetheader)
- [Part](#part)
- [Time](#time)
- [Data](#data)
- [Commit](#commit)
- [CommitSig](#commitsig)
- [BlockIDFlag](#blockidflag)
- [Vote](#vote)
- [CanonicalVote](#canonicalvote)
- [Proposal](#proposal)
- [SignedMsgType](#signedmsgtype)
- [Signature](#signature)
- [EvidenceList](#evidencelist)
- [Evidence](#evidence)
- [DuplicateVoteEvidence](#duplicatevoteevidence)
- [LightClientAttackEvidence](#lightclientattackevidence)
- [LightBlock](#lightblock)
- [SignedHeader](#signedheader)
- [ValidatorSet](#validatorset)
- [Validator](#validator)
- [Address](#address)
- [ConsensusParams](#consensusparams)
- [BlockParams](#blockparams)
- [EvidenceParams](#evidenceparams)
- [ValidatorParams](#validatorparams)
- [VersionParams](#versionparams)
- [Proof](#proof)
## Block
A block consists of a header, transactions, votes (the commit),
and a list of evidence of malfeasance (ie. signing conflicting votes).
| Name | Type | Description | Validation |
|--------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------|
| Header | [Header](#header) | Header corresponding to the block. This field contains information used throughout consensus and other areas of the protocol. To find out what it contains, visit [header] (#header) | Must adhere to the validation rules of [header](#header) |
| Data | [Data](#data) | Data contains a list of transactions. The contents of the transaction is unknown to Tendermint. | This field can be empty or populated, but no validation is performed. Applications can perform validation on individual transactions prior to block creation using [checkTx](../abci/abci.md#checktx).
| Evidence | [EvidenceList](#evidence_list) | Evidence contains a list of infractions committed by validators. | Can be empty, but when populated the validations rules from [evidenceList](#evidence_list) apply |
| LastCommit | [Commit](#commit) | `LastCommit` includes one vote for every validator. All votes must either be for the previous block, nil or absent. If a vote is for the previous block it must have a valid signature from the corresponding validator. The sum of the voting power of the validators that voted must be greater than 2/3 of the total voting power of the complete validator set. The number of votes in a commit is limited to 10000 (see `types.MaxVotesCount`). | Must be empty for the initial height and must adhere to the validation rules of [commit](#commit). |
## Execution
Once a block is validated, it can be executed against the state.
The state follows this recursive equation:
```go
state(initialHeight) = InitialState
state(h+1) <- Execute(state(h), ABCIApp, block(h))
```
where `InitialState` includes the initial consensus parameters and validator set,
and `ABCIApp` is an ABCI application that can return results and changes to the validator
set (TODO). Execute is defined as:
```go
func Execute(s State, app ABCIApp, block Block) State {
// Fuction ApplyBlock executes block of transactions against the app and returns the new root hash of the app state,
// modifications to the validator set and the changes of the consensus parameters.
AppHash, ValidatorChanges, ConsensusParamChanges := app.ApplyBlock(block)
nextConsensusParams := UpdateConsensusParams(state.ConsensusParams, ConsensusParamChanges)
return State{
ChainID: state.ChainID,
InitialHeight: state.InitialHeight,
LastResults: abciResponses.DeliverTxResults,
AppHash: AppHash,
InitialHeight: state.InitialHeight,
LastValidators: state.Validators,
Validators: state.NextValidators,
NextValidators: UpdateValidators(state.NextValidators, ValidatorChanges),
ConsensusParams: nextConsensusParams,
Version: {
Consensus: {
AppVersion: nextConsensusParams.Version.AppVersion,
},
},
}
}
```
Validating a new block is first done prior to the `prevote`, `precommit` & `finalizeCommit` stages.
The steps to validate a new block are:
- Check the validity rules of the block and its fields.
- Check the versions (Block & App) are the same as in local state.
- Check the chainID's match.
- Check the height is correct.
- Check the `LastBlockID` corresponds to BlockID currently in state.
- Check the hashes in the header match those in state.
- Verify the LastCommit against state, this step is skipped for the initial height.
- This is where checking the signatures correspond to the correct block will be made.
- Make sure the proposer is part of the validator set.
- Validate bock time.
- Make sure the new blocks time is after the previous blocks time.
- Calculate the medianTime and check it against the blocks time.
- If the blocks height is the initial height then check if it matches the genesis time.
- Validate the evidence in the block. Note: Evidence can be empty
## Header
A block header contains metadata about the block and about the consensus, as well as commitments to
the data in the current block, the previous block, and the results returned by the application:
| Name | Type | Description | Validation |
|-------------------|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Version | [Version](#version) | Version defines the application and protocol version being used. | Must adhere to the validation rules of [Version](#version) |
| ChainID | String | ChainID is the ID of the chain. This must be unique to your chain. | ChainID must be less than 50 bytes. |
| Height | uint64 | Height is the height for this header. | Must be > 0, >= initialHeight, and == previous Height+1 |
| Time | [Time](#time) | The timestamp is equal to the weighted median of validators present in the last commit. Read more on time in the [BFT-time section](../consensus/bft-time.md). Note: the timestamp of a vote must be greater by at least one millisecond than that of the block being voted on. | Time must be >= previous header timestamp + consensus parameters TimeIotaMs. The timestamp of the first block must be equal to the genesis time (since there's no votes to compute the median). |
| LastBlockID | [BlockID](#blockid) | BlockID of the previous block. | Must adhere to the validation rules of [blockID](#blockid). The first block has `block.Header.LastBlockID == BlockID{}`. |
| LastCommitHash | slice of bytes (`[]byte`) | MerkleRoot of the lastCommit's signatures. The signatures represent the validators that committed to the last block. The first block has an empty slices of bytes for the hash. | Must be of length 32 |
| DataHash | slice of bytes (`[]byte`) | MerkleRoot of the hash of transactions. **Note**: The transactions are hashed before being included in the merkle tree, the leaves of the Merkle tree are the hashes, not the transactions themselves. | Must be of length 32 |
| ValidatorHash | slice of bytes (`[]byte`) | MerkleRoot of the current validator set. The validators are first sorted by voting power (descending), then by address (ascending) prior to computing the MerkleRoot. | Must be of length 32 |
| NextValidatorHash | slice of bytes (`[]byte`) | MerkleRoot of the next validator set. The validators are first sorted by voting power (descending), then by address (ascending) prior to computing the MerkleRoot. | Must be of length 32 |
| ConsensusHash | slice of bytes (`[]byte`) | Hash of the protobuf encoded consensus parameters. | Must be of length 32 |
| AppHash | slice of bytes (`[]byte`) | Arbitrary byte array returned by the application after executing and commiting the previous block. It serves as the basis for validating any merkle proofs that comes from the ABCI application and represents the state of the actual application rather than the state of the blockchain itself. The first block's `block.Header.AppHash` is given by `ResponseInitChain.app_hash`. | This hash is determined by the application, Tendermint can not perform validation on it. |
| LastResultHash | slice of bytes (`[]byte`) | `LastResultsHash` is the root hash of a Merkle tree built from `ResponseDeliverTx` responses (`Log`,`Info`, `Codespace` and `Events` fields are ignored). | Must be of length 32. The first block has `block.Header.ResultsHash == MerkleRoot(nil)`, i.e. the hash of an empty input, for RFC-6962 conformance. |
| EvidenceHash | slice of bytes (`[]byte`) | MerkleRoot of the evidence of Byzantine behaviour included in this block. | Must be of length 32 |
| ProposerAddress | slice of bytes (`[]byte`) | Address of the original proposer of the block. Validator must be in the current validatorSet. | Must be of length 20 |
## Version
NOTE: that this is more specifically the consensus version and doesn't include information like the
P2P Version. (TODO: we should write a comprehensive document about
versioning that this can refer to)
| Name | type | Description | Validation |
|-------|--------|-----------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|
| Block | uint64 | This number represents the version of the block protocol and must be the same throughout an operational network | Must be equal to protocol version being used in a network (`block.Version.Block == state.Version.Consensus.Block`) |
| App | uint64 | App version is decided on by the application. Read [here](../abci/abci.md#info) | `block.Version.App == state.Version.Consensus.App` |
## BlockID
The `BlockID` contains two distinct Merkle roots of the block. The `BlockID` includes these two hashes, as well as the number of parts (ie. `len(MakeParts(block))`)
| Name | Type | Description | Validation |
|---------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------|
| Hash | slice of bytes (`[]byte`) | MerkleRoot of all the fields in the header (ie. `MerkleRoot(header)`. | hash must be of length 32 |
| PartSetHeader | [PartSetHeader](#PartSetHeader) | Used for secure gossiping of the block during consensus, is the MerkleRoot of the complete serialized block cut into parts (ie. `MerkleRoot(MakeParts(block))`). | Must adhere to the validation rules of [PartSetHeader](#PartSetHeader) |
See [MerkleRoot](./encoding.md#MerkleRoot) for details.
## PartSetHeader
| Name | Type | Description | Validation |
|-------|---------------------------|-----------------------------------|----------------------|
| Total | int32 | Total amount of parts for a block | Must be > 0 |
| Hash | slice of bytes (`[]byte`) | MerkleRoot of a serialized block | Must be of length 32 |
## Part
Part defines a part of a block. In Tendermint blocks are broken into `parts` for gossip.
| Name | Type | Description | Validation |
|-------|-----------------|-----------------------------------|----------------------|
| index | int32 | Total amount of parts for a block | Must be > 0 |
| bytes | bytes | MerkleRoot of a serialized block | Must be of length 32 |
| proof | [Proof](#proof) | MerkleRoot of a serialized block | Must be of length 32 |
## Time
Tendermint uses the [Google.Protobuf.Timestamp](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp)
format, which uses two integers, one 64 bit integer for Seconds and a 32 bit integer for Nanoseconds.
## Data
Data is just a wrapper for a list of transactions, where transactions are arbitrary byte arrays:
| Name | Type | Description | Validation |
|------|----------------------------|------------------------|-----------------------------------------------------------------------------|
| Txs | Matrix of bytes ([][]byte) | Slice of transactions. | Validation does not occur on this field, this data is unknown to Tendermint |
## Commit
Commit is a simple wrapper for a list of signatures, with one for each validator. It also contains the relevant BlockID, height and round:
| Name | Type | Description | Validation |
|------------|----------------------------------|----------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|
| Height | uint64 | Height at which this commit was created. | Must be > 0 |
| Round | int32 | Round that the commit corresponds to. | Must be > 0 |
| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | Must adhere to the validation rules of [BlockID](#blockid). |
| Signatures | Array of [CommitSig](#commitsig) | Array of commit signatures that correspond to current validator set. | Length of signatures must be > 0 and adhere to the validation of each individual [Commitsig](#commitsig) |
## CommitSig
`CommitSig` represents a signature of a validator, who has voted either for nil,
a particular `BlockID` or was absent. It's a part of the `Commit` and can be used
to reconstruct the vote set given the validator set.
| Name | Type | Description | Validation |
|------------------|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------|
| BlockIDFlag | [BlockIDFlag](#blockidflag) | Represents the validators participation in consensus: Either voted for the block that received the majority, voted for another block, voted nil or did not vote | Must be one of the fields in the [BlockIDFlag](#blockidflag) enum |
| ValidatorAddress | [Address](#address) | Address of the validator | Must be of length 20 |
| Timestamp | [Time](#time) | This field will vary from `CommitSig` to `CommitSig`. It represents the timestamp of the validator. | [Time](#time) |
| Signature | [Signature](#signature) | Signature corresponding to the validators participation in consensus. | The length of the signature must be > 0 and < than 64 |
NOTE: `ValidatorAddress` and `Timestamp` fields may be removed in the future
(see [ADR-25](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-025-commit.md)).
## BlockIDFlag
BlockIDFlag represents which BlockID the [signature](#commitsig) is for.
```go
enum BlockIDFlag {
BLOCK_ID_FLAG_UNKNOWN = 0;
BLOCK_ID_FLAG_ABSENT = 1; // signatures for other blocks are also considered absent
BLOCK_ID_FLAG_COMMIT = 2;
BLOCK_ID_FLAG_NIL = 3;
}
```
## Vote
A vote is a signed message from a validator for a particular block.
The vote includes information about the validator signing it. When stored in the blockchain or propagated over the network, votes are encoded in Protobuf.
| Name | Type | Description | Validation |
|------------------|---------------------------------|---------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|
| Type | [SignedMsgType](#signedmsgtype) | Either prevote or precommit. [SignedMsgType](#signedmsgtype) | A Vote is valid if its corresponding fields are included in the enum [signedMsgType](#signedmsgtype) |
| Height | uint64 | Height for which this vote was created for | Must be > 0 |
| Round | int32 | Round that the commit corresponds to. | Must be > 0 |
| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | [BlockID](#blockid) |
| Timestamp | [Time](#Time) | Timestamp represents the time at which a validator signed. | [Time](#time) |
| ValidatorAddress | slice of bytes (`[]byte`) | Address of the validator | Length must be equal to 20 |
| ValidatorIndex | int32 | Index at a specific block height that corresponds to the Index of the validator in the set. | must be > 0 |
| Signature | slice of bytes (`[]byte`) | Signature by the validator if they participated in consensus for the associated bock. | Length of signature must be > 0 and < 64 |
## CanonicalVote
CanonicalVote is for validator signing. This type will not be present in a block. Votes are represented via `CanonicalVote` and also encoded using protobuf via `type.SignBytes` which includes the `ChainID`, and uses a different ordering of
the fields.
```proto
message CanonicalVote {
SignedMsgType type = 1;
fixed64 height = 2;
sfixed64 round = 3;
CanonicalBlockID block_id = 4;
google.protobuf.Timestamp timestamp = 5;
string chain_id = 6;
}
```
For signing, votes are represented via [`CanonicalVote`](#canonicalvote) and also encoded using protobuf via
`type.SignBytes` which includes the `ChainID`, and uses a different ordering of
the fields.
We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the `SignBytes`
using the given ChainID:
```go
func (vote *Vote) Verify(chainID string, pubKey crypto.PubKey) error {
if !bytes.Equal(pubKey.Address(), vote.ValidatorAddress) {
return ErrVoteInvalidValidatorAddress
}
if !pubKey.VerifyBytes(types.VoteSignBytes(chainID), vote.Signature) {
return ErrVoteInvalidSignature
}
return nil
}
```
## Proposal
Proposal contains height and round for which this proposal is made, BlockID as a unique identifier
of proposed block, timestamp, and POLRound (a so-called Proof-of-Lock (POL) round) that is needed for
termination of the consensus. If POLRound >= 0, then BlockID corresponds to the block that
is locked in POLRound. The message is signed by the validator private key.
| Name | Type | Description | Validation |
|-----------|---------------------------------|---------------------------------------------------------------------------------------|---------------------------------------------------------|
| Type | [SignedMsgType](#signedmsgtype) | Represents a Proposal [SignedMsgType](#signedmsgtype) | Must be `ProposalType` [signedMsgType](#signedmsgtype) |
| Height | uint64 | Height for which this vote was created for | Must be > 0 |
| Round | int32 | Round that the commit corresponds to. | Must be > 0 |
| POLRound | int64 | Proof of lock | Must be > 0 |
| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | [BlockID](#blockid) |
| Timestamp | [Time](#Time) | Timestamp represents the time at which a validator signed. | [Time](#time) |
| Signature | slice of bytes (`[]byte`) | Signature by the validator if they participated in consensus for the associated bock. | Length of signature must be > 0 and < 64 |
## SignedMsgType
Signed message type represents a signed messages in consensus.
```proto
enum SignedMsgType {
SIGNED_MSG_TYPE_UNKNOWN = 0;
// Votes
SIGNED_MSG_TYPE_PREVOTE = 1;
SIGNED_MSG_TYPE_PRECOMMIT = 2;
// Proposal
SIGNED_MSG_TYPE_PROPOSAL = 32;
}
```
## Signature
Signatures in Tendermint are raw bytes representing the underlying signature.
See the [signature spec](./encoding.md#key-types) for more.
## EvidenceList
EvidenceList is a simple wrapper for a list of evidence:
| Name | Type | Description | Validation |
|----------|--------------------------------|----------------------------------------|-----------------------------------------------------------------|
| Evidence | Array of [Evidence](#evidence) | List of verified [evidence](#evidence) | Validation adheres to individual types of [Evidence](#evidence) |
## Evidence
Evidence in Tendermint is used to indicate breaches in the consensus by a validator.
More information on how evidence works in Tendermint can be found [here](../consensus/evidence.md)
### DuplicateVoteEvidence
`DuplicateVoteEvidence` represents a validator that has voted for two different blocks
in the same round of the same height. Votes are lexicographically sorted on `BlockID`.
| Name | Type | Description | Validation |
|------------------|---------------|--------------------------------------------------------------------|-----------------------------------------------------|
| VoteA | [Vote](#vote) | One of the votes submitted by a validator when they equivocated | VoteA must adhere to [Vote](#vote) validation rules |
| VoteB | [Vote](#vote) | The second vote submitted by a validator when they equivocated | VoteB must adhere to [Vote](#vote) validation rules |
| TotalVotingPower | int64 | The total power of the validator set at the height of equivocation | Must be equal to nodes own copy of the data |
| ValidatorPower | int64 | Power of the equivocating validator at the height | Must be equal to the nodes own copy of the data |
| Timestamp | [Time](#Time) | Time of the block where the equivocation occurred | Must be equal to the nodes own copy of the data |
### LightClientAttackEvidence
`LightClientAttackEvidence` is a generalized evidence that captures all forms of known attacks on
a light client such that a full node can verify, propose and commit the evidence on-chain for
punishment of the malicious validators. There are three forms of attacks: Lunatic, Equivocation
and Amnesia. These attacks are exhaustive. You can find a more detailed overview of this [here](../light-client/accountability#the_misbehavior_of_faulty_validators)
| Name | Type | Description | Validation |
|----------------------|------------------------------------|----------------------------------------------------------------------|------------------------------------------------------------------|
| ConflictingBlock | [LightBlock](#LightBlock) | Read Below | Must adhere to the validation rules of [lightBlock](#lightblock) |
| CommonHeight | int64 | Read Below | must be > 0 |
| Byzantine Validators | Array of [Validators](#Validators) | validators that acted maliciously | Read Below |
| TotalVotingPower | int64 | The total power of the validator set at the height of the infraction | Must be equal to the nodes own copy of the data |
| Timestamp | [Time](#Time) | Time of the block where the infraction occurred | Must be equal to the nodes own copy of the data |
## LightBlock
LightBlock is the core data structure of the [light client](../light-client/README.md). It combines two data structures needed for verification ([signedHeader](#signedheader) & [validatorSet](#validatorset)).
| Name | Type | Description | Validation |
|--------------|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|
| SignedHeader | [SignedHeader](#signedheader) | The header and commit, these are used for verification purposes. To find out more visit [light client docs](../light-client/README.md) | Must not be nil and adhere to the validation rules of [signedHeader](#signedheader) |
| ValidatorSet | [ValidatorSet](#validatorset) | The validatorSet is used to help with verify that the validators in that committed the infraction were truly in the validator set. | Must not be nil and adhere to the validation rules of [validatorSet](#validatorset) |
The `SignedHeader` and `ValidatorSet` are linked by the hash of the validator set(`SignedHeader.ValidatorsHash == ValidatorSet.Hash()`.
## SignedHeader
The SignedhHeader is the [header](#header) accompanied by the commit to prove it.
| Name | Type | Description | Validation |
|--------|-------------------|-------------------|-----------------------------------------------------------------------------------|
| Header | [Header](#Header) | [Header](#header) | Header cannot be nil and must adhere to the [Header](#Header) validation criteria |
| Commit | [Commit](#commit) | [Commit](#commit) | Commit cannot be nil and must adhere to the [Commit](#commit) criteria |
## ValidatorSet
| Name | Type | Description | Validation |
|------------|----------------------------------|----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|
| Validators | Array of [validator](#validator) | List of the active validators at a specific height | The list of validators can not be empty or nil and must adhere to the validation rules of [validator](#validator) |
| Proposer | [validator](#validator) | The block proposer for the corresponding block | The proposer cannot be nil and must adhere to the validation rules of [validator](#validator) |
## Validator
| Name | Type | Description | Validation |
|------------------|---------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------------|
| Address | [Address](#address) | Validators Address | Length must be of size 20 |
| Pubkey | slice of bytes (`[]byte`) | Validators Public Key | must be a length greater than 0 |
| VotingPower | int64 | Validators voting power | cannot be < 0 |
| ProposerPriority | int64 | Validators proposer priority. This is used to gauge when a validator is up next to propose blocks | No validation, value can be negative and positive |
## Address
Address is a type alias of a slice of bytes. The address is calculated by hashing the public key using sha256 and truncating it to only use the first 20 bytes of the slice.
```go
const (
TruncatedSize = 20
)
func SumTruncated(bz []byte) []byte {
hash := sha256.Sum256(bz)
return hash[:TruncatedSize]
}
```
## ConsensusParams
| Name | Type | Description | Field Number |
|-----------|-------------------------------------|------------------------------------------------------------------------------|--------------|
| block | [BlockParams](#blockparams) | Parameters limiting the size of a block and time between consecutive blocks. | 1 |
| evidence | [EvidenceParams](#evidenceparams) | Parameters limiting the validity of evidence of byzantine behaviour. | 2 |
| validator | [ValidatorParams](#validatorparams) | Parameters limiting the types of public keys validators can use. | 3 |
| version | [BlockParams](#blockparams) | The ABCI application version. | 4 |
### BlockParams
| Name | Type | Description | Field Number |
|--------------|-------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| max_bytes | int64 | Max size of a block, in bytes. | 1 |
| max_gas | int64 | Max sum of `GasWanted` in a proposed block. NOTE: blocks that violate this may be committed if there are Byzantine proposers. It's the application's responsibility to handle this when processing a block! | 2 |
### EvidenceParams
| Name | Type | Description | Field Number |
|--------------------|------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| max_age_num_blocks | int64 | Max age of evidence, in blocks. | 1 |
| max_age_duration | [google.protobuf.Duration](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration) | Max age of evidence, in time. It should correspond with an app's "unbonding period" or other similar mechanism for handling [Nothing-At-Stake attacks](https://github.com/ethereum/wiki/wiki/Proof-of-Stake-FAQ#what-is-the-nothing-at-stake-problem-and-how-can-it-be-fixed). | 2 |
| max_bytes | int64 | maximum size in bytes of total evidence allowed to be entered into a block | 3 |
### ValidatorParams
| Name | Type | Description | Field Number |
|---------------|-----------------|-----------------------------------------------------------------------|--------------|
| pub_key_types | repeated string | List of accepted public key types. Uses same naming as `PubKey.Type`. | 1 |
### VersionParams
| Name | Type | Description | Field Number |
|-------------|--------|-------------------------------|--------------|
| app_version | uint64 | The ABCI application version. | 1 |
## Proof
| Name | Type | Description | Field Number |
|-----------|----------------|-----------------------------------------------|--------------|
| total | int64 | Total number of items. | 1 |
| index | int64 | Index item to prove. | 2 |
| leaf_hash | bytes | Hash of item value. | 3 |
| aunts | repeated bytes | Hashes from leaf's sibling to a root's child. | 4 |

300
spec/core/encoding.md Normal file
View File

@@ -0,0 +1,300 @@
# Encoding
## Protocol Buffers
Tendermint uses [Protocol Buffers](https://developers.google.com/protocol-buffers), specifically proto3, for all data structures.
Please see the [Proto3 language guide](https://developers.google.com/protocol-buffers/docs/proto3) for more details.
## Byte Arrays
The encoding of a byte array is simply the raw-bytes prefixed with the length of
the array as a `UVarint` (what proto calls a `Varint`).
For details on varints, see the [protobuf
spec](https://developers.google.com/protocol-buffers/docs/encoding#varints).
For example, the byte-array `[0xA, 0xB]` would be encoded as `0x020A0B`,
while a byte-array containing 300 entires beginning with `[0xA, 0xB, ...]` would
be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300.
## Hashing
Tendermint uses `SHA256` as its hash function.
Objects are always serialized before being hashed.
So `SHA256(obj)` is short for `SHA256(ProtoEncoding(obj))`.
## Public Key Cryptography
Tendermint uses Protobuf [Oneof](https://developers.google.com/protocol-buffers/docs/proto3#oneof)
to distinguish between different types public keys, and signatures.
Additionally, for each public key, Tendermint
defines an Address function that can be used as a more compact identifier in
place of the public key. Here we list the concrete types, their names,
and prefix bytes for public keys and signatures, as well as the address schemes
for each PubKey. Note for brevity we don't
include details of the private keys beyond their type and name.
### Key Types
Each type specifies it's own pubkey, address, and signature format.
#### Ed25519
The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key:
```go
address = SHA256(pubkey)[:20]
```
The signature is the raw 64-byte ED25519 signature.
Tendermint adopted [zip215](https://zips.z.cash/zip-0215) for verification of ed25519 signatures.
> Note: This change will be released in the next major release of Tendermint-Go (0.35).
#### Secp256k1
The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key:
```go
address = SHA256(pubkey)[:20]
```
## Other Common Types
### BitArray
The BitArray is used in some consensus messages to represent votes received from
validators, or parts received in a block. It is represented
with a struct containing the number of bits (`Bits`) and the bit-array itself
encoded in base64 (`Elems`).
| Name | Type |
|-------|----------------------------|
| bits | int64 |
| elems | slice of int64 (`[]int64`) |
Note BitArray receives a special JSON encoding in the form of `x` and `_`
representing `1` and `0`. Ie. the BitArray `10110` would be JSON encoded as
`"x_xx_"`
### Part
Part is used to break up blocks into pieces that can be gossiped in parallel
and securely verified using a Merkle tree of the parts.
Part contains the index of the part (`Index`), the actual
underlying data of the part (`Bytes`), and a Merkle proof that the part is contained in
the set (`Proof`).
| Name | Type |
|-------|---------------------------|
| index | uint32 |
| bytes | slice of bytes (`[]byte`) |
| proof | [proof](#merkle-proof) |
See details of SimpleProof, below.
### MakeParts
Encode an object using Protobuf and slice it into parts.
Tendermint uses a part size of 65536 bytes, and allows a maximum of 1601 parts
(see `types.MaxBlockPartsCount`). This corresponds to the hard-coded block size
limit of 100MB.
```go
func MakeParts(block Block) []Part
```
## Merkle Trees
For an overview of Merkle trees, see
[wikipedia](https://en.wikipedia.org/wiki/Merkle_tree)
We use the RFC 6962 specification of a merkle tree, with sha256 as the hash function.
Merkle trees are used throughout Tendermint to compute a cryptographic digest of a data structure.
The differences between RFC 6962 and the simplest form a merkle tree are that:
1. leaf nodes and inner nodes have different hashes.
This is for "second pre-image resistance", to prevent the proof to an inner node being valid as the proof of a leaf.
The leaf nodes are `SHA256(0x00 || leaf_data)`, and inner nodes are `SHA256(0x01 || left_hash || right_hash)`.
2. When the number of items isn't a power of two, the left half of the tree is as big as it could be.
(The largest power of two less than the number of items) This allows new leaves to be added with less
recomputation. For example:
```md
Simple Tree with 6 items Simple Tree with 7 items
* *
/ \ / \
/ \ / \
/ \ / \
/ \ / \
* * * *
/ \ / \ / \ / \
/ \ / \ / \ / \
/ \ / \ / \ / \
* * h4 h5 * * * h6
/ \ / \ / \ / \ / \
h0 h1 h2 h3 h0 h1 h2 h3 h4 h5
```
### MerkleRoot
The function `MerkleRoot` is a simple recursive function defined as follows:
```go
// SHA256([]byte{})
func emptyHash() []byte {
return tmhash.Sum([]byte{})
}
// SHA256(0x00 || leaf)
func leafHash(leaf []byte) []byte {
return tmhash.Sum(append(0x00, leaf...))
}
// SHA256(0x01 || left || right)
func innerHash(left []byte, right []byte) []byte {
return tmhash.Sum(append(0x01, append(left, right...)...))
}
// largest power of 2 less than k
func getSplitPoint(k int) { ... }
func MerkleRoot(items [][]byte) []byte{
switch len(items) {
case 0:
return empthHash()
case 1:
return leafHash(items[0])
default:
k := getSplitPoint(len(items))
left := MerkleRoot(items[:k])
right := MerkleRoot(items[k:])
return innerHash(left, right)
}
}
```
Note: `MerkleRoot` operates on items which are arbitrary byte arrays, not
necessarily hashes. For items which need to be hashed first, we introduce the
`Hashes` function:
```go
func Hashes(items [][]byte) [][]byte {
return SHA256 of each item
}
```
Note: we will abuse notion and invoke `MerkleRoot` with arguments of type `struct` or type `[]struct`.
For `struct` arguments, we compute a `[][]byte` containing the protobuf encoding of each
field in the struct, in the same order the fields appear in the struct.
For `[]struct` arguments, we compute a `[][]byte` by protobuf encoding the individual `struct` elements.
### Merkle Proof
Proof that a leaf is in a Merkle tree is composed as follows:
| Name | Type |
|----------|----------------------------|
| total | int64 |
| index | int64 |
| leafHash | slice of bytes (`[]byte`) |
| aunts | Matrix of bytes ([][]byte) |
Which is verified as follows:
```golang
func (proof Proof) Verify(rootHash []byte, leaf []byte) bool {
assert(proof.LeafHash, leafHash(leaf)
computedHash := computeHashFromAunts(proof.Index, proof.Total, proof.LeafHash, proof.Aunts)
return computedHash == rootHash
}
func computeHashFromAunts(index, total int, leafHash []byte, innerHashes [][]byte) []byte{
assert(index < total && index >= 0 && total > 0)
if total == 1{
assert(len(proof.Aunts) == 0)
return leafHash
}
assert(len(innerHashes) > 0)
numLeft := getSplitPoint(total) // largest power of 2 less than total
if index < numLeft {
leftHash := computeHashFromAunts(index, numLeft, leafHash, innerHashes[:len(innerHashes)-1])
assert(leftHash != nil)
return innerHash(leftHash, innerHashes[len(innerHashes)-1])
}
rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1])
assert(rightHash != nil)
return innerHash(innerHashes[len(innerHashes)-1], rightHash)
}
```
The number of aunts is limited to 100 (`MaxAunts`) to protect the node against DOS attacks.
This limits the tree size to 2^100 leaves, which should be sufficient for any
conceivable purpose.
### IAVL+ Tree
Because Tendermint only uses a Simple Merkle Tree, application developers are expect to use their own Merkle tree in their applications. For example, the IAVL+ Tree - an immutable self-balancing binary tree for persisting application state is used by the [Cosmos SDK](https://github.com/cosmos/cosmos-sdk/blob/ae77f0080a724b159233bd9b289b2e91c0de21b5/docs/interfaces/lite/specification.md)
## JSON
Tendermint has its own JSON encoding in order to keep backwards compatibility with the previous RPC layer.
Registered types are encoded as:
```json
{
"type": "<type name>",
"value": <JSON>
}
```
For instance, an ED25519 PubKey would look like:
```json
{
"type": "tendermint/PubKeyEd25519",
"value": "uZ4h63OFWuQ36ZZ4Bd6NF+/w9fWUwrOncrQsackrsTk="
}
```
Where the `"value"` is the base64 encoding of the raw pubkey bytes, and the
`"type"` is the type name for Ed25519 pubkeys.
### Signed Messages
Signed messages (eg. votes, proposals) in the consensus are encoded using protobuf.
When signing, the elements of a message are re-ordered so the fixed-length fields
are first, making it easy to quickly check the type, height, and round.
The `ChainID` is also appended to the end.
We call this encoding the SignBytes. For instance, SignBytes for a vote is the protobuf encoding of the following struct:
```protobuf
message CanonicalVote {
SignedMsgType type = 1;
sfixed64 height = 2; // canonicalization requires fixed size encoding here
sfixed64 round = 3; // canonicalization requires fixed size encoding here
CanonicalBlockID block_id = 4;
google.protobuf.Timestamp timestamp = 5;
string chain_id = 6;
}
```
The field ordering and the fixed sized encoding for the first three fields is optimized to ease parsing of SignBytes
in HSMs. It creates fixed offsets for relevant fields that need to be read in this context.
> Note: All canonical messages are length prefixed.
For more details, see the [signing spec](../consensus/signing.md).
Also, see the motivating discussion in
[#1622](https://github.com/tendermint/tendermint/issues/1622).

34
spec/core/genesis.md Normal file
View File

@@ -0,0 +1,34 @@
# Genesis
The genesis file is the starting point of a chain. An application will populate the `app_state` field in the genesis with their required fields. Tendermint is not able to validate this section because it is unaware what application state consists of.
## Genesis Fields
- `genesis_time`: The genesis time is the time the blockchain started or will start. If nodes are started before this time they will sit idle until the time specified.
- `chain_id`: The chainid is the chain identifier. Every chain should have a unique identifier. When conducting a fork based upgrade, we recommend changing the chainid to avoid network or consensus errors.
- `initial_height`: This field is the starting height of the blockchain. When conducting a chain restart to avoid restarting at height 1, the network is able to start at a specified height.
- `consensus_params`
- `block`
- `max_bytes`: The max amount of bytes a block can be.
- `max_gas`: The maximum amount of gas that a block can have.
- `time_iota_ms`: This parameter has no value anymore in Tendermint-core.
- `evidence`
- `max_age_num_blocks`: After this preset amount of blocks has passed a single piece of evidence is considered invalid
- `max_age_duration`: After this preset amount of time has passed a single piece of evidence is considered invalid.
- `max_bytes`: The max amount of bytes of all evidence included in a block.
> Note: For evidence to be considered invalid, evidence must be older than both `max_age_num_blocks` and `max_age_duration`
- `validator`
- `pub_key_types`: Defines which curves are to be accepted as a valid validator consensus key. Tendermint supports ed25519, sr25519 and secp256k1.
- `version`
- `app_version`: The version of the application. This is set by the application and is used to identify which version of the app a user should be using in order to operate a node.
- `validators`
- This is an array of validators. This validator set is used as the starting validator set of the chain. This field can be empty, if the application sets the validator set in `InitChain`.
- `app_hash`: The applications state root hash. This field does not need to be populated at the start of the chain, the application may provide the needed information via `Initchain`.
- `app_state`: This section is filled in by the application and is unknown to Tendermint.

13
spec/core/readme.md Normal file
View File

@@ -0,0 +1,13 @@
---
order: 1
parent:
title: Core
order: 3
---
This section describes the core types and functionality of the Tendermint protocol implementation.
- [Core Data Structures](./data_structures.md)
- [Encoding](./encoding.md)
- [Genesis](./genesis.md)
- [State](./state.md)

121
spec/core/state.md Normal file
View File

@@ -0,0 +1,121 @@
# State
The state contains information whose cryptographic digest is included in block headers, and thus is
necessary for validating new blocks. For instance, the validators set and the results of
transactions are never included in blocks, but their Merkle roots are:
the state keeps track of them.
The `State` object itself is an implementation detail, since it is never
included in a block or gossiped over the network, and we never compute
its hash. The persistence or query interface of the `State` object
is an implementation detail and not included in the specification.
However, the types in the `State` object are part of the specification, since
the Merkle roots of the `State` objects are included in blocks and values are used during
validation.
```go
type State struct {
ChainID string
InitialHeight int64
LastBlockHeight int64
LastBlockID types.BlockID
LastBlockTime time.Time
Version Version
LastResults []Result
AppHash []byte
LastValidators ValidatorSet
Validators ValidatorSet
NextValidators ValidatorSet
ConsensusParams ConsensusParams
}
```
The chain ID and initial height are taken from the genesis file, and not changed again. The
initial height will be `1` in the typical case, `0` is an invalid value.
Note there is a hard-coded limit of 10000 validators. This is inherited from the
limit on the number of votes in a commit.
Further information on [`Validator`'s](./data_structures.md#validator),
[`ValidatorSet`'s](./data_structures.md#validatorset) and
[`ConsensusParams`'s](./data_structures.md#consensusparams) can
be found in [data structures](./data_structures.md)
## Execution
State gets updated at the end of executing a block. Of specific interest is `ResponseEndBlock` and
`ResponseCommit`
```go
type ResponseEndBlock struct {
ValidatorUpdates []ValidatorUpdate `protobuf:"bytes,1,rep,name=validator_updates,json=validatorUpdates,proto3" json:"validator_updates"`
ConsensusParamUpdates *types1.ConsensusParams `protobuf:"bytes,2,opt,name=consensus_param_updates,json=consensusParamUpdates,proto3" json:"consensus_param_updates,omitempty"`
Events []Event `protobuf:"bytes,3,rep,name=events,proto3" json:"events,omitempty"`
}
```
where
```go
type ValidatorUpdate struct {
PubKey crypto.PublicKey `protobuf:"bytes,1,opt,name=pub_key,json=pubKey,proto3" json:"pub_key"`
Power int64 `protobuf:"varint,2,opt,name=power,proto3" json:"power,omitempty"`
}
```
and
```go
type ResponseCommit struct {
// reserve 1
Data []byte `protobuf:"bytes,2,opt,name=data,proto3" json:"data,omitempty"`
RetainHeight int64 `protobuf:"varint,3,opt,name=retain_height,json=retainHeight,proto3" json:"retain_height,omitempty"`
}
```
`ValidatorUpdates` are used to add and remove validators to the current set as well as update
validator power. Setting validator power to 0 in `ValidatorUpdate` will cause the validator to be
removed. `ConsensusParams` are safely copied across (i.e. if a field is nil it gets ignored) and the
`Data` from the `ResponseCommit` is used as the `AppHash`
## Version
```go
type Version struct {
consensus Consensus
software string
}
```
[`Consensus`](./data_structures.md#version) contains the protocol version for the blockchain and the
application.
## Block
The total size of a block is limited in bytes by the `ConsensusParams.Block.MaxBytes`.
Proposed blocks must be less than this size, and will be considered invalid
otherwise.
Blocks should additionally be limited by the amount of "gas" consumed by the
transactions in the block, though this is not yet implemented.
## Evidence
For evidence in a block to be valid, it must satisfy:
```go
block.Header.Time-evidence.Time < ConsensusParams.Evidence.MaxAgeDuration &&
block.Header.Height-evidence.Height < ConsensusParams.Evidence.MaxAgeNumBlocks
```
A block must not contain more than `ConsensusParams.Evidence.MaxBytes` of evidence. This is
implemented to mitigate spam attacks.
## Validator
Validators from genesis file and `ResponseEndBlock` must have pubkeys of type ∈
`ConsensusParams.Validator.PubKeyTypes`.

37
spec/ivy-proofs/Dockerfile Normal file
View File

@@ -0,0 +1,37 @@
# we need python2 support, which was dropped after buster:
FROM debian:buster
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update
RUN apt-get install -y apt-utils
# Install and configure locale `en_US.UTF-8`
RUN apt-get install -y locales && \
sed -i -e "s/# $en_US.*/en_US.UTF-8 UTF-8/" /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=en_US.UTF-8
ENV LANG=en_US.UTF-8
RUN apt-get update
RUN apt-get install -y git python2 python-pip g++ cmake python-ply python-tk tix pkg-config libssl-dev python-setuptools
# create a user:
RUN useradd -ms /bin/bash user
USER user
WORKDIR /home/user
RUN git clone --recurse-submodules https://github.com/kenmcmil/ivy.git
WORKDIR /home/user/ivy/
RUN git checkout 271ee38980699115508eb90a0dd01deeb750a94b
RUN python2.7 build_submodules.py
RUN mkdir -p "/home/user/python/lib/python2.7/site-packages"
ENV PYTHONPATH="/home/user/python/lib/python2.7/site-packages"
# need to install pyparsing manually because otherwise wrong version found
RUN pip install pyparsing
RUN python2.7 setup.py install --prefix="/home/user/python/"
ENV PATH=$PATH:"/home/user/python/bin/"
WORKDIR /home/user/tendermint-proof/
ENTRYPOINT ["/home/user/tendermint-proof/check_proofs.sh"]

33
spec/ivy-proofs/README.md Normal file
View File

@@ -0,0 +1,33 @@
# Ivy Proofs
```copyright
Copyright (c) 2020 Galois, Inc.
SPDX-License-Identifier: Apache-2.0
```
## Contents
This folder contains:
* `tendermint.ivy`, a specification of Tendermint algorithm as described in *The latest gossip on BFT consensus* by E. Buchman, J. Kwon, Z. Milosevic.
* `abstract_tendermint.ivy`, a more abstract specification of Tendermint that is more verification-friendly.
* `classic_safety.ivy`, a proof that Tendermint satisfies the classic safety property of BFT consensus: if every two quorums have a well-behaved node in common, then no two well-behaved nodes ever disagree.
* `accountable_safety_1.ivy`, a proof that, assuming every quorum contains at least one well-behaved node, if two well-behaved nodes disagree, then there is evidence demonstrating at least f+1 nodes misbehaved.
* `accountable_safety_2.ivy`, a proof that, regardless of any assumption about quorums, well-behaved nodes cannot be framed by malicious nodes. In other words, malicious nodes can never construct evidence that incriminates a well-behaved node.
* `network_shim.ivy`, the network model and a convenience `shim` object to interface with the Tendermint specification.
* `domain_model.ivy`, a specification of the domain model underlying the Tendermint specification, i.e. rounds, value, quorums, etc.
All specifications and proofs are written in [Ivy](https://github.com/kenmcmil/ivy).
The license above applies to all files in this folder.
## Building and running
The easiest way to check the proofs is to use [Docker](https://www.docker.com/).
1. Install [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/).
2. Build a Docker image: `docker-compose build`
3. Run the proofs inside the Docker container: `docker-compose run
tendermint-proof`. This will check all the proofs with the `ivy_check`
command and write the output of `ivy_check` to a subdirectory of `./output/'

178
spec/ivy-proofs/abstract_tendermint.ivy Normal file
View File

@@ -0,0 +1,178 @@
#lang ivy1.7
# ---
# layout: page
# title: Abstract specification of Tendermint in Ivy
# ---
# Here we define an abstract version of the Tendermint specification. We use
# two main forms of abstraction: a) We abstract over how information is
# transmitted (there is no network). b) We abstract functions using relations.
# For example, we abstract over a node's current round, instead only tracking
# with a relation which rounds the node has left. We do something similar for
# the `lockedRound` variable. This is in order to avoid using a function from
# node to round, and it allows us to emit verification conditions that are
# efficiently solvable by Z3.
# This specification also defines the observations that are used to adjudicate
# misbehavior. Well-behaved nodes faithfully observe every message that they
# use to take a step, while Byzantine nodes can fake observations about
# themselves (including withholding observations). Misbehavior is defined using
# the collection of all observations made (in reality, those observations must
# be collected first, but we do not model this process).
include domain_model
module abstract_tendermint = {
# Protocol state
# ##############
relation left_round(N:node, R:round)
relation prevoted(N:node, R:round, V:value)
relation precommitted(N:node, R:round, V:value)
relation decided(N:node, R:round, V:value)
relation locked(N:node, R:round, V:value)
# Accountability relations
# ########################
relation observed_prevoted(N:node, R:round, V:value)
relation observed_precommitted(N:node, R:round, V:value)
# relations that are defined in terms of the previous two:
relation observed_equivocation(N:node)
relation observed_unlawful_prevote(N:node)
relation agreement
relation accountability_violation
object defs = { # we hide those definitions and use them only when needed
private {
definition [observed_equivocation_def] observed_equivocation(N) = exists V1,V2,R .
V1 ~= V2 & (observed_precommitted(N,R,V1) & observed_precommitted(N,R,V2) | observed_prevoted(N,R,V1) & observed_prevoted(N,R,V2))
definition [observed_unlawful_prevote_def] observed_unlawful_prevote(N) = exists V1,V2,R1,R2 .
V1 ~= value.nil & V2 ~= value.nil & V1 ~= V2 & R1 < R2 & observed_precommitted(N,R1,V1) & observed_prevoted(N,R2,V2)
& forall Q,R . R1 <= R & R < R2 & nset.is_quorum(Q) -> exists N2 . nset.member(N2,Q) & ~observed_prevoted(N2,R,V2)
definition [agreement_def] agreement = forall N1,N2,R1,R2,V1,V2 . well_behaved(N1) & well_behaved(N2) & decided(N1,R1,V1) & decided(N2,R2,V2) -> V1 = V2
definition [accountability_violation_def] accountability_violation = exists Q1,Q2 . nset.is_quorum(Q1) & nset.is_quorum(Q2) & (forall N . nset.member(N,Q1) & nset.member(N,Q2) -> observed_equivocation(N) | observed_unlawful_prevote(N))
}
}
# Protocol transitions
# ####################
after init {
left_round(N,R) := R < 0;
prevoted(N,R,V) := false;
precommitted(N,R,V) := false;
decided(N,R,V) := false;
locked(N,R,V) := false;
observed_prevoted(N,R,V) := false;
observed_precommitted(N,R,V) := false;
}
# Actions are named after the corresponding line numbers in the Tendermint
# arXiv paper.
action l_11(n:node, r:round) = { # start round r
require ~left_round(n,r);
left_round(n,R) := R < r;
}
action l_22(n:node, rp:round, v:value) = {
require ~left_round(n,rp);
require ~prevoted(n,rp,V) & ~precommitted(n,rp,V);
require (forall R,V . locked(n,R,V) -> V = v) | v = value.nil;
prevoted(n, rp, v) := true;
left_round(n, R) := R < rp; # leave all lower rounds.
observed_prevoted(n, rp, v) := observed_prevoted(n, rp, v) | well_behaved(n); # the node observes itself
}
action l_28(n:node, rp:round, v:value, vr:round, q:nset) = {
require ~left_round(n,rp) & ~prevoted(n,rp,V);
require ~prevoted(n,rp,V) & ~precommitted(n,rp,V);
require vr < rp;
require nset.is_quorum(q) & (forall N . nset.member(N,q) -> (prevoted(N,vr,v) | ~well_behaved(N)));
var proposal:value;
if value.valid(v) & ((forall R0,V0 . locked(n,R0,V0) -> R0 <= vr) | (forall R,V . locked(n,R,V) -> V = v)) {
proposal := v;
}
else {
proposal := value.nil;
};
prevoted(n, rp, proposal) := true;
left_round(n, R) := R < rp; # leave all lower rounds
observed_prevoted(N, vr, v) := observed_prevoted(N, vr, v) | (well_behaved(n) & nset.member(N,q)); # the node observes the prevotes of quorum q
observed_prevoted(n, rp, proposal) := observed_prevoted(n, rp, proposal) | well_behaved(n); # the node observes itself
}
action l_36(n:node, rp:round, v:value, q:nset) = {
require v ~= value.nil;
require ~left_round(n,rp);
require exists V . prevoted(n,rp,V);
require ~precommitted(n,rp,V);
require nset.is_quorum(q) & (forall N . nset.member(N,q) -> (prevoted(N,rp,v) | ~well_behaved(N)));
precommitted(n, rp, v) := true;
left_round(n, R) := R < rp; # leave all lower rounds
locked(n,R,V) := R <= rp & V = v;
observed_prevoted(N, rp, v) := observed_prevoted(N, rp, v) | (well_behaved(n) & nset.member(N,q)); # the node observes the prevotes of quorum q
observed_precommitted(n, rp, v) := observed_precommitted(n, rp, v) | well_behaved(n); # the node observes itself
}
action l_44(n:node, rp:round, q:nset) = {
require ~left_round(n,rp);
require ~precommitted(n,rp,V);
require nset.is_quorum(q) & (forall N .nset.member(N,q) -> (prevoted(N,rp,value.nil) | ~well_behaved(N)));
precommitted(n, rp, value.nil) := true;
left_round(n, R) := R < rp; # leave all lower rounds
observed_prevoted(N, rp, value.nil) := observed_prevoted(N, rp, value.nil) | (well_behaved(n) & nset.member(N,q)); # the node observes the prevotes of quorum q
observed_precommitted(n, rp, value.nil) := observed_precommitted(n, rp, value.nil) | well_behaved(n); # the node observes itself
}
action l_57(n:node, rp:round) = {
require ~left_round(n,rp);
require ~prevoted(n,rp,V);
prevoted(n, rp, value.nil) := true;
left_round(n, R) := R < rp; # leave all lower rounds
observed_prevoted(n, rp, value.nil) := observed_prevoted(n, rp, value.nil) | well_behaved(n); # the node observes itself
}
action l_61(n:node, rp:round) = {
require ~left_round(n,rp);
require ~precommitted(n,rp,V);
precommitted(n, rp, value.nil) := true;
left_round(n, R) := R < rp; # leave all lower rounds
observed_precommitted(n, rp, value.nil) := observed_precommitted(n, rp, value.nil) | well_behaved(n); # the node observes itself
}
action decide(n:node, r:round, v:value, q:nset) = {
require v ~= value.nil;
require nset.is_quorum(q) & (forall N . nset.member(N, q) -> (precommitted(N, r, v) | ~well_behaved(N)));
decided(n, r, v) := true;
observed_precommitted(N, r, v) := observed_precommitted(N, r, v) | (well_behaved(n) & nset.member(N,q)); # the node observes the precommits of quorum q
}
action misbehave = {
# Byzantine nodes can claim they observed whatever they want about themselves,
# but they cannot remove observations. Note that we use assume because we don't
# want those to be checked; we just want them to be true (that's the model of
# Byzantine behavior).
observed_prevoted(N,R,V) := *;
assume (old observed_prevoted(N,R,V)) -> observed_prevoted(N,R,V);
assume well_behaved(N) -> old observed_prevoted(N,R,V) = observed_prevoted(N,R,V);
observed_precommitted(N,R,V) := *;
assume (old observed_precommitted(N,R,V)) -> observed_precommitted(N,R,V);
assume well_behaved(N) -> old observed_precommitted(N,R,V) = observed_precommitted(N,R,V);
}
}

143
spec/ivy-proofs/accountable_safety_1.ivy Normal file
View File

@@ -0,0 +1,143 @@
#lang ivy1.7
# ---
# layout: page
# title: Proof of Classic Safety
# ---
include tendermint
include abstract_tendermint
# Here we prove the first accountability property: if two well-behaved nodes
# disagree, then there are two quorums Q1 and Q2 such that all members of the
# intersection of Q1 and Q2 have violated the accountability properties.
# The proof is done in two steps: first we prove the abstract specification
# satisfies the property, and then we show by refinement that this property
# also holds in the concrete specification.
# To see what is checked in the refinement proof, use `ivy_show isolate=accountable_safety_1 accountable_safety_1.ivy`
# To see what is checked in the abstract correctness proof, use `ivy_show isolate=abstract_accountable_safety_1 accountable_safety_1.ivy`
# To check the whole proof, use `ivy_check accountable_safety_1.ivy`.
# Proof of the accountability property in the abstract specification
# ==================================================================
# We prove with tactics (see `lemma_1` and `lemma_2`) that, if some basic
# invariants hold (see `invs` below), then the accountability property holds.
isolate abstract_accountable_safety = {
instantiate abstract_tendermint
# The main property
# -----------------
# If there is disagreement, then there is evidence that a third of the nodes
# have violated the protocol:
invariant [accountability] agreement | accountability_violation
proof {
apply lemma_1.thm # this reduces to goal to three subgoals: p1, p2, and p3 (see their definition below)
proof [p1] {
assume invs.inv1
}
proof [p2] {
assume invs.inv2
}
proof [p3] {
assume invs.inv3
}
}
# The invariants
# --------------
isolate invs = {
# well-behaved nodes observe their own actions faithfully:
invariant [inv1] well_behaved(N) -> (observed_precommitted(N,R,V) = precommitted(N,R,V))
# if a value is precommitted by a well-behaved node, then a quorum is observed to prevote it:
invariant [inv2] (exists N . well_behaved(N) & precommitted(N,R,V)) & V ~= value.nil -> exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_prevoted(N2,R,V)
# if a value is decided by a well-behaved node, then a quorum is observed to precommit it:
invariant [inv3] (exists N . well_behaved(N) & decided(N,R,V)) -> 0 <= R & V ~= value.nil & exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_precommitted(N2,R,V)
private {
invariant (precommitted(N,R,V) | prevoted(N,R,V)) -> 0 <= R
invariant R < 0 -> left_round(N,R)
}
} with this, nset, round, accountable_bft.max_2f_byzantine
# The theorems proved with tactics
# --------------------------------
# Using complete induction on rounds, we prove that, assuming that the
# invariants inv1, inv2, and inv3 hold, the accountability property holds.
# For technical reasons, we separate the proof in two steps
isolate lemma_1 = {
specification {
theorem [thm] {
property [p1] forall N,R,V . well_behaved(N) -> (observed_precommitted(N,R,V) = precommitted(N,R,V))
property [p2] forall R,V . (exists N . well_behaved(N) & precommitted(N,R,V)) & V ~= value.nil -> exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_prevoted(N2,R,V)
property [p3] forall R,V. (exists N . well_behaved(N) & decided(N,R,V)) -> 0 <= R & V ~= value.nil & exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_precommitted(N2,R,V)
#-------------------------------------------------------------------------------------------------------------------------------------------
property agreement | accountability_violation
}
proof {
assume inductive_property # the theorem follows from what we prove by induction below
}
}
implementation {
# complete induction is not built-in, so we introduce it with an axiom. Note that this only holds for a type where 0 is the smallest element
axiom [complete_induction] {
relation p(X:round)
{ # base case
property p(0)
}
{ # inductive step: show that if the property is true for all X lower or equal to x and y=x+1, then the property is true of y
individual a:round
individual b:round
property (forall X. 0 <= X & X <= a -> p(X)) & round.succ(a,b) -> p(b)
}
#--------------------------
property forall X . 0 <= X -> p(X)
}
# The main lemma: if inv1 and inv2 below hold and a quorum is observed to
# precommit V1 at R1 and another quorum is observed to precommit V2~=V1 at
# R2>=R1, then the intersection of two quorums (i.e. f+1 nodes) is observed to
# violate the protocol. We prove this by complete induction on R2.
theorem [inductive_property] {
property [p1] forall N,R,V . well_behaved(N) -> (observed_precommitted(N,R,V) = precommitted(N,R,V))
property [p2] forall R,V . (exists N . well_behaved(N) & precommitted(N,R,V)) -> V = value.nil | exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_prevoted(N2,R,V)
#-----------------------------------------------------------------------------------------------------------------------
property forall R2. 0 <= R2 -> ((exists V2,Q1,R1,V1,Q1 . V1 ~= value.nil & V2 ~= value.nil & V1 ~= V2 & 0 <= R1 & R1 <= R2 & nset.is_quorum(Q1) & (forall N . nset.member(N,Q1) -> observed_precommitted(N,R1,V1)) & (exists Q2 . nset.is_quorum(Q2) & forall N . nset.member(N,Q2) -> observed_prevoted(N,R2,V2))) -> accountability_violation)
}
proof {
apply complete_induction # the two subgoals (base case and inductive case) are then discharged automatically
# NOTE: this can take a long time depending on the SMT random seed (to try a different seed, use `ivy_check seed=$RANDOM`
}
}
} with this, round, nset, accountable_bft.max_2f_byzantine, defs.observed_equivocation_def, defs.observed_unlawful_prevote_def, defs.accountability_violation_def, defs.agreement_def
} with round
# The final proof
# ===============
isolate accountable_safety_1 = {
# First we instantiate the concrete protocol:
instantiate tendermint(abstract_accountable_safety)
# We then define what we mean by agreement
relation agreement
definition [agreement_def] agreement = forall N1,N2. well_behaved(N1) & well_behaved(N2) & server.decision(N1) ~= value.nil & server.decision(N2) ~= value.nil -> server.decision(N1) = server.decision(N2)
invariant abstract_accountable_safety.agreement -> agreement
invariant [accountability] agreement | abstract_accountable_safety.accountability_violation
} with value, round, proposers, shim, abstract_accountable_safety, abstract_accountable_safety.defs.agreement_def, accountable_safety_1.agreement_def

52
spec/ivy-proofs/accountable_safety_2.ivy Normal file
View File

@@ -0,0 +1,52 @@
#lang ivy1.7
include tendermint
include abstract_tendermint
# Here we prove the second accountability property: no well-behaved node is
# ever observed to violate the accountability properties.
# The proof is done in two steps: first we prove the the abstract specification
# satisfies the property, and then we show by refinement that this property
# also holds in the concrete specification.
# To see what is checked in the refinement proof, use `ivy_show isolate=accountable_safety_2 accountable_safety_2.ivy`
# To see what is checked in the abstract correctness proof, use `ivy_show isolate=abstract_accountable_safety_2 accountable_safety_2.ivy`
# To check the whole proof, use `ivy_check complete=fo accountable_safety_2.ivy`.
# Proof that the property holds in the abstract specification
# ============================================================
isolate abstract_accountable_safety_2 = {
instantiate abstract_tendermint
# the main property:
invariant [wb_never_punished] well_behaved(N) -> ~(observed_equivocation(N) | observed_unlawful_prevote(N))
# the main invariant for proving wb_not_punished:
invariant well_behaved(N) & precommitted(N,R,V) & ~locked(N,R,V) & V ~= value.nil -> exists R2,V2 . V2 ~= value.nil & R < R2 & precommitted(N,R2,V2) & locked(N,R2,V2)
invariant (exists N . well_behaved(N) & precommitted(N,R,V) & V ~= value.nil) -> exists Q . nset.is_quorum(Q) & forall N . nset.member(N,Q) -> observed_prevoted(N,R,V)
invariant well_behaved(N) -> (observed_prevoted(N,R,V) <-> prevoted(N,R,V))
invariant well_behaved(N) -> (observed_precommitted(N,R,V) <-> precommitted(N,R,V))
# nodes stop prevoting or precommitting in lower rounds when doing so in a higher round:
invariant well_behaved(N) & prevoted(N,R2,V2) & R1 < R2 -> left_round(N,R1)
invariant well_behaved(N) & locked(N,R2,V2) & R1 < R2 -> left_round(N,R1)
invariant [precommit_unique_per_round] well_behaved(N) & precommitted(N,R,V1) & precommitted(N,R,V2) -> V1 = V2
} with nset, round, abstract_accountable_safety_2.defs.observed_equivocation_def, abstract_accountable_safety_2.defs.observed_unlawful_prevote_def
# Proof that the property holds in the concrete specification
# ===========================================================
isolate accountable_safety_2 = {
instantiate tendermint(abstract_accountable_safety_2)
invariant well_behaved(N) -> ~(abstract_accountable_safety_2.observed_equivocation(N) | abstract_accountable_safety_2.observed_unlawful_prevote(N))
} with round, value, shim, abstract_accountable_safety_2, abstract_accountable_safety_2.defs.observed_equivocation_def, abstract_accountable_safety_2.defs.observed_unlawful_prevote_def

39
spec/ivy-proofs/check_proofs.sh Executable file
View File

@@ -0,0 +1,39 @@
#!/bin/bash
# returns non-zero error code if any proof fails
success=0
log_dir=$(cat /dev/urandom | tr -cd 'a-f0-9' | head -c 6)
cmd="ivy_check seed=$RANDOM"
mkdir -p output/$log_dir
echo "Checking classic safety:"
res=$($cmd classic_safety.ivy | tee "output/$log_dir/classic_safety.txt" | tail -n 1)
if [ "$res" = "OK" ]; then
echo "OK"
else
echo "FAILED"
success=1
fi
echo "Checking accountable safety 1:"
res=$($cmd accountable_safety_1.ivy | tee "output/$log_dir/accountable_safety_1.txt" | tail -n 1)
if [ "$res" = "OK" ]; then
echo "OK"
else
echo "FAILED"
success=1
fi
echo "Checking accountable safety 2:"
res=$($cmd complete=fo accountable_safety_2.ivy | tee "output/$log_dir/accountable_safety_2.txt" | tail -n 1)
if [ "$res" = "OK" ]; then
echo "OK"
else
echo "FAILED"
success=1
fi
echo
echo "See ivy_check output in the output/ folder"
exit $success

85
spec/ivy-proofs/classic_safety.ivy Normal file
View File

@@ -0,0 +1,85 @@
#lang ivy1.7
# ---
# layout: page
# title: Proof of Classic Safety
# ---
include tendermint
include abstract_tendermint
# Here we prove the classic safety property: assuming that every two quorums
# have a well-behaved node in common, no two well-behaved nodes ever disagree.
# The proof is done in two steps: first we prove the the abstract specification
# satisfies the property, and then we show by refinement that this property
# also holds in the concrete specification.
# To see what is checked in the refinement proof, use `ivy_show isolate=classic_safety classic_safety.ivy`
# To see what is checked in the abstract correctness proof, use `ivy_show isolate=abstract_classic_safety classic_safety.ivy`
# To check the whole proof, use `ivy_check classic_safety.ivy`.
# Note that all the verification conditions sent to Z3 for this proof are in
# EPR.
# Classic safety in the abstract model
# ====================================
# We start by proving that classic safety holds in the abstract model.
isolate abstract_classic_safety = {
instantiate abstract_tendermint
invariant [classic_safety] classic_bft.quorum_intersection & decided(N1,R1,V1) & decided(N2,R2,V2) -> V1 = V2
# The notion of choosable value
# -----------------------------
relation choosable(R:round, V:value)
definition choosable(R,V) = exists Q . nset.is_quorum(Q) & forall N . well_behaved(N) & nset.member(N,Q) -> ~left_round(N,R) | precommitted(N,R,V)
# Main invariants
# ---------------
# `classic_safety` is inductive relative to those invariants
invariant [decision_is_quorum_precommit] (exists N1 . decided(N1,R,V)) -> exists Q. nset.is_quorum(Q) & forall N2. well_behaved(N2) & nset.member(N2, Q) -> precommitted(N2,R,V)
invariant [precommitted_is_quorum_prevote] V ~= value.nil & (exists N1 . precommitted(N1,R,V)) -> exists Q. nset.is_quorum(Q) & forall N2. well_behaved(N2) & nset.member(N2, Q) -> prevoted(N2,R,V)
invariant [prevote_unique_per_round] prevoted(N,R,V1) & prevoted(N,R,V2) -> V1 = V2
# This is the core invariant: as long as a precommitted value is still choosable, it remains protected by a lock and prevents any new value from being prevoted:
invariant [locks] classic_bft.quorum_intersection & V ~= value.nil & precommitted(N,R,V) & choosable(R,V) -> locked(N,R,V) & forall R2,V2 . R < R2 & prevoted(N,R2,V2) -> V2 = V | V2 = value.nil
# Supporting invariants
# ---------------------
# The main invariants are inductive relative to those
invariant decided(N,R,V) -> V ~= value.nil
invariant left_round(N,R2) & R1 < R2 -> left_round(N,R1) # if a node left round R2>R1, then it also left R1:
invariant prevoted(N,R2,V2) & R1 < R2 -> left_round(N,R1)
invariant precommitted(N,R2,V2) & R1 < R2 -> left_round(N,R1)
} with round, nset, classic_bft.quorum_intersection_def
# The refinement proof
# ====================
# Now, thanks to the refinement relation that we establish in
# `concrete_tendermint.ivy`, we prove that classic safety transfers to the
# concrete specification:
isolate classic_safety = {
# We instantiate the `tendermint` module providing `abstract_classic_safety` as abstract model.
instantiate tendermint(abstract_classic_safety)
# We prove that if every two quorums have a well-behaved node in common,
# then well-behaved nodes never disagree:
invariant [classic_safety] classic_bft.quorum_intersection & server.decision(N1) ~= value.nil & server.decision(N2) ~= value.nil -> server.decision(N1) = server.decision(N2)
} with value, round, proposers, shim, abstract_classic_safety # here we list all the specifications that we rely on for this proof

13
spec/ivy-proofs/count_lines.sh Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/bash
r='^\s*$\|^\s*\#\|^\s*\}\s*$\|^\s*{\s*$' # removes comments and blank lines and lines that contain only { or }
N1=`cat tendermint.ivy domain_model.ivy network_shim.ivy | grep -v $r'\|.*invariant.*' | wc -l`
N2=`cat abstract_tendermint.ivy | grep "observed_" | wc -l` # the observed_* variables specify the observations of the nodes
SPEC_LINES=`expr $N1 + $N2`
echo "spec lines: $SPEC_LINES"
N3=`cat abstract_tendermint.ivy | grep -v $r'\|.*observed_.*' | wc -l`
N4=`cat accountable_safety_1.ivy | grep -v $r | wc -l`
PROOF_LINES=`expr $N3 + $N4`
echo "proof lines: $PROOF_LINES"
RATIO=`bc <<< "scale=2;$PROOF_LINES / $SPEC_LINES"`
echo "proof-to-code ratio for the accountable-safety property: $RATIO"

8
spec/ivy-proofs/docker-compose.yml Normal file
View File

@@ -0,0 +1,8 @@
version: '3'
services:
tendermint-proof:
build: .
volumes:
- ./:/home/user/tendermint-proof:ro
- ./output:/home/user/tendermint-proof/output:rw

143
spec/ivy-proofs/domain_model.ivy Normal file
View File

@@ -0,0 +1,143 @@
#lang ivy1.7
include order # this is a file from the standard library (`ivy/ivy/include/1.7/order.ivy`)
isolate round = {
type this
individual minus_one:this
relation succ(R1:round, R2:round)
action incr(i:this) returns (j:this)
specification {
# to simplify verification, we treat rounds as an abstract totally ordered set with a successor relation.
instantiate totally_ordered(this)
property minus_one < 0
property succ(X,Z) -> (X < Z & ~(X < Y & Y < Z))
after incr {
ensure succ(i,j)
}
}
implementation {
# here we prove that the abstraction is sound.
interpret this -> int # rounds are integers in the Tendermint specification.
definition minus_one = 0-1
definition succ(R1,R2) = R2 = R1 + 1
implement incr {
j := i+1;
}
}
}
instance node : iterable # nodes are a set with an order, that can be iterated over (see order.ivy in the standard library)
relation well_behaved(N:node) # whether a node is well-behaved or not. NOTE: Used only in the proof and the Byzantine model; Nodes do know know who is well-behaved and who is not.
isolate proposers = {
# each round has a unique proposer in Tendermint. In order to avoid a
# function from round to node (which makes verification more difficult), we
# abstract over this function using a relation.
relation is_proposer(N:node, R:round)
export action get_proposer(r:round) returns (n:node)
specification {
property is_proposer(N1,R) & is_proposer(N2,R) -> N1 = N2
after get_proposer {
ensure is_proposer(n,r);
}
}
implementation {
function f(R:round):node
definition f(r:round) = <<<r % `node.size`>>>
definition is_proposer(N,R) = N = f(R)
implement get_proposer {
n := f(r);
}
}
}
isolate value = { # the type of values
type this
relation valid(V:value)
individual nil:value
specification {
property ~valid(nil)
}
implementation {
interpret value -> bv[2]
definition nil = <<< -1 >>> # let's say nil is -1
definition valid(V) = V ~= nil
}
}
object nset = { # the type of node sets
type this # a set of N=3f+i nodes for 0<i<=3
relation member(N:node, S:nset) # set-membership relation
relation is_quorum(S:nset) # intent: sets of cardinality at least 2f+i+1
relation is_blocking(S:nset) # intent: at least f+1 nodes
export action insert(s:nset, n:node) returns (t:nset)
export action empty returns (s:nset)
implementation {
# NOTE: this is not checked at all by Ivy; it is however useful to generate C++ code and run it for debugging purposes
<<< header
#include <set>
#include <exception>
namespace hash_space {
template <typename T>
class hash<std::set<T> > {
public:
size_t operator()(const std::set<T> &s) const {
hash<T> h;
size_t res = 0;
for (const T &e : s)
res += h(e);
return res;
}
};
}
>>>
interpret nset -> <<< std::set<`node`> >>>
definition member(n:node, s:nset) = <<< `s`.find(`n`) != `s`.end() >>>
definition is_quorum(s:nset) = <<< 3*`s`.size() > 2*`node.size` >>>
definition is_blocking(s:nset) = <<< 3*`s`.size() > `node.size` >>>
implement empty {
<<<
>>>
}
implement insert {
<<<
`t` = `s`;
`t`.insert(`n`);
>>>
}
<<< encode `nset`
std::ostream &operator <<(std::ostream &s, const `nset` &a) {
s << "{";
for (auto iter = a.begin(); iter != a.end(); iter++) {
if (iter != a.begin()) s << ", ";
s << *iter;
}
s << "}";
return s;
}
template <>
`nset` _arg<`nset`>(std::vector<ivy_value> &args, unsigned idx, long long bound) {
throw std::invalid_argument("Not implemented"); // no syntax for nset values in the REPL
}
>>>
}
}
object classic_bft = {
relation quorum_intersection
private {
definition [quorum_intersection_def] quorum_intersection = forall Q1,Q2. exists N. well_behaved(N) & nset.member(N, Q1) & nset.member(N, Q2) # every two quorums have a well-behaved node in common
}
}
trusted isolate accountable_bft = {
# this is our baseline assumption about quorums:
private {
property [max_2f_byzantine] exists N . well_behaved(N) & nset.member(N,Q) # every quorum has a well-behaved member
}
}

133
spec/ivy-proofs/network_shim.ivy Normal file
View File

@@ -0,0 +1,133 @@
#lang ivy1.7
# ---
# layout: page
# title: Network model and network shim
# ---
# Here we define a network module, which is our model of the network, and a
# shim module that sits on top of the network and which, upon receiving a
# message, calls the appropriate protocol handler.
include domain_model
# Here we define an enumeration type for identifying the 3 different types of
# messages that nodes send.
object msg_kind = { # TODO: merge with step_t
type this = {proposal, prevote, precommit}
}
# Here we define the type of messages `msg`. Its members are structs with the fields described below.
object msg = {
type this = struct {
m_kind : msg_kind,
m_src : node,
m_round : round,
m_value : value,
m_vround : round
}
}
# This is our model of the network:
isolate net = {
export action recv(dst:node,v:msg)
action send(src:node,dst:node,v:msg)
# Note that the `recv` action is exported, meaning that it can be called
# non-deterministically by the environment any time it is enabled. In other
# words, a packet that is in flight can be received at any time. In this
# sense, the network is fully asynchronous. Moreover, there is no
# requirement that a given message will be received at all.
# The state of the network consists of all the packets that have been
# sent so far, along with their destination.
relation sent(V:msg, N:node)
after init {
sent(V, N) := false
}
before send {
sent(v,dst) := true
}
before recv {
require sent(v,dst) # only sent messages can be received.
}
}
# The network shim sits on top of the network and, upon receiving a message,
# calls the appropriate protocol handler. It also exposes a `broadcast` action
# that sends to all nodes.
isolate shim = {
# In order not repeat the same code for each handler, we use a handler
# module parameterized by the type of message it will handle. Below we
# instantiate this module for the 3 types of messages of Tendermint
module handler(p_kind) = {
action handle(dst:node,m:msg)
object spec = {
before handle {
assert sent(m,dst) & m.m_kind = p_kind
}
}
}
instance proposal_handler : handler(msg_kind.proposal)
instance prevote_handler : handler(msg_kind.prevote)
instance precommit_handler : handler(msg_kind.precommit)
relation sent(M:msg,N:node)
action broadcast(src:node,m:msg)
action send(src:node,dst:node,m:msg)
specification {
after init {
sent(M,D) := false;
}
before broadcast {
sent(m,D) := true
}
before send {
sent(m,dst) := true
}
}
# Here we give an implementation of it that satisfies its specification:
implementation {
implement net.recv(dst:node,m:msg) {
if m.m_kind = msg_kind.proposal {
call proposal_handler.handle(dst,m)
}
else if m.m_kind = msg_kind.prevote {
call prevote_handler.handle(dst,m)
}
else if m.m_kind = msg_kind.precommit {
call precommit_handler.handle(dst,m)
}
}
implement broadcast { # broadcast sends to all nodes, including the sender.
var iter := node.iter.create(0);
while ~iter.is_end
invariant net.sent(M,D) -> sent(M,D)
{
var n := iter.val;
call net.send(src,n,m);
iter := iter.next;
}
}
implement send {
call net.send(src,dst,m)
}
private {
invariant net.sent(M,D) -> sent(M,D)
}
}
} with net, node # to prove that the shim implementation satisfies the shim specification, we rely on the specification of net and node.

4
spec/ivy-proofs/output/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
# Ignore everything in this directory
*
# Except this file
!.gitignore

420
spec/ivy-proofs/tendermint.ivy Normal file
View File

@@ -0,0 +1,420 @@
#lang ivy1.7
# ---
# layout: page
# title: Specification of Tendermint in Ivy
# ---
# This specification closely follows the pseudo-code given in "The latest
# gossip on BFT consensus" by E. Buchman, J. Kwon, Z. Milosevic
# <https://arxiv.org/abs/1807.04938>
include domain_model
include network_shim
# We model the Tendermint protocol as an Ivy object. Like in Object-Oriented
# Programming, the basic structuring unit in Ivy is the object. Objects have
# internal state and actions (i.e. methods in OO parlance) that modify their
# state. We model Tendermint as an object whose actions represent steps taken
# by individual nodes in the protocol. Actions in Ivy can have preconditions,
# and a valid execution is a sequence of actions whose preconditions are all
# satisfied in the state in which they are called.
# For technical reasons, we define below a `tendermint` module instead of an
# object. Ivy modules are a little bit like classes in OO programs, and like
# classes they can be instantiated to obtain objects. To instantiate the
# `tendermint` module, we must provide an abstract-protocol object. This allows
# us to use different abstract-protocol objects for different parts of the
# proof, and to do so without too much notational burden (we could have used
# Ivy monitors, but then we would need to prefix every variable name by the
# name of the object containing it, which clutters things a bit compared to the
# approach we took).
# The abstract-protocol object is called by the resulting tendermint object so
# as to run the abstract protocol alongside the concrete protocol. This allows
# us to transfer properties proved of the abstract protocol to the concrete
# protocol, as follows. First, we prove that running the abstract protocol in
# this way results in a valid execution of the abstract protocol. This is done
# by checking that all preconditions of the abstract actions are satisfied at
# their call sites. Second, we establish a relation between abstract state and
# concrete state (in the form of invariants of the resulting, two-object
# transition system) that allow us to transfer properties proved in the
# abstract protocol to the concrete protocol (for example, we prove that any
# decision made in the Tendermint protocol is also made in the abstract
# protocol; if the abstract protocol satisfies the agreement property, this
# allows us to conclude that the Tendermint protocol also does).
# The abstract protocol object that we will use is always the same, and only
# the abstract properties that we prove about it change in the different
# instantiations of the `tendermint` module. Thus we provide common invariants
# that a) allow to prove that the abstract preconditions are met, and b)
# provide a refinement relation (see end of the module) relating the state of
# Tendermint to the state of the abstract protocol.
# In the model, Byzantine nodes can send whatever messages they want, except
# that they cannot forge sender identities. This reflects the fact that, in
# practice, nodes use public key cryptography to sign their messages.
# Finally, note that the observations that serve to adjudicate misbehavior are
# defined only in the abstract protocol (they happen in the abstract actions).
module tendermint(abstract_protocol) = {
# the initial value of a node:
function init_val(N:node): value
# the three type of steps
object step_t = {
type this = {propose, prevote, precommit}
} # refer to those e.g. as step_t.propose
object server(n:node) = {
# the current round of a node
individual round_p: round
individual step: step_t
individual decision: value
individual lockedValue: value
individual lockedRound: round
individual validValue: value
individual validRound: round
relation done_l34(R:round)
relation done_l36(R:round, V:value)
relation done_l47(R:round)
# variables for scheduling request
relation propose_timer_scheduled(R:round)
relation prevote_timer_scheduled(R:round)
relation precommit_timer_scheduled(R:round)
relation _recved_proposal(Sender:node, R:round, V:value, VR:round)
relation _recved_prevote(Sender:node, R:round, V:value)
relation _recved_precommit(Sender:node, R:round, V:value)
relation _has_started
after init {
round_p := 0;
step := step_t.propose;
decision := value.nil;
lockedValue := value.nil;
lockedRound := round.minus_one;
validValue := value.nil;
validRound := round.minus_one;
done_l34(R) := false;
done_l36(R, V) := false;
done_l47(R) := false;
propose_timer_scheduled(R) := false;
prevote_timer_scheduled(R) := false;
precommit_timer_scheduled(R) := false;
_recved_proposal(Sender, R, V, VR) := false;
_recved_prevote(Sender, R, V) := false;
_recved_precommit(Sender, R, V) := false;
_has_started := false;
}
action getValue returns (v:value) = {
v := init_val(n)
}
export action start = {
require ~_has_started;
_has_started := true;
# line 10
call startRound(0);
}
# line 11-21
action startRound(r:round) = {
# line 12
round_p := r;
# line 13
step := step_t.propose;
var proposal : value;
# line 14
if (proposers.get_proposer(r) = n) {
if validValue ~= value.nil { # line 15
proposal := validValue; # line 16
} else {
proposal := getValue(); # line 18
};
call broadcast_proposal(r, proposal, validRound); # line 19
} else {
propose_timer_scheduled(r) := true; # line 21
};
call abstract_protocol.l_11(n, r);
}
# This action, as not exported, can only be called at specific call sites.
action broadcast_proposal(r:round, v:value, vr:round) = {
var m: msg;
m.m_kind := msg_kind.proposal;
m.m_src := n;
m.m_round := r;
m.m_value := v;
m.m_vround := vr;
call shim.broadcast(n,m);
}
implement shim.proposal_handler.handle(msg:msg) {
_recved_proposal(msg.m_src, msg.m_round, msg.m_value, msg.m_vround) := true;
}
# line 22-27
export action l_22(v:value) = {
require _has_started;
require _recved_proposal(proposers.get_proposer(round_p), round_p, v, round.minus_one);
require step = step_t.propose;
if (value.valid(v) & (lockedRound = round.minus_one | lockedValue = v)) {
call broadcast_prevote(round_p, v); # line 24
call abstract_protocol.l_22(n, round_p, v);
} else {
call broadcast_prevote(round_p, value.nil); # line 26
call abstract_protocol.l_22(n, round_p, value.nil);
};
# line 27
step := step_t.prevote;
}
# line 28-33
export action l_28(r:round, v:value, vr:round, q:nset) = {
require _has_started;
require r = round_p;
require _recved_proposal(proposers.get_proposer(r), r, v, vr);
require nset.is_quorum(q);
require nset.member(N,q) -> _recved_prevote(N,vr,v);
require step = step_t.propose;
require vr >= 0 & vr < r;
# line 29
if (value.valid(v) & (lockedRound <= vr | lockedValue = v)) {
call broadcast_prevote(r, v);
} else {
call broadcast_prevote(r, value.nil);
};
call abstract_protocol.l_28(n,r,v,vr,q);
step := step_t.prevote;
}
action broadcast_prevote(r:round, v:value) = {
var m: msg;
m.m_kind := msg_kind.prevote;
m.m_src := n;
m.m_round := r;
m.m_value := v;
call shim.broadcast(n,m);
}
implement shim.prevote_handler.handle(msg:msg) {
_recved_prevote(msg.m_src, msg.m_round, msg.m_value) := true;
}
# line 34-35
export action l_34(r:round, q:nset) = {
require _has_started;
require round_p = r;
require nset.is_quorum(q);
require exists V . nset.member(N,q) -> _recved_prevote(N,r,V);
require step = step_t.prevote;
require ~done_l34(r);
done_l34(r) := true;
prevote_timer_scheduled(r) := true;
}
# line 36-43
export action l_36(r:round, v:value, q:nset) = {
require _has_started;
require r = round_p;
require exists VR . round.minus_one <= VR & VR < r & _recved_proposal(proposers.get_proposer(r), r, v, VR);
require nset.is_quorum(q);
require nset.member(N,q) -> _recved_prevote(N,r,v);
require value.valid(v);
require step = step_t.prevote | step = step_t.precommit;
require ~done_l36(r,v);
done_l36(r, v) := true;
if step = step_t.prevote {
lockedValue := v; # line 38
lockedRound := r; # line 39
call broadcast_precommit(r, v); # line 40
step := step_t.precommit; # line 41
call abstract_protocol.l_36(n, r, v, q);
};
validValue := v; # line 42
validRound := r; # line 43
}
# line 44-46
export action l_44(r:round, q:nset) = {
require _has_started;
require r = round_p;
require nset.is_quorum(q);
require nset.member(N,q) -> _recved_prevote(N,r,value.nil);
require step = step_t.prevote;
call broadcast_precommit(r, value.nil); # line 45
step := step_t.precommit; # line 46
call abstract_protocol.l_44(n, r, q);
}
action broadcast_precommit(r:round, v:value) = {
var m: msg;
m.m_kind := msg_kind.precommit;
m.m_src := n;
m.m_round := r;
m.m_value := v;
call shim.broadcast(n,m);
}
implement shim.precommit_handler.handle(msg:msg) {
_recved_precommit(msg.m_src, msg.m_round, msg.m_value) := true;
}
# line 47-48
export action l_47(r:round, q:nset) = {
require _has_started;
require round_p = r;
require nset.is_quorum(q);
require nset.member(N,q) -> exists V . _recved_precommit(N,r,V);
require ~done_l47(r);
done_l47(r) := true;
precommit_timer_scheduled(r) := true;
}
# line 49-54
export action l_49_decide(r:round, v:value, q:nset) = {
require _has_started;
require exists VR . round.minus_one <= VR & VR < r & _recved_proposal(proposers.get_proposer(r), r, v, VR);
require nset.is_quorum(q);
require nset.member(N,q) -> _recved_precommit(N,r,v);
require decision = value.nil;
if value.valid(v) {
decision := v;
# MORE for next height
call abstract_protocol.decide(n, r, v, q);
}
}
# line 55-56
export action l_55(r:round, b:nset) = {
require _has_started;
require nset.is_blocking(b);
require nset.member(N,b) -> exists VR . round.minus_one <= VR & VR < r & exists V . _recved_proposal(N,r,V,VR) | _recved_prevote(N,r,V) | _recved_precommit(N,r,V);
require r > round_p;
call startRound(r); # line 56
}
# line 57-60
export action onTimeoutPropose(r:round) = {
require _has_started;
require propose_timer_scheduled(r);
require r = round_p;
require step = step_t.propose;
call broadcast_prevote(r,value.nil);
step := step_t.prevote;
call abstract_protocol.l_57(n,r);
propose_timer_scheduled(r) := false;
}
# line 61-64
export action onTimeoutPrevote(r:round) = {
require _has_started;
require prevote_timer_scheduled(r);
require r = round_p;
require step = step_t.prevote;
call broadcast_precommit(r,value.nil);
step := step_t.precommit;
call abstract_protocol.l_61(n,r);
prevote_timer_scheduled(r) := false;
}
# line 65-67
export action onTimeoutPrecommit(r:round) = {
require _has_started;
require precommit_timer_scheduled(r);
require r = round_p;
call startRound(round.incr(r));
precommit_timer_scheduled(r) := false;
}
# The Byzantine actions
# ---------------------
# Byzantine nodes can send whatever they want, but they cannot send
# messages on behalf of well-behaved nodes. In practice this is implemented
# using cryptography (e.g. public-key cryptography).
export action byzantine_send(m:msg, dst:node) = {
require ~well_behaved(n);
require ~well_behaved(m.m_src); # cannot forge the identity of well-behaved nodes
call shim.send(n,dst,m);
}
# Byzantine nodes can also report fake observations, as defined in the abstract protocol.
export action fake_observations = {
call abstract_protocol.misbehave
}
# Invariants
# ----------
# We provide common invariants that a) allow to prove that the abstract
# preconditions are met, and b) provide a refinement relation.
specification {
invariant 0 <= round_p
invariant abstract_protocol.left_round(n,R) <-> R < round_p
invariant lockedRound ~= round.minus_one -> forall R,V . abstract_protocol.locked(n,R,V) <-> R <= lockedRound & lockedValue = V
invariant lockedRound = round.minus_one -> forall R,V . ~abstract_protocol.locked(n,R,V)
invariant forall M:msg . well_behaved(M.m_src) & M.m_kind = msg_kind.prevote & shim.sent(M,N) -> abstract_protocol.prevoted(M.m_src,M.m_round,M.m_value)
invariant well_behaved(N) & _recved_prevote(N,R,V) -> abstract_protocol.prevoted(N,R,V)
invariant forall M:msg . well_behaved(M.m_src) & M.m_kind = msg_kind.precommit & shim.sent(M,N) -> abstract_protocol.precommitted(M.m_src,M.m_round,M.m_value)
invariant well_behaved(N) & _recved_precommit(N,R,V) -> abstract_protocol.precommitted(N,R,V)
invariant (step = step_t.prevote | step = step_t.propose) -> ~abstract_protocol.precommitted(n,round_p,V)
invariant step = step_t.propose -> ~abstract_protocol.prevoted(n,round_p,V)
invariant step = step_t.prevote -> exists V . abstract_protocol.prevoted(n,round_p,V)
invariant round_p < R -> ~(abstract_protocol.prevoted(n,R,V) | abstract_protocol.precommitted(n,R,V))
invariant ~_has_started -> step = step_t.propose & ~(abstract_protocol.prevoted(n,R,V) | abstract_protocol.precommitted(n,R,V)) & round_p = 0
invariant decision ~= value.nil -> exists R . abstract_protocol.decided(n,R,decision)
}
}
}

127
spec/ivy-proofs/tendermint_test.ivy Normal file
View File

@@ -0,0 +1,127 @@
#lang ivy1.7
include tendermint
include abstract_tendermint
isolate ghost_ = {
instantiate abstract_tendermint
}
isolate protocol = {
instantiate tendermint(ghost_) # here we instantiate the parameter of the tendermint module with `ghost_`; however note that we don't extract any code for `ghost_` (it's not in the list of object in the extract, and it's thus sliced away).
implementation {
definition init_val(n:node) = <<< `n`%2 >>>
}
# attribute test = impl
} with ghost_, shim, value, round, proposers
# Here we run a simple scenario that exhibits an execution in which nodes make
# a decision. We do this to rule out trivial modeling errors.
# One option to check that this scenario is valid is to run it in Ivy's REPL.
# For this, first compile the scenario:
#```ivyc target=repl isolate=code trace=true tendermint_test.ivy
# Then, run the produced binary (e.g. for 4 nodes):
#``` ./tendermint_test 4
# Finally, call the action:
#``` scenarios.scenario_1
# Note that Ivy will check at runtime that all action preconditions are
# satisfied. For example, runing the scenario twice will cause a violation of
# the precondition of the `start` action, because a node cannot start twice
# (see `require ~_has_started` in action `start`).
# Another possibility would be to run `ivy_check` on the scenario, but that
# does not seem to work at the moment.
isolate scenarios = {
individual all:nset # will be used as parameter to actions requiring a quorum
after init {
var iter := node.iter.create(0);
while ~iter.is_end
{
all := all.insert(iter.val);
iter := iter.next;
};
assert nset.is_quorum(all); # we can also use asserts to make sure we are getting what we expect
}
export action scenario_1 = {
# all nodes start:
var iter := node.iter.create(0);
while ~iter.is_end
{
call protocol.server.start(iter.val);
iter := iter.next;
};
# all nodes receive the leader's proposal:
var m:msg;
m.m_kind := msg_kind.proposal;
m.m_src := 0;
m.m_round := 0;
m.m_value := 0;
m.m_vround := round.minus_one;
iter := node.iter.create(0);
while ~iter.is_end
{
call net.recv(iter.val,m);
iter := iter.next;
};
# all nodes prevote:
iter := node.iter.create(0);
while ~iter.is_end
{
call protocol.server.l_22(iter.val,0);
iter := iter.next;
};
# all nodes receive each other's prevote messages;
m.m_kind := msg_kind.prevote;
m.m_vround := 0;
iter := node.iter.create(0);
while ~iter.is_end
{
var iter2 := node.iter.create(0); # the sender
while ~iter2.is_end
{
m.m_src := iter2.val;
call net.recv(iter.val,m);
iter2 := iter2.next;
};
iter := iter.next;
};
# all nodes precommit:
iter := node.iter.create(0);
while ~iter.is_end
{
call protocol.server.l_36(iter.val,0,0,all);
iter := iter.next;
};
# all nodes receive each other's pre-commits
m.m_kind := msg_kind.precommit;
iter := node.iter.create(0);
while ~iter.is_end
{
var iter2 := node.iter.create(0); # the sender
while ~iter2.is_end
{
m.m_src := iter2.val;
call net.recv(iter.val,m);
iter2 := iter2.next;
};
iter := iter.next;
};
# now all nodes can decide:
iter := node.iter.create(0);
while ~iter.is_end
{
call protocol.server.l_49_decide(iter.val,0,0,all);
iter := iter.next;
};
}
# TODO: add more scenarios
} with round, node, proposers, value, nset, protocol, shim, net
# extract code = protocol, shim, round, node
extract code = round, node, proposers, value, nset, protocol, shim, net, scenarios

205
spec/light-client/README.md Normal file
View File

@@ -0,0 +1,205 @@
---
order: 1
parent:
title: Light Client
order: 5
---
# Light Client Specification
This directory contains work-in-progress English and TLA+ specifications for the Light Client
protocol. Implementations of the light client can be found in
[Rust](https://github.com/informalsystems/tendermint-rs/tree/master/light-client) and
[Go](https://github.com/tendermint/tendermint/tree/master/light).
Light clients are assumed to be initialized once from a trusted source
with a trusted header and validator set. The light client
protocol allows a client to then securely update its trusted state by requesting and
verifying a minimal set of data from a network of full nodes (at least one of which is correct).
The light client is decomposed into two main components:
- [Commit Verification](#Commit-Verification) - verify signed headers and associated validator
set changes from a single full node, called primary
- [Attack Detection](#Attack-Detection) - verify commits across multiple full nodes (called secondaries) and detect conflicts (ie. the existence of a lightclient attack)
In case a lightclient attack is detected, the lightclient submits evidence to a full node which is responsible for "accountability", that is, punishing attackers:
- [Accountability](#Accountability) - given evidence for an attack, compute a set of validators that are responsible for it.
## Commit Verification
The [English specification](verification/verification_001_published.md) describes the light client
commit verification problem in terms of the temporal properties
[LCV-DIST-SAFE.1](https://github.com/informalsystems/tendermint-rs/blob/master/docs/spec/lightclient/verification/verification_001_published.md#lcv-dist-safe1) and
[LCV-DIST-LIVE.1](https://github.com/informalsystems/tendermint-rs/blob/master/docs/spec/lightclient/verification/verification_001_published.md#lcv-dist-live1).
Commit verification is assumed to operate within the Tendermint Failure Model, where +2/3 of validators are correct for some time period and
validator sets can change arbitrarily at each height.
A light client protocol is also provided, including all checks that
need to be performed on headers, commits, and validator sets
to satisfy the temporal properties - so a light client can continuously
synchronize with a blockchain. Clients can skip possibly
many intermediate headers by exploiting overlap in trusted and untrusted validator sets.
When there is not enough overlap, a bisection routine can be used to find a
minimal set of headers that do provide the required overlap.
The [TLA+ specification ver. 001](verification/Lightclient_A_1.tla)
is a formal description of the
commit verification protocol executed by a client, including the safety and
termination, which can be model checked with Apalache.
A more detailed TLA+ specification of
[Light client verification ver. 003](verification/Lightclient_003_draft.tla)
is currently under peer review.
The `MC*.tla` files contain concrete parameters for the
[TLA+ specification](verification/Lightclient_A_1.tla), in order to do model checking.
For instance, [MC4_3_faulty.tla](verification/MC4_3_faulty.tla) contains the following parameters
for the nodes, heights, the trusting period, the clock drifts,
correctness of the primary node, and the ratio of the faulty processes:
```tla
AllNodes == {"n1", "n2", "n3", "n4"}
TRUSTED_HEIGHT == 1
TARGET_HEIGHT == 3
TRUSTING_PERIOD == 1400 \* the trusting period in some time units
CLOCK_DRIFT = 10 \* how much we assume the local clock is drifting
REAL_CLOCK_DRIFT = 3 \* how much the local clock is actually drifting
IS_PRIMARY_CORRECT == FALSE
FAULTY_RATIO == <<1, 3>> \* < 1 / 3 faulty validators
```
To run a complete set of experiments, clone [apalache](https://github.com/informalsystems/apalache) and [apalache-tests](https://github.com/informalsystems/apalache-tests) into a directory `$DIR` and run the following commands:
```sh
$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 002bmc-apalache-ok.csv $DIR/apalache . out
./out/run-all.sh
```
After the experiments have finished, you can collect the logs by executing the following command:
```sh
cd ./out
$DIR/apalache-tests/scripts/parse-logs.py --human .
```
All lines in `results.csv` should report `Deadlock`, which means that the algorithm
has terminated and no invariant violation was found.
Similar to [002bmc-apalache-ok.csv](verification/002bmc-apalache-ok.csv),
file [003bmc-apalache-error.csv](verification/003bmc-apalache-error.csv) specifies
the set of experiments that should result in counterexamples:
```sh
$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 003bmc-apalache-error.csv $DIR/apalache . out
./out/run-all.sh
```
All lines in `results.csv` should report `Error`.
The following table summarizes the experimental results for Light client verification
version 001. The TLA+ properties can be found in the
[TLA+ specification](verification/Lightclient_A_1.tla).
The experiments were run in an AWS instance equipped with 32GB
RAM and a 4-core Intel® Xeon® CPU E5-2686 v4 @ 2.30GHz CPU.
We write “✗=k” when a bug is reported at depth k, and “✓<=k” when
no bug is reported up to depth k.
![Experimental results](experiments.png)
The experimental results for version 003 are to be added.
## Attack Detection
The [English specification](detection/detection_003_reviewed.md)
defines light client attacks (and how they differ from blockchain
forks), and describes the problem of a light client detecting
these attacks by communicating with a network of full nodes,
where at least one is correct.
The specification also contains a detection protocol that checks
whether the header obtained from the primary via the verification
protocol matches corresponding headers provided by the secondaries.
If this is not the case, the protocol analyses the verification traces
of the involved full nodes
and generates
[evidence](detection/detection_003_reviewed.md#tmbc-lc-evidence-data1)
of misbehavior that can be submitted to a full node so that
the faulty validators can be punished.
The [TLA+ specification](detection/LCDetector_003_draft.tla)
is a formal description of the
detection protocol for two peers, including the safety and
termination, which can be model checked with Apalache.
The `LCD_MC*.tla` files contain concrete parameters for the
[TLA+ specification](detection/LCDetector_003_draft.tla),
in order to run the model checker.
For instance, [LCD_MC4_4_faulty.tla](detection/MC4_4_faulty.tla)
contains the following parameters
for the nodes, heights, the trusting period, the clock drifts,
correctness of the nodes, and the ratio of the faulty processes:
```tla
AllNodes == {"n1", "n2", "n3", "n4"}
TRUSTED_HEIGHT == 1
TARGET_HEIGHT == 3
TRUSTING_PERIOD == 1400 \* the trusting period in some time units
CLOCK_DRIFT = 10 \* how much we assume the local clock is drifting
REAL_CLOCK_DRIFT = 3 \* how much the local clock is actually drifting
IS_PRIMARY_CORRECT == FALSE
IS_SECONDARY_CORRECT == FALSE
FAULTY_RATIO == <<1, 3>> \* < 1 / 3 faulty validators
```
To run a complete set of experiments, clone [apalache](https://github.com/informalsystems/apalache) and [apalache-tests](https://github.com/informalsystems/apalache-tests) into a directory `$DIR` and run the following commands:
```sh
$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 004bmc-apalache-ok.csv $DIR/apalache . out
./out/run-all.sh
```
After the experiments have finished, you can collect the logs by executing the following command:
```sh
cd ./out
$DIR/apalache-tests/scripts/parse-logs.py --human .
```
All lines in `results.csv` should report `Deadlock`, which means that the algorithm
has terminated and no invariant violation was found.
Similar to [004bmc-apalache-ok.csv](verification/004bmc-apalache-ok.csv),
file [005bmc-apalache-error.csv](verification/005bmc-apalache-error.csv) specifies
the set of experiments that should result in counterexamples:
```sh
$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 005bmc-apalache-error.csv $DIR/apalache . out
./out/run-all.sh
```
All lines in `results.csv` should report `Error`.
The detailed experimental results are to be added soon.
## Accountability
The [English specification](attacks/isolate-attackers_002_reviewed.md)
defines the protocol that is executed on a full node upon receiving attack [evidence](detection/detection_003_reviewed.md#tmbc-lc-evidence-data1) from a lightclient. In particular, the protocol handles three types of attacks
- lunatic
- equivocation
- amnesia
We discussed in the [last part](attacks/isolate-attackers_002_reviewed.md#Part-III---Completeness) of the English specification
that the non-lunatic cases are defined by having the same validator set in the conflicting blocks. For these cases,
computer-aided analysis of [Tendermint Consensus in TLA+](./accountability/README.md) shows that equivocation and amnesia capture all non-lunatic attacks.
The [TLA+ specification](attacks/Isolation_001_draft.tla)
is a formal description of the
protocol, including the safety property, which can be model checked with Apalache.
Similar to the other specifications, [MC_5_3.tla](attacks/MC_5_3.tla) contains concrete parameters to run the model checker. The specification can be checked within seconds.
[tendermint-accountability](./accountability/README.md)

13
spec/light-client/accountability/001indinv-apalache.csv Normal file
View File

@@ -0,0 +1,13 @@
no,filename,tool,timeout,init,inv,next,args
1,MC_n4_f1.tla,apalache,10h,TypedInv,TypedInv,,--length=1 --cinit=ConstInit
2,MC_n4_f2.tla,apalache,10h,TypedInv,TypedInv,,--length=1 --cinit=ConstInit
3,MC_n5_f1.tla,apalache,10h,TypedInv,TypedInv,,--length=1 --cinit=ConstInit
4,MC_n5_f2.tla,apalache,10h,TypedInv,TypedInv,,--length=1 --cinit=ConstInit
5,MC_n4_f1.tla,apalache,20h,Init,TypedInv,,--length=0 --cinit=ConstInit
6,MC_n4_f2.tla,apalache,20h,Init,TypedInv,,--length=0 --cinit=ConstInit
7,MC_n5_f1.tla,apalache,20h,Init,TypedInv,,--length=0 --cinit=ConstInit
8,MC_n5_f2.tla,apalache,20h,Init,TypedInv,,--length=0 --cinit=ConstInit
9,MC_n4_f1.tla,apalache,20h,TypedInv,Agreement,,--length=0 --cinit=ConstInit
10,MC_n4_f2.tla,apalache,20h,TypedInv,Accountability,,--length=0 --cinit=ConstInit
11,MC_n5_f1.tla,apalache,20h,TypedInv,Agreement,,--length=0 --cinit=ConstInit
12,MC_n5_f2.tla,apalache,20h,TypedInv,Accountability,,--length=0 --cinit=ConstInit
1 no filename tool timeout init inv next args
2 1 MC_n4_f1.tla apalache 10h TypedInv TypedInv --length=1 --cinit=ConstInit
3 2 MC_n4_f2.tla apalache 10h TypedInv TypedInv --length=1 --cinit=ConstInit
4 3 MC_n5_f1.tla apalache 10h TypedInv TypedInv --length=1 --cinit=ConstInit
5 4 MC_n5_f2.tla apalache 10h TypedInv TypedInv --length=1 --cinit=ConstInit
6 5 MC_n4_f1.tla apalache 20h Init TypedInv --length=0 --cinit=ConstInit
7 6 MC_n4_f2.tla apalache 20h Init TypedInv --length=0 --cinit=ConstInit
8 7 MC_n5_f1.tla apalache 20h Init TypedInv --length=0 --cinit=ConstInit
9 8 MC_n5_f2.tla apalache 20h Init TypedInv --length=0 --cinit=ConstInit
10 9 MC_n4_f1.tla apalache 20h TypedInv Agreement --length=0 --cinit=ConstInit
11 10 MC_n4_f2.tla apalache 20h TypedInv Accountability --length=0 --cinit=ConstInit
12 11 MC_n5_f1.tla apalache 20h TypedInv Agreement --length=0 --cinit=ConstInit
13 12 MC_n5_f2.tla apalache 20h TypedInv Accountability --length=0 --cinit=ConstInit

22
spec/light-client/accountability/MC_n4_f1.tla Normal file
View File

@@ -0,0 +1,22 @@
----------------------------- MODULE MC_n4_f1 -------------------------------
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
INSTANCE TendermintAccDebug_004_draft WITH
Corr <- {"c1", "c2", "c3"},
Faulty <- {"f1"},
N <- 4,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

22
spec/light-client/accountability/MC_n4_f2.tla Normal file
View File

@@ -0,0 +1,22 @@
----------------------------- MODULE MC_n4_f2 -------------------------------
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
INSTANCE TendermintAccDebug_004_draft WITH
Corr <- {"c1", "c2"},
Faulty <- {"f3", "f4"},
N <- 4,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

40
spec/light-client/accountability/MC_n4_f2_amnesia.tla Normal file
View File

@@ -0,0 +1,40 @@
---------------------- MODULE MC_n4_f2_amnesia -------------------------------
EXTENDS Sequences
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
\* the variable declared in TendermintAccTrace3
VARIABLE
toReplay
\* old apalache annotations, fix with the new release
a <: b == a
INSTANCE TendermintAccTrace_004_draft WITH
Corr <- {"c1", "c2"},
Faulty <- {"f3", "f4"},
N <- 4,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2,
Trace <- <<
"UponProposalInPropose",
"UponProposalInPrevoteOrCommitAndPrevote",
"UponProposalInPrecommitNoDecision",
"OnRoundCatchup",
"UponProposalInPropose",
"UponProposalInPrevoteOrCommitAndPrevote",
"UponProposalInPrecommitNoDecision"
>> <: Seq(STRING)
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

22
spec/light-client/accountability/MC_n4_f3.tla Normal file
View File

@@ -0,0 +1,22 @@
----------------------------- MODULE MC_n4_f3 -------------------------------
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
INSTANCE TendermintAccDebug_004_draft WITH
Corr <- {"c1"},
Faulty <- {"f2", "f3", "f4"},
N <- 4,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

22
spec/light-client/accountability/MC_n5_f1.tla Normal file
View File

@@ -0,0 +1,22 @@
----------------------------- MODULE MC_n5_f1 -------------------------------
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
INSTANCE TendermintAccDebug_004_draft WITH
Corr <- {"c1", "c2", "c3", "c4"},
Faulty <- {"f5"},
N <- 5,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

22
spec/light-client/accountability/MC_n5_f2.tla Normal file
View File

@@ -0,0 +1,22 @@
----------------------------- MODULE MC_n5_f2 -------------------------------
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
INSTANCE TendermintAccDebug_004_draft WITH
Corr <- {"c1", "c2", "c3"},
Faulty <- {"f4", "f5"},
N <- 5,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

22
spec/light-client/accountability/MC_n6_f1.tla Normal file
View File

@@ -0,0 +1,22 @@
----------------------------- MODULE MC_n6_f1 -------------------------------
CONSTANT Proposer \* the proposer function from 0..NRounds to 1..N
\* the variables declared in TendermintAcc3
VARIABLES
round, step, decision, lockedValue, lockedRound, validValue, validRound,
msgsPropose, msgsPrevote, msgsPrecommit, evidence, action
INSTANCE TendermintAccDebug_004_draft WITH
Corr <- {"c1", "c2", "c3", "c4", "c5"},
Faulty <- {"f6"},
N <- 4,
T <- 1,
ValidValues <- { "v0", "v1" },
InvalidValues <- {"v2"},
MaxRound <- 2
\* run Apalache with --cinit=ConstInit
ConstInit == \* the proposer is arbitrary -- works for safety
Proposer \in [Rounds -> AllProcs]
=============================================================================

308
spec/light-client/accountability/README.md Normal file
View File

@@ -0,0 +1,308 @@
---
order: 1
parent:
title: Accountability
order: 4
---
# Fork accountability
## Problem Statement
Tendermint consensus guarantees the following specifications for all heights:
* agreement -- no two correct full nodes decide differently.
* validity -- the decided block satisfies the predefined predicate *valid()*.
* termination -- all correct full nodes eventually decide,
If the faulty validators have less than 1/3 of voting power in the current validator set. In the case where this assumption
does not hold, each of the specification may be violated.
The agreement property says that for a given height, any two correct validators that decide on a block for that height decide on the same block. That the block was indeed generated by the blockchain, can be verified starting from a trusted (genesis) block, and checking that all subsequent blocks are properly signed.
However, faulty nodes may forge blocks and try to convince users (light clients) that the blocks had been correctly generated. In addition, Tendermint agreement might be violated in the case where 1/3 or more of the voting power belongs to faulty validators: Two correct validators decide on different blocks. The latter case motivates the term "fork": as Tendermint consensus also agrees on the next validator set, correct validators may have decided on disjoint next validator sets, and the chain branches into two or more partitions (possibly having faulty validators in common) and each branch continues to generate blocks independently of the other.
We say that a fork is a case in which there are two commits for different blocks at the same height of the blockchain. The proplem is to ensure that in those cases we are able to detect faulty validators (and not mistakenly accuse correct validators), and incentivize therefore validators to behave according to the protocol specification.
**Conceptual Limit.** In order to prove misbehavior of a node, we have to show that the behavior deviates from correct behavior with respect to a given algorithm. Thus, an algorithm that detects misbehavior of nodes executing some algorithm *A* must be defined with respect to algorithm *A*. In our case, *A* is Tendermint consensus (+ other protocols in the infrastructure; e.g.,full nodes and the Light Client). If the consensus algorithm is changed/updated/optimized in the future, we have to check whether changes to the accountability algorithm are also required. All the discussions in this document are thus inherently specific to Tendermint consensus and the Light Client specification.
**Q:** Should we distinguish agreement for validators and full nodes for agreement? The case where all correct validators agree on a block, but a correct full node decides on a different block seems to be slightly less severe that the case where two correct validators decide on different blocks. Still, if a contaminated full node becomes validator that may be problematic later on. Also it is not clear how gossiping is impaired if a contaminated full node is on a different branch.
*Remark.* In the case 1/3 or more of the voting power belongs to faulty validators, also validity and termination can be broken. Termination can be broken if faulty processes just do not send the messages that are needed to make progress. Due to asynchrony, this is not punishable, because faulty validators can always claim they never received the messages that would have forced them to send messages.
## The Misbehavior of Faulty Validators
Forks are the result of faulty validators deviating from the protocol. In principle several such deviations can be detected without a fork actually occurring:
1. double proposal: A faulty proposer proposes two different values (blocks) for the same height and the same round in Tendermint consensus.
2. double signing: Tendermint consensus forces correct validators to prevote and precommit for at most one value per round. In case a faulty validator sends multiple prevote and/or precommit messages for different values for the same height/round, this is a misbehavior.
3. lunatic validator: Tendermint consensus forces correct validators to prevote and precommit only for values *v* that satisfy *valid(v)*. If faulty validators prevote and precommit for *v* although *valid(v)=false* this is misbehavior.
*Remark.* In isolation, Point 3 is an attack on validity (rather than agreement). However, the prevotes and precommits can then also be used to forge blocks.
1. amnesia: Tendermint consensus has a locking mechanism. If a validator has some value v locked, then it can only prevote/precommit for v or nil. Sending prevote/precomit message for a different value v' (that is not nil) while holding lock on value v is misbehavior.
2. spurious messages: In Tendermint consensus most of the message send instructions are guarded by threshold guards, e.g., one needs to receive *2f + 1* prevote messages to send precommit. Faulty validators may send precommit without having received the prevote messages.
Independently of a fork happening, punishing this behavior might be important to prevent forks altogether. This should keep attackers from misbehaving: if less than 1/3 of the voting power is faulty, this misbehavior is detectable but will not lead to a safety violation. Thus, unless they have 1/3 or more (or in some cases more than 2/3) of the voting power attackers have the incentive to not misbehave. If attackers control too much voting power, we have to deal with forks, as discussed in this document.
## Two types of forks
* Fork-Full. Two correct validators decide on different blocks for the same height. Since also the next validator sets are decided upon, the correct validators may be partitioned to participate in two distinct branches of the forked chain.
As in this case we have two different blocks (both having the same right/no right to exist), a central system invariant (one block per height decided by correct validators) is violated. As full nodes are contaminated in this case, the contamination can spread also to light clients. However, even without breaking this system invariant, light clients can be subject to a fork:
* Fork-Light. All correct validators decide on the same block for height *h*, but faulty processes (validators or not), forge a different block for that height, in order to fool users (who use the light client).
# Attack scenarios
## On-chain attacks
### Equivocation (one round)
There are several scenarios in which forks might happen. The first is double signing within a round.
* F1. Equivocation: faulty validators sign multiple vote messages (prevote and/or precommit) for different values *during the same round r* at a given height h.
### Flip-flopping
Tendermint consensus implements a locking mechanism: If a correct validator *p* receives proposal for value v and *2f + 1* prevotes for a value *id(v)* in round *r*, it locks *v* and remembers *r*. In this case, *p* also sends a precommit message for *id(v)*, which later may serve as proof that *p* locked *v*.
In subsequent rounds, *p* only sends prevote messages for a value it had previously locked. However, it is possible to change the locked value if in a future round *r' > r*, if the process receives proposal and *2f + 1* prevotes for a different value *v'*. In this case, *p* could send a prevote/precommit for *id(v')*. This algorithmic feature can be exploited in two ways:
* F2. Faulty Flip-flopping (Amnesia): faulty validators precommit some value *id(v)* in round *r* (value *v* is locked in round *r*) and then prevote for different value *id(v')* in higher round *r' > r* without previously correctly unlocking value *v*. In this case faulty processes "forget" that they have locked value *v* and prevote some other value in the following rounds.
Some correct validators might have decided on *v* in *r*, and other correct validators decide on *v'* in *r'*. Here we can have branching on the main chain (Fork-Full).
* F3. Correct Flip-flopping (Back to the past): There are some precommit messages signed by (correct) validators for value *id(v)* in round *r*. Still, *v* is not decided upon, and all processes move on to the next round. Then correct validators (correctly) lock and decide a different value *v'* in some round *r' > r*. And the correct validators continue; there is no branching on the main chain.
However, faulty validators may use the correct precommit messages from round *r* together with a posteriori generated faulty precommit messages for round *r* to forge a block for a value that was not decided on the main chain (Fork-Light).
## Off-chain attacks
F1-F3 may contaminate the state of full nodes (and even validators). Contaminated (but otherwise correct) full nodes may thus communicate faulty blocks to light clients.
Similarly, without actually interfering with the main chain, we can have the following:
* F4. Phantom validators: faulty validators vote (sign prevote and precommit messages) in heights in which they are not part of the validator sets (at the main chain).
* F5. Lunatic validator: faulty validator that sign vote messages to support (arbitrary) application state that is different from the application state that resulted from valid state transitions.
## Types of victims
We consider three types of potential attack victims:
* FN: full node
* LCS: light client with sequential header verification
* LCB: light client with bisection based header verification
F1 and F2 can be used by faulty validators to actually create multiple branches on the blockchain. That means that correctly operating full nodes decide on different blocks for the same height. Until a fork is detected locally by a full node (by receiving evidence from others or by some other local check that fails), the full node can spread corrupted blocks to light clients.
*Remark.* If full nodes take a branch different from the one taken by the validators, it may be that the liveness of the gossip protocol may be affected. We should eventually look at this more closely. However, as it does not influence safety it is not a primary concern.
F3 is similar to F1, except that no two correct validators decide on different blocks. It may still be the case that full nodes become affected.
In addition, without creating a fork on the main chain, light clients can be contaminated by more than a third of validators that are faulty and sign a forged header
F4 cannot fool correct full nodes as they know the current validator set. Similarly, LCS know who the validators are. Hence, F4 is an attack against LCB that do not necessarily know the complete prefix of headers (Fork-Light), as they trust a header that is signed by at least one correct validator (trusting period method).
The following table gives an overview of how the different attacks may affect different nodes. F1-F3 are *on-chain* attacks so they can corrupt the state of full nodes. Then if a light client (LCS or LCB) contacts a full node to obtain headers (or blocks), the corrupted state may propagate to the light client.
F4 and F5 are *off-chain*, that is, these attacks cannot be used to corrupt the state of full nodes (which have sufficient knowledge on the state of the chain to not be fooled).
| Attack | FN | LCS | LCB |
|:------:|:------:|:------:|:------:|
| F1 | direct | FN | FN |
| F2 | direct | FN | FN |
| F3 | direct | FN | FN |
| F4 | | | direct |
| F5 | | | direct |
**Q:** Light clients are more vulnerable than full nodes, because the former do only verify headers but do not execute transactions. What kind of certainty is gained by a full node that executes a transaction?
As a full node verifies all transactions, it can only be
contaminated by an attack if the blockchain itself violates its invariant (one block per height), that is, in case of a fork that leads to branching.
## Detailed Attack Scenarios
### Equivocation based attacks
In case of equivocation based attacks, faulty validators sign multiple votes (prevote and/or precommit) in the same
round of some height. This attack can be executed on both full nodes and light clients. It requires 1/3 or more of voting power to be executed.
#### Scenario 1: Equivocation on the main chain
Validators:
* CA - a set of correct validators with less than 1/3 of the voting power
* CB - a set of correct validators with less than 1/3 of the voting power
* CA and CB are disjoint
* F - a set of faulty validators with 1/3 or more voting power
Observe that this setting violates the Tendermint failure model.
Execution:
* A faulty proposer proposes block A to CA
* A faulty proposer proposes block B to CB
* Validators from the set CA and CB prevote for A and B, respectively.
* Faulty validators from the set F prevote both for A and B.
* The faulty prevote messages
* for A arrive at CA long before the B messages
* for B arrive at CB long before the A messages
* Therefore correct validators from set CA and CB will observe
more than 2/3 of prevotes for A and B and precommit for A and B, respectively.
* Faulty validators from the set F precommit both values A and B.
* Thus, we have more than 2/3 commits for both A and B.
Consequences:
* Creating evidence of misbehavior is simple in this case as we have multiple messages signed by the same faulty processes for different values in the same round.
* We have to ensure that these different messages reach a correct process (full node, monitor?), which can submit evidence.
* This is an attack on the full node level (Fork-Full).
* It extends also to the light clients,
* For both we need a detection and recovery mechanism.
#### Scenario 2: Equivocation to a light client (LCS)
Validators:
* a set F of faulty validators with more than 2/3 of the voting power.
Execution:
* for the main chain F behaves nicely
* F coordinates to sign a block B that is different from the one on the main chain.
* the light clients obtains B and trusts at as it is signed by more than 2/3 of the voting power.
Consequences:
Once equivocation is used to attack light client it opens space
for different kind of attacks as application state can be diverged in any direction. For example, it can modify validator set such that it contains only validators that do not have any stake bonded. Note that after a light client is fooled by a fork, that means that an attacker can change application state and validator set arbitrarily.
In order to detect such (equivocation-based attack), the light client would need to cross check its state with some correct validator (or to obtain a hash of the state from the main chain using out of band channels).
*Remark.* The light client would be able to create evidence of misbehavior, but this would require to pull potentially a lot of data from correct full nodes. Maybe we need to figure out different architecture where a light client that is attacked will push all its data for the current unbonding period to a correct node that will inspect this data and submit corresponding evidence. There are also architectures that assumes a special role (sometimes called fisherman) whose goal is to collect as much as possible useful data from the network, to do analysis and create evidence transactions. That functionality is outside the scope of this document.
*Remark.* The difference between LCS and LCB might only be in the amount of voting power needed to convince light client about arbitrary state. In case of LCB where security threshold is at minimum, an attacker can arbitrarily modify application state with 1/3 or more of voting power, while in case of LCS it requires more than 2/3 of the voting power.
### Flip-flopping: Amnesia based attacks
In case of amnesia, faulty validators lock some value *v* in some round *r*, and then vote for different value *v'* in higher rounds without correctly unlocking value *v*. This attack can be used both on full nodes and light clients.
#### Scenario 3: At most 2/3 of faults
Validators:
* a set F of faulty validators with 1/3 or more but at most 2/3 of the voting power
* a set C of correct validators
Execution:
* Faulty validators commit (without exposing it on the main chain) a block A in round *r* by collecting more than 2/3 of the
voting power (containing correct and faulty validators).
* All validators (correct and faulty) reach a round *r' > r*.
* Some correct validators in C do not lock any value before round *r'*.
* The faulty validators in F deviate from Tendermint consensus by ignoring that they locked A in *r*, and propose a different block B in *r'*.
* As the validators in C that have not locked any value find B acceptable, they accept the proposal for B and commit a block B.
*Remark.* In this case, the more than 1/3 of faulty validators do not need to commit an equivocation (F1) as they only vote once per round in the execution.
Detecting faulty validators in the case of such an attack can be done by the fork accountability mechanism described in: <https://docs.google.com/document/d/11ZhMsCj3y7zIZz4udO9l25xqb0kl7gmWqNpGVRzOeyY/edit?usp=sharing>.
If a light client is attacked using this attack with 1/3 or more of voting power (and less than 2/3), the attacker cannot change the application state arbitrarily. Rather, the attacker is limited to a state a correct validator finds acceptable: In the execution above, correct validators still find the value acceptable, however, the block the light client trusts deviates from the one on the main chain.
#### Scenario 4: More than 2/3 of faults
In case there is an attack with more than 2/3 of the voting power, an attacker can arbitrarily change application state.
Validators:
* a set F1 of faulty validators with 1/3 or more of the voting power
* a set F2 of faulty validators with less than 1/3 of the voting power
Execution
* Similar to Scenario 3 (however, messages by correct validators are not needed)
* The faulty validators in F1 lock value A in round *r*
* They sign a different value in follow-up rounds
* F2 does not lock A in round *r*
Consequences:
* The validators in F1 will be detectable by the the fork accountability mechanisms.
* The validators in F2 cannot be detected using this mechanism.
Only in case they signed something which conflicts with the application this can be used against them. Otherwise they do not do anything incorrect.
* This case is not covered by the report <https://docs.google.com/document/d/11ZhMsCj3y7zIZz4udO9l25xqb0kl7gmWqNpGVRzOeyY/edit?usp=sharing> as it only assumes at most 2/3 of faulty validators.
**Q:** do we need to define a special kind of attack for the case where a validator sign arbitrarily state? It seems that detecting such attack requires a different mechanism that would require as an evidence a sequence of blocks that led to that state. This might be very tricky to implement.
### Back to the past
In this kind of attack, faulty validators take advantage of the fact that they did not sign messages in some of the past rounds. Due to the asynchronous network in which Tendermint operates, we cannot easily differentiate between such an attack and delayed message. This kind of attack can be used at both full nodes and light clients.
#### Scenario 5
Validators:
* C1 - a set of correct validators with over 1/3 of the voting power
* C2 - a set of correct validators with 1/3 of the voting power
* C1 and C2 are disjoint
* F - a set of faulty validators with less than 1/3 voting power
* one additional faulty process *q*
* F and *q* violate the Tendermint failure model.
Execution:
* in a round *r* of height *h* we have C1 precommitting a value A,
* C2 precommits nil,
* F does not send any message
* *q* precommits nil.
* In some round *r' > r*, F and *q* and C2 commit some other value B different from A.
* F and *fp* "go back to the past" and sign precommit message for value A in round *r*.
* Together with precomit messages of C1 this is sufficient for a commit for value A.
Consequences:
* Only a single faulty validator that previously precommited nil did equivocation, while the other 1/3 of faulty validators actually executed an attack that has exactly the same sequence of messages as part of amnesia attack. Detecting this kind of attack boil down to mechanisms for equivocation and amnesia.
**Q:** should we keep this as a separate kind of attack? It seems that equivocation, amnesia and phantom validators are the only kind of attack we need to support and this gives us security also in other cases. This would not be surprising as equivocation and amnesia are attacks that followed from the protocol and phantom attack is not really an attack to Tendermint but more to the Proof of Stake module.
### Phantom validators
In case of phantom validators, processes that are not part of the current validator set but are still bonded (as attack happen during their unbonding period) can be part of the attack by signing vote messages. This attack can be executed against both full nodes and light clients.
#### Scenario 6
Validators:
* F -- a set of faulty validators that are not part of the validator set on the main chain at height *h + k*
Execution:
* There is a fork, and there exist two different headers for height *h + k*, with different validator sets:
* VS2 on the main chain
* forged header VS2', signed by F (and others)
* a light client has a trust in a header for height *h* (and the corresponding validator set VS1).
* As part of bisection header verification, it verifies the header at height *h + k* with new validator set VS2'.
Consequences:
* To detect this, a node needs to see both, the forged header and the canonical header from the chain.
* If this is the case, detecting these kind of attacks is easy as it just requires verifying if processes are signing messages in heights in which they are not part of the validator set.
**Remark.** We can have phantom-validator-based attacks as a follow up of equivocation or amnesia based attack where forked state contains validators that are not part of the validator set at the main chain. In this case, they keep signing messages contributed to a forked chain (the wrong branch) although they are not part of the validator set on the main chain. This attack can also be used to attack full node during a period of time it is eclipsed.
**Remark.** Phantom validator evidence has been removed from implementation as it was deemed, although possibly a plausible form of evidence, not relevant. Any attack on
the light client involving a phantom validator will have needed to be initiated by 1/3+ lunatic
validators that can forge a new validator set that includes the phantom validator. Only in
that case will the light client accept the phantom validators vote. We need only worry about
punishing the 1/3+ lunatic cabal, that is the root cause of the attack.
### Lunatic validator
Lunatic validator agrees to sign commit messages for arbitrary application state. It is used to attack light clients.
Note that detecting this behavior require application knowledge. Detecting this behavior can probably be done by
referring to the block before the one in which height happen.
**Q:** can we say that in this case a validator declines to check if a proposed value is valid before voting for it?

105
spec/light-client/accountability/Synopsis.md Normal file
View File

@@ -0,0 +1,105 @@
# Synopsis
A TLA+ specification of a simplified Tendermint consensus, tuned for
fork accountability. The simplifications are as follows:
- the procotol runs for one height, that is, one-shot consensus
- this specification focuses on safety, so timeouts are modelled with
with non-determinism
- the proposer function is non-determinstic, no fairness is assumed
- the messages by the faulty processes are injected right in the initial states
- every process has the voting power of 1
- hashes are modelled as identity
Having the above assumptions in mind, the specification follows the pseudo-code
of the Tendermint paper: <https://arxiv.org/abs/1807.04938>
Byzantine processes can demonstrate arbitrary behavior, including
no communication. However, we have to show that under the collective evidence
collected by the correct processes, at least `f+1` Byzantine processes demonstrate
one of the following behaviors:
- Equivocation: a Byzantine process sends two different values
in the same round.
- Amnesia: a Byzantine process locks a value, although it has locked
another value in the past.
# TLA+ modules
- [TendermintAcc_004_draft](TendermintAcc_004_draft.tla) is the protocol
specification,
- [TendermintAccInv_004_draft](TendermintAccInv_004_draft.tla) contains an
inductive invariant for establishing the protocol safety as well as the
forking cases,
- `MC_n<n>_f<f>`, e.g., [MC_n4_f1](MC_n4_f1.tla), contains fixed constants for
model checking with the [Apalache model
checker](https://github.com/informalsystems/apalache),
- [TendermintAccTrace_004_draft](TendermintAccTrace_004_draft.tla) shows how
to restrict the execution space to a fixed sequence of actions (e.g., to
instantiate a counterexample),
- [TendermintAccDebug_004_draft](TendermintAccDebug_004_draft.tla) contains
the useful definitions for debugging the protocol specification with TLC and
Apalache.
# Reasoning about fork scenarios
The theorem statements can be found in
[TendermintAccInv_004_draft.tla](TendermintAccInv_004_draft.tla).
First, we would like to show that `TypedInv` is an inductive invariant.
Formally, the statement looks as follows:
```tla
THEOREM TypedInvIsInductive ==
\/ FaultyQuorum
\//\ Init => TypedInv
/\ TypedInv /\ [Next]_vars => TypedInv'
```
When over two-thirds of processes are faulty, `TypedInv` is not inductive.
However, there is no hope to repair the protocol in this case. We run
[Apalache](https://github.com/informalsystems/apalache) to prove this theorem
only for fixed instances of 4 to 5 validators. Apalache does not parse theorem
statements at the moment, so we ran Apalache using a shell script. To find a
parameterized argument, one has to use a theorem prover, e.g., TLAPS.
Second, we would like to show that the invariant implies `Agreement`, that is,
no fork, provided that less than one third of processes is faulty. By combining
this theorem with the previous theorem, we conclude that the protocol indeed
satisfies Agreement under the condition `LessThanThirdFaulty`.
```tla
THEOREM AgreementWhenLessThanThirdFaulty ==
LessThanThirdFaulty /\ TypedInv => Agreement
```
Third, in the general case, we either have no fork, or two fork scenarios:
```tla
THEOREM AgreementOrFork ==
~FaultyQuorum /\ TypedInv => Accountability
```
# Model checking results
Check the report on [model checking with Apalache](./results/001indinv-apalache-report.md).
To run the model checking experiments, use the script:
```console
./run.sh
```
This script assumes that the apalache build is available in
`~/devl/apalache-unstable`.

100
spec/light-client/accountability/TendermintAccDebug_004_draft.tla Normal file
View File

@@ -0,0 +1,100 @@
------------------ MODULE TendermintAccDebug_004_draft -------------------------
(*
A few definitions that we use for debugging TendermintAcc3, which do not belong
to the specification itself.
* Version 3. Modular and parameterized definitions.
Igor Konnov, 2020.
*)
EXTENDS TendermintAccInv_004_draft
\* make them parameters?
NFaultyProposals == 0 \* the number of injected faulty PROPOSE messages
NFaultyPrevotes == 6 \* the number of injected faulty PREVOTE messages
NFaultyPrecommits == 6 \* the number of injected faulty PRECOMMIT messages
\* Given a set of allowed messages Msgs, this operator produces a function from
\* rounds to sets of messages.
\* Importantly, there will be exactly k messages in the image of msgFun.
\* We use this action to produce k faults in an initial state.
ProduceFaults(msgFun, From, k) ==
\E f \in [1..k -> From]:
msgFun = [r \in Rounds |-> {m \in {f[i]: i \in 1..k}: m.round = r}]
\* As TLC explodes with faults, we may have initial states without faults
InitNoFaults ==
/\ round = [p \in Corr |-> 0]
/\ step = [p \in Corr |-> "PROPOSE"]
/\ decision = [p \in Corr |-> NilValue]
/\ lockedValue = [p \in Corr |-> NilValue]
/\ lockedRound = [p \in Corr |-> NilRound]
/\ validValue = [p \in Corr |-> NilValue]
/\ validRound = [p \in Corr |-> NilRound]
/\ msgsPropose = [r \in Rounds |-> EmptyMsgSet]
/\ msgsPrevote = [r \in Rounds |-> EmptyMsgSet]
/\ msgsPrecommit = [r \in Rounds |-> EmptyMsgSet]
/\ evidence = EmptyMsgSet
(*
A specialized version of Init that injects NFaultyProposals proposals,
NFaultyPrevotes prevotes, NFaultyPrecommits precommits by the faulty processes
*)
InitFewFaults ==
/\ round = [p \in Corr |-> 0]
/\ step = [p \in Corr |-> "PROPOSE"]
/\ decision = [p \in Corr |-> NilValue]
/\ lockedValue = [p \in Corr |-> NilValue]
/\ lockedRound = [p \in Corr |-> NilRound]
/\ validValue = [p \in Corr |-> NilValue]
/\ validRound = [p \in Corr |-> NilRound]
/\ ProduceFaults(msgsPrevote',
SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: Rounds, id: Values]),
NFaultyPrevotes)
/\ ProduceFaults(msgsPrecommit',
SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: Rounds, id: Values]),
NFaultyPrecommits)
/\ ProduceFaults(msgsPropose',
SetOfMsgs([type: {"PROPOSAL"}, src: Faulty, round: Rounds,
proposal: Values, validRound: Rounds \cup {NilRound}]),
NFaultyProposals)
/\ evidence = EmptyMsgSet
\* Add faults incrementally
NextWithFaults ==
\* either the protocol makes a step
\/ Next
\* or a faulty process sends a message
\//\ UNCHANGED <<round, step, decision, lockedValue,
lockedRound, validValue, validRound, evidence>>
/\ \E p \in Faulty:
\E r \in Rounds:
\//\ UNCHANGED <<msgsPrevote, msgsPrecommit>>
/\ \E proposal \in ValidValues \union {NilValue}:
\E vr \in RoundsOrNil:
BroadcastProposal(p, r, proposal, vr)
\//\ UNCHANGED <<msgsPropose, msgsPrecommit>>
/\ \E id \in ValidValues \union {NilValue}:
BroadcastPrevote(p, r, id)
\//\ UNCHANGED <<msgsPropose, msgsPrevote>>
/\ \E id \in ValidValues \union {NilValue}:
BroadcastPrecommit(p, r, id)
(******************************** PROPERTIES ***************************************)
\* simple reachability properties to see that the spec is progressing
NoPrevote == \A p \in Corr: step[p] /= "PREVOTE"
NoPrecommit == \A p \in Corr: step[p] /= "PRECOMMIT"
NoValidPrecommit ==
\A r \in Rounds:
\A m \in msgsPrecommit[r]:
m.id = NilValue \/ m.src \in Faulty
NoHigherRounds == \A p \in Corr: round[p] < 1
NoDecision == \A p \in Corr: decision[p] = NilValue
=============================================================================

370
spec/light-client/accountability/TendermintAccInv_004_draft.tla Normal file
View File

@@ -0,0 +1,370 @@
------------------- MODULE TendermintAccInv_004_draft --------------------------
(*
An inductive invariant for TendermintAcc3, which capture the forked
and non-forked cases.
* Version 3. Modular and parameterized definitions.
* Version 2. Bugfixes in the spec and an inductive invariant.
Igor Konnov, 2020.
*)
EXTENDS TendermintAcc_004_draft
(************************** TYPE INVARIANT ***********************************)
(* first, we define the sets of all potential messages *)
AllProposals ==
SetOfMsgs([type: {"PROPOSAL"},
src: AllProcs,
round: Rounds,
proposal: ValuesOrNil,
validRound: RoundsOrNil])
AllPrevotes ==
SetOfMsgs([type: {"PREVOTE"},
src: AllProcs,
round: Rounds,
id: ValuesOrNil])
AllPrecommits ==
SetOfMsgs([type: {"PRECOMMIT"},
src: AllProcs,
round: Rounds,
id: ValuesOrNil])
(* the standard type invariant -- importantly, it is inductive *)
TypeOK ==
/\ round \in [Corr -> Rounds]
/\ step \in [Corr -> { "PROPOSE", "PREVOTE", "PRECOMMIT", "DECIDED" }]
/\ decision \in [Corr -> ValidValues \union {NilValue}]
/\ lockedValue \in [Corr -> ValidValues \union {NilValue}]
/\ lockedRound \in [Corr -> RoundsOrNil]
/\ validValue \in [Corr -> ValidValues \union {NilValue}]
/\ validRound \in [Corr -> RoundsOrNil]
/\ msgsPropose \in [Rounds -> SUBSET AllProposals]
/\ BenignRoundsInMessages(msgsPropose)
/\ msgsPrevote \in [Rounds -> SUBSET AllPrevotes]
/\ BenignRoundsInMessages(msgsPrevote)
/\ msgsPrecommit \in [Rounds -> SUBSET AllPrecommits]
/\ BenignRoundsInMessages(msgsPrecommit)
/\ evidence \in SUBSET (AllProposals \union AllPrevotes \union AllPrecommits)
/\ action \in {
"Init",
"InsertProposal",
"UponProposalInPropose",
"UponProposalInProposeAndPrevote",
"UponQuorumOfPrevotesAny",
"UponProposalInPrevoteOrCommitAndPrevote",
"UponQuorumOfPrecommitsAny",
"UponProposalInPrecommitNoDecision",
"OnTimeoutPropose",
"OnQuorumOfNilPrevotes",
"OnRoundCatchup"
}
(************************** INDUCTIVE INVARIANT *******************************)
EvidenceContainsMessages ==
\* evidence contains only the messages from:
\* msgsPropose, msgsPrevote, and msgsPrecommit
\A m \in evidence:
LET r == m.round
t == m.type
IN
CASE t = "PROPOSAL" -> m \in msgsPropose[r]
[] t = "PREVOTE" -> m \in msgsPrevote[r]
[] OTHER -> m \in msgsPrecommit[r]
NoFutureMessagesForLargerRounds(p) ==
\* a correct process does not send messages for the future rounds
\A r \in { rr \in Rounds: rr > round[p] }:
/\ \A m \in msgsPropose[r]: m.src /= p
/\ \A m \in msgsPrevote[r]: m.src /= p
/\ \A m \in msgsPrecommit[r]: m.src /= p
NoFutureMessagesForCurrentRound(p) ==
\* a correct process does not send messages in the future
LET r == round[p] IN
/\ Proposer[r] = p \/ \A m \in msgsPropose[r]: m.src /= p
/\ \/ step[p] \in {"PREVOTE", "PRECOMMIT", "DECIDED"}
\/ \A m \in msgsPrevote[r]: m.src /= p
/\ \/ step[p] \in {"PRECOMMIT", "DECIDED"}
\/ \A m \in msgsPrecommit[r]: m.src /= p
\* the correct processes never send future messages
AllNoFutureMessagesSent ==
\A p \in Corr:
/\ NoFutureMessagesForCurrentRound(p)
/\ NoFutureMessagesForLargerRounds(p)
\* a correct process in the PREVOTE state has sent a PREVOTE message
IfInPrevoteThenSentPrevote(p) ==
step[p] = "PREVOTE" =>
\E m \in msgsPrevote[round[p]]:
/\ m.id \in ValidValues \cup { NilValue }
/\ m.src = p
AllIfInPrevoteThenSentPrevote ==
\A p \in Corr: IfInPrevoteThenSentPrevote(p)
\* a correct process in the PRECOMMIT state has sent a PRECOMMIT message
IfInPrecommitThenSentPrecommit(p) ==
step[p] = "PRECOMMIT" =>
\E m \in msgsPrecommit[round[p]]:
/\ m.id \in ValidValues \cup { NilValue }
/\ m.src = p
AllIfInPrecommitThenSentPrecommit ==
\A p \in Corr: IfInPrecommitThenSentPrecommit(p)
\* a process in the PRECOMMIT state has sent a PRECOMMIT message
IfInDecidedThenValidDecision(p) ==
step[p] = "DECIDED" <=> decision[p] \in ValidValues
AllIfInDecidedThenValidDecision ==
\A p \in Corr: IfInDecidedThenValidDecision(p)
\* a decided process should have received a proposal on its decision
IfInDecidedThenReceivedProposal(p) ==
step[p] = "DECIDED" =>
\E r \in Rounds: \* r is not necessarily round[p]
/\ \E m \in msgsPropose[r] \intersect evidence:
/\ m.src = Proposer[r]
/\ m.proposal = decision[p]
\* not inductive: /\ m.src \in Corr => (m.validRound <= r)
AllIfInDecidedThenReceivedProposal ==
\A p \in Corr:
IfInDecidedThenReceivedProposal(p)
\* a decided process has received two-thirds of precommit messages
IfInDecidedThenReceivedTwoThirds(p) ==
step[p] = "DECIDED" =>
\E r \in Rounds:
LET PV ==
{ m \in msgsPrecommit[r] \intersect evidence: m.id = decision[p] }
IN
Cardinality(PV) >= THRESHOLD2
AllIfInDecidedThenReceivedTwoThirds ==
\A p \in Corr:
IfInDecidedThenReceivedTwoThirds(p)
\* for a round r, there is proposal by the round proposer for a valid round vr
ProposalInRound(r, proposedVal, vr) ==
\E m \in msgsPropose[r]:
/\ m.src = Proposer[r]
/\ m.proposal = proposedVal
/\ m.validRound = vr
TwoThirdsPrevotes(vr, v) ==
LET PV == { mm \in msgsPrevote[vr] \intersect evidence: mm.id = v } IN
Cardinality(PV) >= THRESHOLD2
\* if a process sends a PREVOTE, then there are three possibilities:
\* 1) the process is faulty, 2) the PREVOTE cotains Nil,
\* 3) there is a proposal in an earlier (valid) round and two thirds of PREVOTES
IfSentPrevoteThenReceivedProposalOrTwoThirds(r) ==
\A mpv \in msgsPrevote[r]:
\/ mpv.src \in Faulty
\* lockedRound and lockedValue is beyond my comprehension
\/ mpv.id = NilValue
\//\ mpv.src \in Corr
/\ mpv.id /= NilValue
/\ \/ ProposalInRound(r, mpv.id, NilRound)
\/ \E vr \in { rr \in Rounds: rr < r }:
/\ ProposalInRound(r, mpv.id, vr)
/\ TwoThirdsPrevotes(vr, mpv.id)
AllIfSentPrevoteThenReceivedProposalOrTwoThirds ==
\A r \in Rounds:
IfSentPrevoteThenReceivedProposalOrTwoThirds(r)
\* if a correct process has sent a PRECOMMIT, then there are two thirds,
\* either on a valid value, or a nil value
IfSentPrecommitThenReceivedTwoThirds ==
\A r \in Rounds:
\A mpc \in msgsPrecommit[r]:
mpc.src \in Corr =>
\/ /\ mpc.id \in ValidValues
/\ LET PV ==
{ m \in msgsPrevote[r] \intersect evidence: m.id = mpc.id }
IN
Cardinality(PV) >= THRESHOLD2
\/ /\ mpc.id = NilValue
/\ Cardinality(msgsPrevote[r]) >= THRESHOLD2
\* if a correct process has sent a precommit message in a round, it should
\* have sent a prevote
IfSentPrecommitThenSentPrevote ==
\A r \in Rounds:
\A mpc \in msgsPrecommit[r]:
mpc.src \in Corr =>
\E m \in msgsPrevote[r]:
m.src = mpc.src
\* there is a locked round if a only if there is a locked value
LockedRoundIffLockedValue(p) ==
(lockedRound[p] = NilRound) <=> (lockedValue[p] = NilValue)
AllLockedRoundIffLockedValue ==
\A p \in Corr:
LockedRoundIffLockedValue(p)
\* when a process locked a round, it must have sent a precommit on the locked value.
IfLockedRoundThenSentCommit(p) ==
lockedRound[p] /= NilRound
=> \E r \in { rr \in Rounds: rr <= round[p] }:
\E m \in msgsPrecommit[r]:
m.src = p /\ m.id = lockedValue[p]
AllIfLockedRoundThenSentCommit ==
\A p \in Corr:
IfLockedRoundThenSentCommit(p)
\* a process always locks the latest round, for which it has sent a PRECOMMIT
LatestPrecommitHasLockedRound(p) ==
LET pPrecommits ==
{mm \in UNION { msgsPrecommit[r]: r \in Rounds }: mm.src = p /\ mm.id /= NilValue }
IN
pPrecommits /= {} <: {MT}
=> LET latest ==
CHOOSE m \in pPrecommits:
\A m2 \in pPrecommits:
m2.round <= m.round
IN
/\ lockedRound[p] = latest.round
/\ lockedValue[p] = latest.id
AllLatestPrecommitHasLockedRound ==
\A p \in Corr:
LatestPrecommitHasLockedRound(p)
\* Every correct process sends only one value or NilValue.
\* This test has quantifier alternation -- a threat to all decision procedures.
\* Luckily, the sets Corr and ValidValues are small.
NoEquivocationByCorrect(r, msgs) ==
\A p \in Corr:
\E v \in ValidValues \union {NilValue}:
\A m \in msgs[r]:
\/ m.src /= p
\/ m.id = v
\* a proposer nevers sends two values
ProposalsByProposer(r, msgs) ==
\* if the proposer is not faulty, it sends only one value
\E v \in ValidValues:
\A m \in msgs[r]:
\/ m.src \in Faulty
\/ m.src = Proposer[r] /\ m.proposal = v
AllNoEquivocationByCorrect ==
\A r \in Rounds:
/\ ProposalsByProposer(r, msgsPropose)
/\ NoEquivocationByCorrect(r, msgsPrevote)
/\ NoEquivocationByCorrect(r, msgsPrecommit)
\* construct the set of the message senders
Senders(M) == { m.src: m \in M }
\* The final piece by Josef Widder:
\* if T + 1 processes precommit on the same value in a round,
\* then in the future rounds there are less than 2T + 1 prevotes for another value
PrecommitsLockValue ==
\A r \in Rounds:
\A v \in ValidValues \union {NilValue}:
\/ LET Precommits == {m \in msgsPrecommit[r]: m.id = v}
IN
Cardinality(Senders(Precommits)) < THRESHOLD1
\/ \A fr \in { rr \in Rounds: rr > r }: \* future rounds
\A w \in (ValuesOrNil) \ {v}:
LET Prevotes == {m \in msgsPrevote[fr]: m.id = w}
IN
Cardinality(Senders(Prevotes)) < THRESHOLD2
\* a combination of all lemmas
Inv ==
/\ EvidenceContainsMessages
/\ AllNoFutureMessagesSent
/\ AllIfInPrevoteThenSentPrevote
/\ AllIfInPrecommitThenSentPrecommit
/\ AllIfInDecidedThenReceivedProposal
/\ AllIfInDecidedThenReceivedTwoThirds
/\ AllIfInDecidedThenValidDecision
/\ AllLockedRoundIffLockedValue
/\ AllIfLockedRoundThenSentCommit
/\ AllLatestPrecommitHasLockedRound
/\ AllIfSentPrevoteThenReceivedProposalOrTwoThirds
/\ IfSentPrecommitThenSentPrevote
/\ IfSentPrecommitThenReceivedTwoThirds
/\ AllNoEquivocationByCorrect
/\ PrecommitsLockValue
\* this is the inductive invariant we like to check
TypedInv == TypeOK /\ Inv
\* UNUSED FOR SAFETY
ValidRoundNotSmallerThanLockedRound(p) ==
validRound[p] >= lockedRound[p]
\* UNUSED FOR SAFETY
ValidRoundIffValidValue(p) ==
(validRound[p] = NilRound) <=> (validValue[p] = NilValue)
\* UNUSED FOR SAFETY
AllValidRoundIffValidValue ==
\A p \in Corr: ValidRoundIffValidValue(p)
\* if validRound is defined, then there are two-thirds of PREVOTEs
IfValidRoundThenTwoThirds(p) ==
\/ validRound[p] = NilRound
\/ LET PV == { m \in msgsPrevote[validRound[p]]: m.id = validValue[p] } IN
Cardinality(PV) >= THRESHOLD2
\* UNUSED FOR SAFETY
AllIfValidRoundThenTwoThirds ==
\A p \in Corr: IfValidRoundThenTwoThirds(p)
\* a valid round can be only set to a valid value that was proposed earlier
IfValidRoundThenProposal(p) ==
\/ validRound[p] = NilRound
\/ \E m \in msgsPropose[validRound[p]]:
m.proposal = validValue[p]
\* UNUSED FOR SAFETY
AllIfValidRoundThenProposal ==
\A p \in Corr: IfValidRoundThenProposal(p)
(******************************** THEOREMS ***************************************)
(* Under this condition, the faulty processes can decide alone *)
FaultyQuorum == Cardinality(Faulty) >= THRESHOLD2
(* The standard condition of the Tendermint security model *)
LessThanThirdFaulty == N > 3 * T /\ Cardinality(Faulty) <= T
(*
TypedInv is an inductive invariant, provided that there is no faulty quorum.
We run Apalache to prove this theorem only for fixed instances of 4 to 10 processes.
(We run Apalache manually, as it does not parse theorem statements at the moment.)
To get a parameterized argument, one has to use a theorem prover, e.g., TLAPS.
*)
THEOREM TypedInvIsInductive ==
\/ FaultyQuorum \* if there are 2 * T + 1 faulty processes, we give up
\//\ Init => TypedInv
/\ TypedInv /\ [Next]_vars => TypedInv'
(*
There should be no fork, when there are less than 1/3 faulty processes.
*)
THEOREM AgreementWhenLessThanThirdFaulty ==
LessThanThirdFaulty /\ TypedInv => Agreement
(*
In a more general case, when there are less than 2/3 faulty processes,
there is either Agreement (no fork), or two scenarios exist:
equivocation by Faulty, or amnesia by Faulty.
*)
THEOREM AgreementOrFork ==
~FaultyQuorum /\ TypedInv => Accountability
=============================================================================

33
spec/light-client/accountability/TendermintAccTrace_004_draft.tla Normal file
View File

@@ -0,0 +1,33 @@
------------------ MODULE TendermintAccTrace_004_draft -------------------------
(*
When Apalache is running too slow and we have an idea of a counterexample,
we use this module to restrict the behaviors only to certain actions.
Once the whole trace is replayed, the system deadlocks.
Version 1.
Igor Konnov, 2020.
*)
EXTENDS Sequences, Apalache, TendermintAcc_004_draft
\* a sequence of action names that should appear in the given order,
\* excluding "Init"
CONSTANT Trace
VARIABLE toReplay
TraceInit ==
/\ toReplay = Trace
/\ action' := "Init"
/\ Init
TraceNext ==
/\ Len(toReplay) > 0
/\ toReplay' = Tail(toReplay)
\* Here is the trick. We restrict the action to the expected one,
\* so the other actions will be pruned
/\ action' := Head(toReplay)
/\ Next
================================================================================

474
spec/light-client/accountability/TendermintAcc_004_draft.tla Normal file
View File

@@ -0,0 +1,474 @@
-------------------- MODULE TendermintAcc_004_draft ---------------------------
(*
A TLA+ specification of a simplified Tendermint consensus, tuned for
fork accountability. The simplifications are as follows:
- the protocol runs for one height, that is, it is one-shot consensus
- this specification focuses on safety, so timeouts are modelled
with non-determinism
- the proposer function is non-determinstic, no fairness is assumed
- the messages by the faulty processes are injected right in the initial states
- every process has the voting power of 1
- hashes are modelled as identity
Having the above assumptions in mind, the specification follows the pseudo-code
of the Tendermint paper: https://arxiv.org/abs/1807.04938
Byzantine processes can demonstrate arbitrary behavior, including
no communication. We show that if agreement is violated, then the Byzantine
processes demonstrate one of the two behaviours:
- Equivocation: a Byzantine process may send two different values
in the same round.
- Amnesia: a Byzantine process may lock a value without unlocking
the previous value that it has locked in the past.
* Version 4. Remove defective processes, fix bugs, collect global evidence.
* Version 3. Modular and parameterized definitions.
* Version 2. Bugfixes in the spec and an inductive invariant.
* Version 1. A preliminary specification.
Zarko Milosevic, Igor Konnov, Informal Systems, 2019-2020.
*)
EXTENDS Integers, FiniteSets
(********************* PROTOCOL PARAMETERS **********************************)
CONSTANTS
Corr, \* the set of correct processes
Faulty, \* the set of Byzantine processes, may be empty
N, \* the total number of processes: correct, defective, and Byzantine
T, \* an upper bound on the number of Byzantine processes
ValidValues, \* the set of valid values, proposed both by correct and faulty
InvalidValues, \* the set of invalid values, never proposed by the correct ones
MaxRound, \* the maximal round number
Proposer \* the proposer function from 0..NRounds to 1..N
ASSUME(N = Cardinality(Corr \union Faulty))
(*************************** DEFINITIONS ************************************)
AllProcs == Corr \union Faulty \* the set of all processes
Rounds == 0..MaxRound \* the set of potential rounds
NilRound == -1 \* a special value to denote a nil round, outside of Rounds
RoundsOrNil == Rounds \union {NilRound}
Values == ValidValues \union InvalidValues \* the set of all values
NilValue == "None" \* a special value for a nil round, outside of Values
ValuesOrNil == Values \union {NilValue}
\* a value hash is modeled as identity
Id(v) == v
\* The validity predicate
IsValid(v) == v \in ValidValues
\* the two thresholds that are used in the algorithm
THRESHOLD1 == T + 1 \* at least one process is not faulty
THRESHOLD2 == 2 * T + 1 \* a quorum when having N > 3 * T
(********************* TYPE ANNOTATIONS FOR APALACHE **************************)
\* the operator for type annotations
a <: b == a
\* the type of message records
MT == [type |-> STRING, src |-> STRING, round |-> Int,
proposal |-> STRING, validRound |-> Int, id |-> STRING]
\* a type annotation for a message
AsMsg(m) == m <: MT
\* a type annotation for a set of messages
SetOfMsgs(S) == S <: {MT}
\* a type annotation for an empty set of messages
EmptyMsgSet == SetOfMsgs({})
(********************* PROTOCOL STATE VARIABLES ******************************)
VARIABLES
round, \* a process round number: Corr -> Rounds
step, \* a process step: Corr -> { "PROPOSE", "PREVOTE", "PRECOMMIT", "DECIDED" }
decision, \* process decision: Corr -> ValuesOrNil
lockedValue, \* a locked value: Corr -> ValuesOrNil
lockedRound, \* a locked round: Corr -> RoundsOrNil
validValue, \* a valid value: Corr -> ValuesOrNil
validRound \* a valid round: Corr -> RoundsOrNil
\* book-keeping variables
VARIABLES
msgsPropose, \* PROPOSE messages broadcast in the system, Rounds -> Messages
msgsPrevote, \* PREVOTE messages broadcast in the system, Rounds -> Messages
msgsPrecommit, \* PRECOMMIT messages broadcast in the system, Rounds -> Messages
evidence, \* the messages that were used by the correct processes to make transitions
action \* we use this variable to see which action was taken
(* to see a type invariant, check TendermintAccInv3 *)
\* a handy definition used in UNCHANGED
vars == <<round, step, decision, lockedValue, lockedRound,
validValue, validRound, evidence, msgsPropose, msgsPrevote, msgsPrecommit>>
(********************* PROTOCOL INITIALIZATION ******************************)
FaultyProposals(r) ==
SetOfMsgs([type: {"PROPOSAL"}, src: Faulty,
round: {r}, proposal: Values, validRound: RoundsOrNil])
AllFaultyProposals ==
SetOfMsgs([type: {"PROPOSAL"}, src: Faulty,
round: Rounds, proposal: Values, validRound: RoundsOrNil])
FaultyPrevotes(r) ==
SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: {r}, id: Values])
AllFaultyPrevotes ==
SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: Rounds, id: Values])
FaultyPrecommits(r) ==
SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: {r}, id: Values])
AllFaultyPrecommits ==
SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: Rounds, id: Values])
BenignRoundsInMessages(msgfun) ==
\* the message function never contains a message for a wrong round
\A r \in Rounds:
\A m \in msgfun[r]:
r = m.round
\* The initial states of the protocol. Some faults can be in the system already.
Init ==
/\ round = [p \in Corr |-> 0]
/\ step = [p \in Corr |-> "PROPOSE"]
/\ decision = [p \in Corr |-> NilValue]
/\ lockedValue = [p \in Corr |-> NilValue]
/\ lockedRound = [p \in Corr |-> NilRound]
/\ validValue = [p \in Corr |-> NilValue]
/\ validRound = [p \in Corr |-> NilRound]
/\ msgsPropose \in [Rounds -> SUBSET AllFaultyProposals]
/\ msgsPrevote \in [Rounds -> SUBSET AllFaultyPrevotes]
/\ msgsPrecommit \in [Rounds -> SUBSET AllFaultyPrecommits]
/\ BenignRoundsInMessages(msgsPropose)
/\ BenignRoundsInMessages(msgsPrevote)
/\ BenignRoundsInMessages(msgsPrecommit)
/\ evidence = EmptyMsgSet
/\ action' = "Init"
(************************ MESSAGE PASSING ********************************)
BroadcastProposal(pSrc, pRound, pProposal, pValidRound) ==
LET newMsg ==
AsMsg([type |-> "PROPOSAL", src |-> pSrc, round |-> pRound,
proposal |-> pProposal, validRound |-> pValidRound])
IN
msgsPropose' = [msgsPropose EXCEPT ![pRound] = msgsPropose[pRound] \union {newMsg}]
BroadcastPrevote(pSrc, pRound, pId) ==
LET newMsg == AsMsg([type |-> "PREVOTE",
src |-> pSrc, round |-> pRound, id |-> pId])
IN
msgsPrevote' = [msgsPrevote EXCEPT ![pRound] = msgsPrevote[pRound] \union {newMsg}]
BroadcastPrecommit(pSrc, pRound, pId) ==
LET newMsg == AsMsg([type |-> "PRECOMMIT",
src |-> pSrc, round |-> pRound, id |-> pId])
IN
msgsPrecommit' = [msgsPrecommit EXCEPT ![pRound] = msgsPrecommit[pRound] \union {newMsg}]
(********************* PROTOCOL TRANSITIONS ******************************)
\* lines 12-13
StartRound(p, r) ==
/\ step[p] /= "DECIDED" \* a decided process does not participate in consensus
/\ round' = [round EXCEPT ![p] = r]
/\ step' = [step EXCEPT ![p] = "PROPOSE"]
\* lines 14-19, a proposal may be sent later
InsertProposal(p) ==
LET r == round[p] IN
/\ p = Proposer[r]
/\ step[p] = "PROPOSE"
\* if the proposer is sending a proposal, then there are no other proposals
\* by the correct processes for the same round
/\ \A m \in msgsPropose[r]: m.src /= p
/\ \E v \in ValidValues:
LET proposal == IF validValue[p] /= NilValue THEN validValue[p] ELSE v IN
BroadcastProposal(p, round[p], proposal, validRound[p])
/\ UNCHANGED <<evidence, round, decision, lockedValue, lockedRound,
validValue, step, validRound, msgsPrevote, msgsPrecommit>>
/\ action' = "InsertProposal"
\* lines 22-27
UponProposalInPropose(p) ==
\E v \in Values:
/\ step[p] = "PROPOSE" (* line 22 *)
/\ LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> v, validRound |-> NilRound]) IN
/\ msg \in msgsPropose[round[p]] \* line 22
/\ evidence' = {msg} \union evidence
/\ LET mid == (* line 23 *)
IF IsValid(v) /\ (lockedRound[p] = NilRound \/ lockedValue[p] = v)
THEN Id(v)
ELSE NilValue
IN
BroadcastPrevote(p, round[p], mid) \* lines 24-26
/\ step' = [step EXCEPT ![p] = "PREVOTE"]
/\ UNCHANGED <<round, decision, lockedValue, lockedRound,
validValue, validRound, msgsPropose, msgsPrecommit>>
/\ action' = "UponProposalInPropose"
\* lines 28-33
UponProposalInProposeAndPrevote(p) ==
\E v \in Values, vr \in Rounds:
/\ step[p] = "PROPOSE" /\ 0 <= vr /\ vr < round[p] \* line 28, the while part
/\ LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> v, validRound |-> vr])
IN
/\ msg \in msgsPropose[round[p]] \* line 28
/\ LET PV == { m \in msgsPrevote[vr]: m.id = Id(v) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 28
/\ evidence' = PV \union {msg} \union evidence
/\ LET mid == (* line 29 *)
IF IsValid(v) /\ (lockedRound[p] <= vr \/ lockedValue[p] = v)
THEN Id(v)
ELSE NilValue
IN
BroadcastPrevote(p, round[p], mid) \* lines 24-26
/\ step' = [step EXCEPT ![p] = "PREVOTE"]
/\ UNCHANGED <<round, decision, lockedValue, lockedRound,
validValue, validRound, msgsPropose, msgsPrecommit>>
/\ action' = "UponProposalInProposeAndPrevote"
\* lines 34-35 + lines 61-64 (onTimeoutPrevote)
UponQuorumOfPrevotesAny(p) ==
/\ step[p] = "PREVOTE" \* line 34 and 61
/\ \E MyEvidence \in SUBSET msgsPrevote[round[p]]:
\* find the unique voters in the evidence
LET Voters == { m.src: m \in MyEvidence } IN
\* compare the number of the unique voters against the threshold
/\ Cardinality(Voters) >= THRESHOLD2 \* line 34
/\ evidence' = MyEvidence \union evidence
/\ BroadcastPrecommit(p, round[p], NilValue)
/\ step' = [step EXCEPT ![p] = "PRECOMMIT"]
/\ UNCHANGED <<round, decision, lockedValue, lockedRound,
validValue, validRound, msgsPropose, msgsPrevote>>
/\ action' = "UponQuorumOfPrevotesAny"
\* lines 36-46
UponProposalInPrevoteOrCommitAndPrevote(p) ==
\E v \in ValidValues, vr \in RoundsOrNil:
/\ step[p] \in {"PREVOTE", "PRECOMMIT"} \* line 36
/\ LET msg ==
AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]],
round |-> round[p], proposal |-> v, validRound |-> vr]) IN
/\ msg \in msgsPropose[round[p]] \* line 36
/\ LET PV == { m \in msgsPrevote[round[p]]: m.id = Id(v) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 36
/\ evidence' = PV \union {msg} \union evidence
/\ IF step[p] = "PREVOTE"
THEN \* lines 38-41:
/\ lockedValue' = [lockedValue EXCEPT ![p] = v]
/\ lockedRound' = [lockedRound EXCEPT ![p] = round[p]]
/\ BroadcastPrecommit(p, round[p], Id(v))
/\ step' = [step EXCEPT ![p] = "PRECOMMIT"]
ELSE
UNCHANGED <<lockedValue, lockedRound, msgsPrecommit, step>>
\* lines 42-43
/\ validValue' = [validValue EXCEPT ![p] = v]
/\ validRound' = [validRound EXCEPT ![p] = round[p]]
/\ UNCHANGED <<round, decision, msgsPropose, msgsPrevote>>
/\ action' = "UponProposalInPrevoteOrCommitAndPrevote"
\* lines 47-48 + 65-67 (onTimeoutPrecommit)
UponQuorumOfPrecommitsAny(p) ==
/\ \E MyEvidence \in SUBSET msgsPrecommit[round[p]]:
\* find the unique committers in the evidence
LET Committers == { m.src: m \in MyEvidence } IN
\* compare the number of the unique committers against the threshold
/\ Cardinality(Committers) >= THRESHOLD2 \* line 47
/\ evidence' = MyEvidence \union evidence
/\ round[p] + 1 \in Rounds
/\ StartRound(p, round[p] + 1)
/\ UNCHANGED <<decision, lockedValue, lockedRound, validValue,
validRound, msgsPropose, msgsPrevote, msgsPrecommit>>
/\ action' = "UponQuorumOfPrecommitsAny"
\* lines 49-54
UponProposalInPrecommitNoDecision(p) ==
/\ decision[p] = NilValue \* line 49
/\ \E v \in ValidValues (* line 50*) , r \in Rounds, vr \in RoundsOrNil:
/\ LET msg == AsMsg([type |-> "PROPOSAL", src |-> Proposer[r],
round |-> r, proposal |-> v, validRound |-> vr]) IN
/\ msg \in msgsPropose[r] \* line 49
/\ LET PV == { m \in msgsPrecommit[r]: m.id = Id(v) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 49
/\ evidence' = PV \union {msg} \union evidence
/\ decision' = [decision EXCEPT ![p] = v] \* update the decision, line 51
\* The original algorithm does not have 'DECIDED', but it increments the height.
\* We introduced 'DECIDED' here to prevent the process from changing its decision.
/\ step' = [step EXCEPT ![p] = "DECIDED"]
/\ UNCHANGED <<round, lockedValue, lockedRound, validValue,
validRound, msgsPropose, msgsPrevote, msgsPrecommit>>
/\ action' = "UponProposalInPrecommitNoDecision"
\* the actions below are not essential for safety, but added for completeness
\* lines 20-21 + 57-60
OnTimeoutPropose(p) ==
/\ step[p] = "PROPOSE"
/\ p /= Proposer[round[p]]
/\ BroadcastPrevote(p, round[p], NilValue)
/\ step' = [step EXCEPT ![p] = "PREVOTE"]
/\ UNCHANGED <<round, lockedValue, lockedRound, validValue,
validRound, decision, evidence, msgsPropose, msgsPrecommit>>
/\ action' = "OnTimeoutPropose"
\* lines 44-46
OnQuorumOfNilPrevotes(p) ==
/\ step[p] = "PREVOTE"
/\ LET PV == { m \in msgsPrevote[round[p]]: m.id = Id(NilValue) } IN
/\ Cardinality(PV) >= THRESHOLD2 \* line 36
/\ evidence' = PV \union evidence
/\ BroadcastPrecommit(p, round[p], Id(NilValue))
/\ step' = [step EXCEPT ![p] = "PRECOMMIT"]
/\ UNCHANGED <<round, lockedValue, lockedRound, validValue,
validRound, decision, msgsPropose, msgsPrevote>>
/\ action' = "OnQuorumOfNilPrevotes"
\* lines 55-56
OnRoundCatchup(p) ==
\E r \in {rr \in Rounds: rr > round[p]}:
LET RoundMsgs == msgsPropose[r] \union msgsPrevote[r] \union msgsPrecommit[r] IN
\E MyEvidence \in SUBSET RoundMsgs:
LET Faster == { m.src: m \in MyEvidence } IN
/\ Cardinality(Faster) >= THRESHOLD1
/\ evidence' = MyEvidence \union evidence
/\ StartRound(p, r)
/\ UNCHANGED <<decision, lockedValue, lockedRound, validValue,
validRound, msgsPropose, msgsPrevote, msgsPrecommit>>
/\ action' = "OnRoundCatchup"
(*
* A system transition. In this specificatiom, the system may eventually deadlock,
* e.g., when all processes decide. This is expected behavior, as we focus on safety.
*)
Next ==
\E p \in Corr:
\/ InsertProposal(p)
\/ UponProposalInPropose(p)
\/ UponProposalInProposeAndPrevote(p)
\/ UponQuorumOfPrevotesAny(p)
\/ UponProposalInPrevoteOrCommitAndPrevote(p)
\/ UponQuorumOfPrecommitsAny(p)
\/ UponProposalInPrecommitNoDecision(p)
\* the actions below are not essential for safety, but added for completeness
\/ OnTimeoutPropose(p)
\/ OnQuorumOfNilPrevotes(p)
\/ OnRoundCatchup(p)
(**************************** FORK SCENARIOS ***************************)
\* equivocation by a process p
EquivocationBy(p) ==
\E m1, m2 \in evidence:
/\ m1 /= m2
/\ m1.src = p
/\ m2.src = p
/\ m1.round = m2.round
/\ m1.type = m2.type
\* amnesic behavior by a process p
AmnesiaBy(p) ==
\E r1, r2 \in Rounds:
/\ r1 < r2
/\ \E v1, v2 \in ValidValues:
/\ v1 /= v2
/\ AsMsg([type |-> "PRECOMMIT", src |-> p,
round |-> r1, id |-> Id(v1)]) \in evidence
/\ AsMsg([type |-> "PREVOTE", src |-> p,
round |-> r2, id |-> Id(v2)]) \in evidence
/\ \A r \in { rnd \in Rounds: r1 <= rnd /\ rnd < r2 }:
LET prevotes ==
{ m \in evidence:
m.type = "PREVOTE" /\ m.round = r /\ m.id = Id(v2) }
IN
Cardinality(prevotes) < THRESHOLD2
(******************************** PROPERTIES ***************************************)
\* the safety property -- agreement
Agreement ==
\A p, q \in Corr:
\/ decision[p] = NilValue
\/ decision[q] = NilValue
\/ decision[p] = decision[q]
\* the protocol validity
Validity ==
\A p \in Corr: decision[p] \in ValidValues \union {NilValue}
(*
The protocol safety. Two cases are possible:
1. There is no fork, that is, Agreement holds true.
2. A subset of faulty processes demonstrates equivocation or amnesia.
*)
Accountability ==
\/ Agreement
\/ \E Detectable \in SUBSET Faulty:
/\ Cardinality(Detectable) >= THRESHOLD1
/\ \A p \in Detectable:
EquivocationBy(p) \/ AmnesiaBy(p)
(****************** FALSE INVARIANTS TO PRODUCE EXAMPLES ***********************)
\* This property is violated. You can check it to see how amnesic behavior
\* appears in the evidence variable.
NoAmnesia ==
\A p \in Faulty: ~AmnesiaBy(p)
\* This property is violated. You can check it to see an example of equivocation.
NoEquivocation ==
\A p \in Faulty: ~EquivocationBy(p)
\* This property is violated. You can check it to see an example of agreement.
\* It is not exactly ~Agreement, as we do not want to see the states where
\* decision[p] = NilValue
NoAgreement ==
\A p, q \in Corr:
(p /= q /\ decision[p] /= NilValue /\ decision[q] /= NilValue)
=> decision[p] /= decision[q]
\* Either agreement holds, or the faulty processes indeed demonstrate amnesia.
\* This property is violated. A counterexample should demonstrate equivocation.
AgreementOrAmnesia ==
Agreement \/ (\A p \in Faulty: AmnesiaBy(p))
\* We expect this property to be violated. It shows us a protocol run,
\* where one faulty process demonstrates amnesia without equivocation.
\* However, the absence of amnesia
\* is a tough constraint for Apalache. It has not reported a counterexample
\* for n=4,f=2, length <= 5.
ShowMeAmnesiaWithoutEquivocation ==
(~Agreement /\ \E p \in Faulty: ~EquivocationBy(p))
=> \A p \in Faulty: ~AmnesiaBy(p)
\* This property is violated on n=4,f=2, length=4 in less than 10 min.
\* Two faulty processes may demonstrate amnesia without equivocation.
AmnesiaImpliesEquivocation ==
(\E p \in Faulty: AmnesiaBy(p)) => (\E q \in Faulty: EquivocationBy(q))
(*
This property is violated. You can check it to see that all correct processes
may reach MaxRound without making a decision.
*)
NeverUndecidedInMaxRound ==
LET AllInMax == \A p \in Corr: round[p] = MaxRound
AllDecided == \A p \in Corr: decision[p] /= NilValue
IN
AllInMax => AllDecided
=============================================================================

1063
spec/light-client/accountability/results/001indinv-apalache-mem-log.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

1141
spec/light-client/accountability/results/001indinv-apalache-mem.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

1015
spec/light-client/accountability/results/001indinv-apalache-ncells.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

1133
spec/light-client/accountability/results/001indinv-apalache-nclauses.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

61
spec/light-client/accountability/results/001indinv-apalache-report.md Normal file
View File

@@ -0,0 +1,61 @@
# Results of 001indinv-apalache
## 1. Awesome plots
### 1.1. Time (logarithmic scale)
![time-log](001indinv-apalache-time-log.svg "Time Log")
### 1.2. Time (linear)
![time-log](001indinv-apalache-time.svg "Time Log")
### 1.3. Memory (logarithmic scale)
![mem-log](001indinv-apalache-mem-log.svg "Memory Log")
### 1.4. Memory (linear)
![mem](001indinv-apalache-mem.svg "Memory Log")
### 1.5. Number of arena cells (linear)
![ncells](001indinv-apalache-ncells.svg "Number of arena cells")
### 1.6. Number of SMT clauses (linear)
![nclauses](001indinv-apalache-nclauses.svg "Number of SMT clauses")
## 2. Input parameters
no | filename | tool | timeout | init | inv | next | args
----|----------------|------------|-----------|------------|------------------|--------|------------------------------
1 | MC_n4_f1.tla | apalache | 10h | TypedInv | TypedInv | | --length=1 --cinit=ConstInit
2 | MC_n4_f2.tla | apalache | 10h | TypedInv | TypedInv | | --length=1 --cinit=ConstInit
3 | MC_n5_f1.tla | apalache | 10h | TypedInv | TypedInv | | --length=1 --cinit=ConstInit
4 | MC_n5_f2.tla | apalache | 10h | TypedInv | TypedInv | | --length=1 --cinit=ConstInit
5 | MC_n4_f1.tla | apalache | 20h | Init | TypedInv | | --length=0 --cinit=ConstInit
6 | MC_n4_f2.tla | apalache | 20h | Init | TypedInv | | --length=0 --cinit=ConstInit
7 | MC_n5_f1.tla | apalache | 20h | Init | TypedInv | | --length=0 --cinit=ConstInit
8 | MC_n5_f2.tla | apalache | 20h | Init | TypedInv | | --length=0 --cinit=ConstInit
9 | MC_n4_f1.tla | apalache | 20h | TypedInv | Agreement | | --length=0 --cinit=ConstInit
10 | MC_n4_f2.tla | apalache | 20h | TypedInv | Accountability | | --length=0 --cinit=ConstInit
11 | MC_n5_f1.tla | apalache | 20h | TypedInv | Agreement | | --length=0 --cinit=ConstInit
12 | MC_n5_f2.tla | apalache | 20h | TypedInv | Accountability | | --length=0 --cinit=ConstInit
## 3. Detailed results: 001indinv-apalache-unstable.csv
01:no | 02:tool | 03:status | 04:time_sec | 05:depth | 05:mem_kb | 10:ninit_trans | 11:ninit_trans | 12:ncells | 13:nclauses | 14:navg_clause_len
-------|------------|-------------|---------------|------------|-------------|------------------|------------------|-------------|---------------|--------------------
1 | apalache | NoError | 11m | 1 | 3.0GB | 0 | 0 | 217K | 1.0M | 89
2 | apalache | NoError | 11m | 1 | 3.0GB | 0 | 0 | 207K | 1.0M | 88
3 | apalache | NoError | 16m | 1 | 4.0GB | 0 | 0 | 311K | 2.0M | 101
4 | apalache | NoError | 14m | 1 | 3.0GB | 0 | 0 | 290K | 1.0M | 103
5 | apalache | NoError | 9s | 0 | 563MB | 0 | 0 | 2.0K | 14K | 42
6 | apalache | NoError | 10s | 0 | 657MB | 0 | 0 | 2.0K | 28K | 43
7 | apalache | NoError | 8s | 0 | 635MB | 0 | 0 | 2.0K | 17K | 44
8 | apalache | NoError | 10s | 0 | 667MB | 0 | 0 | 3.0K | 32K | 45
9 | apalache | NoError | 5m05s | 0 | 2.0GB | 0 | 0 | 196K | 889K | 108
10 | apalache | NoError | 8m08s | 0 | 6.0GB | 0 | 0 | 2.0M | 3.0M | 34
11 | apalache | NoError | 9m09s | 0 | 3.0GB | 0 | 0 | 284K | 1.0M | 128
12 | apalache | NoError | 14m | 0 | 7.0GB | 0 | 0 | 4.0M | 5.0M | 38

1134
spec/light-client/accountability/results/001indinv-apalache-time-log.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

957
spec/light-client/accountability/results/001indinv-apalache-time.svg Normal file
View File

@@ -0,0 +1,957 @@
<?xml version="1.0" encoding="utf-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Created with matplotlib (https://matplotlib.org/) -->
<svg height="345.6pt" version="1.1" viewBox="0 0 460.8 345.6" width="460.8pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<metadata>
<rdf:RDF xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<cc:Work>
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
<dc:date>2020-12-11T20:07:39.136767</dc:date>
<dc:format>image/svg+xml</dc:format>
<dc:creator>
<cc:Agent>
<dc:title>Matplotlib v3.3.3, https://matplotlib.org/</dc:title>
</cc:Agent>
</dc:creator>
</cc:Work>
</rdf:RDF>
</metadata>
<defs>
<style type="text/css">*{stroke-linecap:butt;stroke-linejoin:round;}</style>
</defs>
<g id="figure_1">
<g id="patch_1">
<path d="M 0 345.6
L 460.8 345.6
L 460.8 0
L 0 0
z
" style="fill:#ffffff;"/>
</g>
<g id="axes_1">
<g id="patch_2">
<path d="M 57.6 307.584
L 414.72 307.584
L 414.72 41.472
L 57.6 41.472
z
" style="fill:#ffffff;"/>
</g>
<g id="matplotlib.axis_1">
<g id="xtick_1">
<g id="line2d_1">
<path clip-path="url(#p902cfd873e)" d="M 103.346777 307.584
L 103.346777 41.472
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_2">
<defs>
<path d="M 0 0
L 0 3.5
" id="m6c16508061" style="stroke:#000000;stroke-width:0.8;"/>
</defs>
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="103.346777" xlink:href="#m6c16508061" y="307.584"/>
</g>
</g>
<g id="text_1">
<!-- 2 -->
<g transform="translate(100.165527 322.182437)scale(0.1 -0.1)">
<defs>
<path d="M 19.1875 8.296875
L 53.609375 8.296875
L 53.609375 0
L 7.328125 0
L 7.328125 8.296875
Q 12.9375 14.109375 22.625 23.890625
Q 32.328125 33.6875 34.8125 36.53125
Q 39.546875 41.84375 41.421875 45.53125
Q 43.3125 49.21875 43.3125 52.78125
Q 43.3125 58.59375 39.234375 62.25
Q 35.15625 65.921875 28.609375 65.921875
Q 23.96875 65.921875 18.8125 64.3125
Q 13.671875 62.703125 7.8125 59.421875
L 7.8125 69.390625
Q 13.765625 71.78125 18.9375 73
Q 24.125 74.21875 28.421875 74.21875
Q 39.75 74.21875 46.484375 68.546875
Q 53.21875 62.890625 53.21875 53.421875
Q 53.21875 48.921875 51.53125 44.890625
Q 49.859375 40.875 45.40625 35.40625
Q 44.1875 33.984375 37.640625 27.21875
Q 31.109375 20.453125 19.1875 8.296875
z
" id="DejaVuSans-50"/>
</defs>
<use xlink:href="#DejaVuSans-50"/>
</g>
</g>
</g>
<g id="xtick_2">
<g id="line2d_3">
<path clip-path="url(#p902cfd873e)" d="M 162.374876 307.584
L 162.374876 41.472
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_4">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="162.374876" xlink:href="#m6c16508061" y="307.584"/>
</g>
</g>
<g id="text_2">
<!-- 4 -->
<g transform="translate(159.193626 322.182437)scale(0.1 -0.1)">
<defs>
<path d="M 37.796875 64.3125
L 12.890625 25.390625
L 37.796875 25.390625
z
M 35.203125 72.90625
L 47.609375 72.90625
L 47.609375 25.390625
L 58.015625 25.390625
L 58.015625 17.1875
L 47.609375 17.1875
L 47.609375 0
L 37.796875 0
L 37.796875 17.1875
L 4.890625 17.1875
L 4.890625 26.703125
z
" id="DejaVuSans-52"/>
</defs>
<use xlink:href="#DejaVuSans-52"/>
</g>
</g>
</g>
<g id="xtick_3">
<g id="line2d_5">
<path clip-path="url(#p902cfd873e)" d="M 221.402975 307.584
L 221.402975 41.472
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_6">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="221.402975" xlink:href="#m6c16508061" y="307.584"/>
</g>
</g>
<g id="text_3">
<!-- 6 -->
<g transform="translate(218.221725 322.182437)scale(0.1 -0.1)">
<defs>
<path d="M 33.015625 40.375
Q 26.375 40.375 22.484375 35.828125
Q 18.609375 31.296875 18.609375 23.390625
Q 18.609375 15.53125 22.484375 10.953125
Q 26.375 6.390625 33.015625 6.390625
Q 39.65625 6.390625 43.53125 10.953125
Q 47.40625 15.53125 47.40625 23.390625
Q 47.40625 31.296875 43.53125 35.828125
Q 39.65625 40.375 33.015625 40.375
z
M 52.59375 71.296875
L 52.59375 62.3125
Q 48.875 64.0625 45.09375 64.984375
Q 41.3125 65.921875 37.59375 65.921875
Q 27.828125 65.921875 22.671875 59.328125
Q 17.53125 52.734375 16.796875 39.40625
Q 19.671875 43.65625 24.015625 45.921875
Q 28.375 48.1875 33.59375 48.1875
Q 44.578125 48.1875 50.953125 41.515625
Q 57.328125 34.859375 57.328125 23.390625
Q 57.328125 12.15625 50.6875 5.359375
Q 44.046875 -1.421875 33.015625 -1.421875
Q 20.359375 -1.421875 13.671875 8.265625
Q 6.984375 17.96875 6.984375 36.375
Q 6.984375 53.65625 15.1875 63.9375
Q 23.390625 74.21875 37.203125 74.21875
Q 40.921875 74.21875 44.703125 73.484375
Q 48.484375 72.75 52.59375 71.296875
z
" id="DejaVuSans-54"/>
</defs>
<use xlink:href="#DejaVuSans-54"/>
</g>
</g>
</g>
<g id="xtick_4">
<g id="line2d_7">
<path clip-path="url(#p902cfd873e)" d="M 280.431074 307.584
L 280.431074 41.472
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_8">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="280.431074" xlink:href="#m6c16508061" y="307.584"/>
</g>
</g>
<g id="text_4">
<!-- 8 -->
<g transform="translate(277.249824 322.182437)scale(0.1 -0.1)">
<defs>
<path d="M 31.78125 34.625
Q 24.75 34.625 20.71875 30.859375
Q 16.703125 27.09375 16.703125 20.515625
Q 16.703125 13.921875 20.71875 10.15625
Q 24.75 6.390625 31.78125 6.390625
Q 38.8125 6.390625 42.859375 10.171875
Q 46.921875 13.96875 46.921875 20.515625
Q 46.921875 27.09375 42.890625 30.859375
Q 38.875 34.625 31.78125 34.625
z
M 21.921875 38.8125
Q 15.578125 40.375 12.03125 44.71875
Q 8.5 49.078125 8.5 55.328125
Q 8.5 64.0625 14.71875 69.140625
Q 20.953125 74.21875 31.78125 74.21875
Q 42.671875 74.21875 48.875 69.140625
Q 55.078125 64.0625 55.078125 55.328125
Q 55.078125 49.078125 51.53125 44.71875
Q 48 40.375 41.703125 38.8125
Q 48.828125 37.15625 52.796875 32.3125
Q 56.78125 27.484375 56.78125 20.515625
Q 56.78125 9.90625 50.3125 4.234375
Q 43.84375 -1.421875 31.78125 -1.421875
Q 19.734375 -1.421875 13.25 4.234375
Q 6.78125 9.90625 6.78125 20.515625
Q 6.78125 27.484375 10.78125 32.3125
Q 14.796875 37.15625 21.921875 38.8125
z
M 18.3125 54.390625
Q 18.3125 48.734375 21.84375 45.5625
Q 25.390625 42.390625 31.78125 42.390625
Q 38.140625 42.390625 41.71875 45.5625
Q 45.3125 48.734375 45.3125 54.390625
Q 45.3125 60.0625 41.71875 63.234375
Q 38.140625 66.40625 31.78125 66.40625
Q 25.390625 66.40625 21.84375 63.234375
Q 18.3125 60.0625 18.3125 54.390625
z
" id="DejaVuSans-56"/>
</defs>
<use xlink:href="#DejaVuSans-56"/>
</g>
</g>
</g>
<g id="xtick_5">
<g id="line2d_9">
<path clip-path="url(#p902cfd873e)" d="M 339.459174 307.584
L 339.459174 41.472
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_10">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="339.459174" xlink:href="#m6c16508061" y="307.584"/>
</g>
</g>
<g id="text_5">
<!-- 10 -->
<g transform="translate(333.096674 322.182437)scale(0.1 -0.1)">
<defs>
<path d="M 12.40625 8.296875
L 28.515625 8.296875
L 28.515625 63.921875
L 10.984375 60.40625
L 10.984375 69.390625
L 28.421875 72.90625
L 38.28125 72.90625
L 38.28125 8.296875
L 54.390625 8.296875
L 54.390625 0
L 12.40625 0
z
" id="DejaVuSans-49"/>
<path d="M 31.78125 66.40625
Q 24.171875 66.40625 20.328125 58.90625
Q 16.5 51.421875 16.5 36.375
Q 16.5 21.390625 20.328125 13.890625
Q 24.171875 6.390625 31.78125 6.390625
Q 39.453125 6.390625 43.28125 13.890625
Q 47.125 21.390625 47.125 36.375
Q 47.125 51.421875 43.28125 58.90625
Q 39.453125 66.40625 31.78125 66.40625
z
M 31.78125 74.21875
Q 44.046875 74.21875 50.515625 64.515625
Q 56.984375 54.828125 56.984375 36.375
Q 56.984375 17.96875 50.515625 8.265625
Q 44.046875 -1.421875 31.78125 -1.421875
Q 19.53125 -1.421875 13.0625 8.265625
Q 6.59375 17.96875 6.59375 36.375
Q 6.59375 54.828125 13.0625 64.515625
Q 19.53125 74.21875 31.78125 74.21875
z
" id="DejaVuSans-48"/>
</defs>
<use xlink:href="#DejaVuSans-49"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="xtick_6">
<g id="line2d_11">
<path clip-path="url(#p902cfd873e)" d="M 398.487273 307.584
L 398.487273 41.472
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_12">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="398.487273" xlink:href="#m6c16508061" y="307.584"/>
</g>
</g>
<g id="text_6">
<!-- 12 -->
<g transform="translate(392.124773 322.182437)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-49"/>
<use x="63.623047" xlink:href="#DejaVuSans-50"/>
</g>
</g>
</g>
<g id="text_7">
<!-- benchmark -->
<g transform="translate(207.937344 335.860562)scale(0.1 -0.1)">
<defs>
<path d="M 48.6875 27.296875
Q 48.6875 37.203125 44.609375 42.84375
Q 40.53125 48.484375 33.40625 48.484375
Q 26.265625 48.484375 22.1875 42.84375
Q 18.109375 37.203125 18.109375 27.296875
Q 18.109375 17.390625 22.1875 11.75
Q 26.265625 6.109375 33.40625 6.109375
Q 40.53125 6.109375 44.609375 11.75
Q 48.6875 17.390625 48.6875 27.296875
z
M 18.109375 46.390625
Q 20.953125 51.265625 25.265625 53.625
Q 29.59375 56 35.59375 56
Q 45.5625 56 51.78125 48.09375
Q 58.015625 40.1875 58.015625 27.296875
Q 58.015625 14.40625 51.78125 6.484375
Q 45.5625 -1.421875 35.59375 -1.421875
Q 29.59375 -1.421875 25.265625 0.953125
Q 20.953125 3.328125 18.109375 8.203125
L 18.109375 0
L 9.078125 0
L 9.078125 75.984375
L 18.109375 75.984375
z
" id="DejaVuSans-98"/>
<path d="M 56.203125 29.59375
L 56.203125 25.203125
L 14.890625 25.203125
Q 15.484375 15.921875 20.484375 11.0625
Q 25.484375 6.203125 34.421875 6.203125
Q 39.59375 6.203125 44.453125 7.46875
Q 49.3125 8.734375 54.109375 11.28125
L 54.109375 2.78125
Q 49.265625 0.734375 44.1875 -0.34375
Q 39.109375 -1.421875 33.890625 -1.421875
Q 20.796875 -1.421875 13.15625 6.1875
Q 5.515625 13.8125 5.515625 26.8125
Q 5.515625 40.234375 12.765625 48.109375
Q 20.015625 56 32.328125 56
Q 43.359375 56 49.78125 48.890625
Q 56.203125 41.796875 56.203125 29.59375
z
M 47.21875 32.234375
Q 47.125 39.59375 43.09375 43.984375
Q 39.0625 48.390625 32.421875 48.390625
Q 24.90625 48.390625 20.390625 44.140625
Q 15.875 39.890625 15.1875 32.171875
z
" id="DejaVuSans-101"/>
<path d="M 54.890625 33.015625
L 54.890625 0
L 45.90625 0
L 45.90625 32.71875
Q 45.90625 40.484375 42.875 44.328125
Q 39.84375 48.1875 33.796875 48.1875
Q 26.515625 48.1875 22.3125 43.546875
Q 18.109375 38.921875 18.109375 30.90625
L 18.109375 0
L 9.078125 0
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.1875
Q 21.34375 51.125 25.703125 53.5625
Q 30.078125 56 35.796875 56
Q 45.21875 56 50.046875 50.171875
Q 54.890625 44.34375 54.890625 33.015625
z
" id="DejaVuSans-110"/>
<path d="M 48.78125 52.59375
L 48.78125 44.1875
Q 44.96875 46.296875 41.140625 47.34375
Q 37.3125 48.390625 33.40625 48.390625
Q 24.65625 48.390625 19.8125 42.84375
Q 14.984375 37.3125 14.984375 27.296875
Q 14.984375 17.28125 19.8125 11.734375
Q 24.65625 6.203125 33.40625 6.203125
Q 37.3125 6.203125 41.140625 7.25
Q 44.96875 8.296875 48.78125 10.40625
L 48.78125 2.09375
Q 45.015625 0.34375 40.984375 -0.53125
Q 36.96875 -1.421875 32.421875 -1.421875
Q 20.0625 -1.421875 12.78125 6.34375
Q 5.515625 14.109375 5.515625 27.296875
Q 5.515625 40.671875 12.859375 48.328125
Q 20.21875 56 33.015625 56
Q 37.15625 56 41.109375 55.140625
Q 45.0625 54.296875 48.78125 52.59375
z
" id="DejaVuSans-99"/>
<path d="M 54.890625 33.015625
L 54.890625 0
L 45.90625 0
L 45.90625 32.71875
Q 45.90625 40.484375 42.875 44.328125
Q 39.84375 48.1875 33.796875 48.1875
Q 26.515625 48.1875 22.3125 43.546875
Q 18.109375 38.921875 18.109375 30.90625
L 18.109375 0
L 9.078125 0
L 9.078125 75.984375
L 18.109375 75.984375
L 18.109375 46.1875
Q 21.34375 51.125 25.703125 53.5625
Q 30.078125 56 35.796875 56
Q 45.21875 56 50.046875 50.171875
Q 54.890625 44.34375 54.890625 33.015625
z
" id="DejaVuSans-104"/>
<path d="M 52 44.1875
Q 55.375 50.25 60.0625 53.125
Q 64.75 56 71.09375 56
Q 79.640625 56 84.28125 50.015625
Q 88.921875 44.046875 88.921875 33.015625
L 88.921875 0
L 79.890625 0
L 79.890625 32.71875
Q 79.890625 40.578125 77.09375 44.375
Q 74.3125 48.1875 68.609375 48.1875
Q 61.625 48.1875 57.5625 43.546875
Q 53.515625 38.921875 53.515625 30.90625
L 53.515625 0
L 44.484375 0
L 44.484375 32.71875
Q 44.484375 40.625 41.703125 44.40625
Q 38.921875 48.1875 33.109375 48.1875
Q 26.21875 48.1875 22.15625 43.53125
Q 18.109375 38.875 18.109375 30.90625
L 18.109375 0
L 9.078125 0
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.1875
Q 21.1875 51.21875 25.484375 53.609375
Q 29.78125 56 35.6875 56
Q 41.65625 56 45.828125 52.96875
Q 50 49.953125 52 44.1875
z
" id="DejaVuSans-109"/>
<path d="M 34.28125 27.484375
Q 23.390625 27.484375 19.1875 25
Q 14.984375 22.515625 14.984375 16.5
Q 14.984375 11.71875 18.140625 8.90625
Q 21.296875 6.109375 26.703125 6.109375
Q 34.1875 6.109375 38.703125 11.40625
Q 43.21875 16.703125 43.21875 25.484375
L 43.21875 27.484375
z
M 52.203125 31.203125
L 52.203125 0
L 43.21875 0
L 43.21875 8.296875
Q 40.140625 3.328125 35.546875 0.953125
Q 30.953125 -1.421875 24.3125 -1.421875
Q 15.921875 -1.421875 10.953125 3.296875
Q 6 8.015625 6 15.921875
Q 6 25.140625 12.171875 29.828125
Q 18.359375 34.515625 30.609375 34.515625
L 43.21875 34.515625
L 43.21875 35.40625
Q 43.21875 41.609375 39.140625 45
Q 35.0625 48.390625 27.6875 48.390625
Q 23 48.390625 18.546875 47.265625
Q 14.109375 46.140625 10.015625 43.890625
L 10.015625 52.203125
Q 14.9375 54.109375 19.578125 55.046875
Q 24.21875 56 28.609375 56
Q 40.484375 56 46.34375 49.84375
Q 52.203125 43.703125 52.203125 31.203125
z
" id="DejaVuSans-97"/>
<path d="M 41.109375 46.296875
Q 39.59375 47.171875 37.8125 47.578125
Q 36.03125 48 33.890625 48
Q 26.265625 48 22.1875 43.046875
Q 18.109375 38.09375 18.109375 28.8125
L 18.109375 0
L 9.078125 0
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.1875
Q 20.953125 51.171875 25.484375 53.578125
Q 30.03125 56 36.53125 56
Q 37.453125 56 38.578125 55.875
Q 39.703125 55.765625 41.0625 55.515625
z
" id="DejaVuSans-114"/>
<path d="M 9.078125 75.984375
L 18.109375 75.984375
L 18.109375 31.109375
L 44.921875 54.6875
L 56.390625 54.6875
L 27.390625 29.109375
L 57.625 0
L 45.90625 0
L 18.109375 26.703125
L 18.109375 0
L 9.078125 0
z
" id="DejaVuSans-107"/>
</defs>
<use xlink:href="#DejaVuSans-98"/>
<use x="63.476562" xlink:href="#DejaVuSans-101"/>
<use x="125" xlink:href="#DejaVuSans-110"/>
<use x="188.378906" xlink:href="#DejaVuSans-99"/>
<use x="243.359375" xlink:href="#DejaVuSans-104"/>
<use x="306.738281" xlink:href="#DejaVuSans-109"/>
<use x="404.150391" xlink:href="#DejaVuSans-97"/>
<use x="465.429688" xlink:href="#DejaVuSans-114"/>
<use x="506.542969" xlink:href="#DejaVuSans-107"/>
</g>
</g>
</g>
<g id="matplotlib.axis_2">
<g id="ytick_1">
<g id="line2d_13">
<path clip-path="url(#p902cfd873e)" d="M 57.6 297.404198
L 414.72 297.404198
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_14">
<defs>
<path d="M 0 0
L -3.5 0
" id="m92e578bc9b" style="stroke:#000000;stroke-width:0.8;"/>
</defs>
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="57.6" xlink:href="#m92e578bc9b" y="297.404198"/>
</g>
</g>
<g id="text_8">
<!-- 0 -->
<g transform="translate(44.2375 301.203417)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="ytick_2">
<g id="line2d_15">
<path clip-path="url(#p902cfd873e)" d="M 57.6 249.499248
L 414.72 249.499248
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_16">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="57.6" xlink:href="#m92e578bc9b" y="249.499248"/>
</g>
</g>
<g id="text_9">
<!-- 200 -->
<g transform="translate(31.5125 253.298466)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-50"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
<use x="127.246094" xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="ytick_3">
<g id="line2d_17">
<path clip-path="url(#p902cfd873e)" d="M 57.6 201.594297
L 414.72 201.594297
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_18">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="57.6" xlink:href="#m92e578bc9b" y="201.594297"/>
</g>
</g>
<g id="text_10">
<!-- 400 -->
<g transform="translate(31.5125 205.393516)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-52"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
<use x="127.246094" xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="ytick_4">
<g id="line2d_19">
<path clip-path="url(#p902cfd873e)" d="M 57.6 153.689347
L 414.72 153.689347
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_20">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="57.6" xlink:href="#m92e578bc9b" y="153.689347"/>
</g>
</g>
<g id="text_11">
<!-- 600 -->
<g transform="translate(31.5125 157.488565)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-54"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
<use x="127.246094" xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="ytick_5">
<g id="line2d_21">
<path clip-path="url(#p902cfd873e)" d="M 57.6 105.784396
L 414.72 105.784396
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_22">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="57.6" xlink:href="#m92e578bc9b" y="105.784396"/>
</g>
</g>
<g id="text_12">
<!-- 800 -->
<g transform="translate(31.5125 109.583615)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-56"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
<use x="127.246094" xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="ytick_6">
<g id="line2d_23">
<path clip-path="url(#p902cfd873e)" d="M 57.6 57.879446
L 414.72 57.879446
" style="fill:none;stroke:#b0b0b0;stroke-linecap:square;stroke-opacity:0.2;stroke-width:0.8;"/>
</g>
<g id="line2d_24">
<g>
<use style="stroke:#000000;stroke-width:0.8;" x="57.6" xlink:href="#m92e578bc9b" y="57.879446"/>
</g>
</g>
<g id="text_13">
<!-- 1000 -->
<g transform="translate(25.15 61.678664)scale(0.1 -0.1)">
<use xlink:href="#DejaVuSans-49"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
<use x="127.246094" xlink:href="#DejaVuSans-48"/>
<use x="190.869141" xlink:href="#DejaVuSans-48"/>
</g>
</g>
</g>
<g id="text_14">
<!-- time, sec -->
<g transform="translate(19.070312 197.432687)rotate(-90)scale(0.1 -0.1)">
<defs>
<path d="M 18.3125 70.21875
L 18.3125 54.6875
L 36.8125 54.6875
L 36.8125 47.703125
L 18.3125 47.703125
L 18.3125 18.015625
Q 18.3125 11.328125 20.140625 9.421875
Q 21.96875 7.515625 27.59375 7.515625
L 36.8125 7.515625
L 36.8125 0
L 27.59375 0
Q 17.1875 0 13.234375 3.875
Q 9.28125 7.765625 9.28125 18.015625
L 9.28125 47.703125
L 2.6875 47.703125
L 2.6875 54.6875
L 9.28125 54.6875
L 9.28125 70.21875
z
" id="DejaVuSans-116"/>
<path d="M 9.421875 54.6875
L 18.40625 54.6875
L 18.40625 0
L 9.421875 0
z
M 9.421875 75.984375
L 18.40625 75.984375
L 18.40625 64.59375
L 9.421875 64.59375
z
" id="DejaVuSans-105"/>
<path d="M 11.71875 12.40625
L 22.015625 12.40625
L 22.015625 4
L 14.015625 -11.625
L 7.71875 -11.625
L 11.71875 4
z
" id="DejaVuSans-44"/>
<path id="DejaVuSans-32"/>
<path d="M 44.28125 53.078125
L 44.28125 44.578125
Q 40.484375 46.53125 36.375 47.5
Q 32.28125 48.484375 27.875 48.484375
Q 21.1875 48.484375 17.84375 46.4375
Q 14.5 44.390625 14.5 40.28125
Q 14.5 37.15625 16.890625 35.375
Q 19.28125 33.59375 26.515625 31.984375
L 29.59375 31.296875
Q 39.15625 29.25 43.1875 25.515625
Q 47.21875 21.78125 47.21875 15.09375
Q 47.21875 7.46875 41.1875 3.015625
Q 35.15625 -1.421875 24.609375 -1.421875
Q 20.21875 -1.421875 15.453125 -0.5625
Q 10.6875 0.296875 5.421875 2
L 5.421875 11.28125
Q 10.40625 8.6875 15.234375 7.390625
Q 20.0625 6.109375 24.8125 6.109375
Q 31.15625 6.109375 34.5625 8.28125
Q 37.984375 10.453125 37.984375 14.40625
Q 37.984375 18.0625 35.515625 20.015625
Q 33.0625 21.96875 24.703125 23.78125
L 21.578125 24.515625
Q 13.234375 26.265625 9.515625 29.90625
Q 5.8125 33.546875 5.8125 39.890625
Q 5.8125 47.609375 11.28125 51.796875
Q 16.75 56 26.8125 56
Q 31.78125 56 36.171875 55.265625
Q 40.578125 54.546875 44.28125 53.078125
z
" id="DejaVuSans-115"/>
</defs>
<use xlink:href="#DejaVuSans-116"/>
<use x="39.208984" xlink:href="#DejaVuSans-105"/>
<use x="66.992188" xlink:href="#DejaVuSans-109"/>
<use x="164.404297" xlink:href="#DejaVuSans-101"/>
<use x="225.927734" xlink:href="#DejaVuSans-44"/>
<use x="257.714844" xlink:href="#DejaVuSans-32"/>
<use x="289.501953" xlink:href="#DejaVuSans-115"/>
<use x="341.601562" xlink:href="#DejaVuSans-101"/>
<use x="403.125" xlink:href="#DejaVuSans-99"/>
</g>
</g>
</g>
<g id="line2d_25">
<path clip-path="url(#p902cfd873e)" d="M 73.832727 128.778772
L 103.346777 129.976396
L 132.860826 53.568
L 162.374876 84.466693
L 191.888926 295.248475
L 221.402975 295.00895
L 250.917025 295.488
L 280.431074 295.00895
L 309.945124 215.965782
L 339.459174 173.569901
L 368.973223 156.803168
L 398.487273 86.622416
" style="fill:none;stroke:#ff0000;stroke-linecap:square;stroke-opacity:0.7;stroke-width:1.5;"/>
<defs>
<path d="M -3 3
L 3 3
L 3 -3
L -3 -3
z
" id="m40d9e306aa" style="stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;"/>
</defs>
<g clip-path="url(#p902cfd873e)">
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="73.832727" xlink:href="#m40d9e306aa" y="128.778772"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="103.346777" xlink:href="#m40d9e306aa" y="129.976396"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="132.860826" xlink:href="#m40d9e306aa" y="53.568"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="162.374876" xlink:href="#m40d9e306aa" y="84.466693"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="191.888926" xlink:href="#m40d9e306aa" y="295.248475"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="221.402975" xlink:href="#m40d9e306aa" y="295.00895"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="250.917025" xlink:href="#m40d9e306aa" y="295.488"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="280.431074" xlink:href="#m40d9e306aa" y="295.00895"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="309.945124" xlink:href="#m40d9e306aa" y="215.965782"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="339.459174" xlink:href="#m40d9e306aa" y="173.569901"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="368.973223" xlink:href="#m40d9e306aa" y="156.803168"/>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="398.487273" xlink:href="#m40d9e306aa" y="86.622416"/>
</g>
</g>
<g id="patch_3">
<path d="M 57.6 307.584
L 57.6 41.472
" style="fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;"/>
</g>
<g id="patch_4">
<path d="M 414.72 307.584
L 414.72 41.472
" style="fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;"/>
</g>
<g id="patch_5">
<path d="M 57.6 307.584
L 414.72 307.584
" style="fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;"/>
</g>
<g id="patch_6">
<path d="M 57.6 41.472
L 414.72 41.472
" style="fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;"/>
</g>
<g id="legend_1">
<g id="patch_7">
<path d="M 230.468437 64.150125
L 407.72 64.150125
Q 409.72 64.150125 409.72 62.150125
L 409.72 48.472
Q 409.72 46.472 407.72 46.472
L 230.468437 46.472
Q 228.468437 46.472 228.468437 48.472
L 228.468437 62.150125
Q 228.468437 64.150125 230.468437 64.150125
z
" style="fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;"/>
</g>
<g id="line2d_26">
<path d="M 232.468437 54.570438
L 252.468437 54.570438
" style="fill:none;stroke:#ff0000;stroke-linecap:square;stroke-opacity:0.7;stroke-width:1.5;"/>
</g>
<g id="line2d_27">
<g>
<use style="fill:#ff0000;fill-opacity:0.7;stroke:#ff0000;stroke-linejoin:miter;stroke-opacity:0.7;" x="242.468437" xlink:href="#m40d9e306aa" y="54.570438"/>
</g>
</g>
<g id="text_15">
<!-- 001indinv-apalache-unstable -->
<g transform="translate(260.468437 58.070438)scale(0.1 -0.1)">
<defs>
<path d="M 45.40625 46.390625
L 45.40625 75.984375
L 54.390625 75.984375
L 54.390625 0
L 45.40625 0
L 45.40625 8.203125
Q 42.578125 3.328125 38.25 0.953125
Q 33.9375 -1.421875 27.875 -1.421875
Q 17.96875 -1.421875 11.734375 6.484375
Q 5.515625 14.40625 5.515625 27.296875
Q 5.515625 40.1875 11.734375 48.09375
Q 17.96875 56 27.875 56
Q 33.9375 56 38.25 53.625
Q 42.578125 51.265625 45.40625 46.390625
z
M 14.796875 27.296875
Q 14.796875 17.390625 18.875 11.75
Q 22.953125 6.109375 30.078125 6.109375
Q 37.203125 6.109375 41.296875 11.75
Q 45.40625 17.390625 45.40625 27.296875
Q 45.40625 37.203125 41.296875 42.84375
Q 37.203125 48.484375 30.078125 48.484375
Q 22.953125 48.484375 18.875 42.84375
Q 14.796875 37.203125 14.796875 27.296875
z
" id="DejaVuSans-100"/>
<path d="M 2.984375 54.6875
L 12.5 54.6875
L 29.59375 8.796875
L 46.6875 54.6875
L 56.203125 54.6875
L 35.6875 0
L 23.484375 0
z
" id="DejaVuSans-118"/>
<path d="M 4.890625 31.390625
L 31.203125 31.390625
L 31.203125 23.390625
L 4.890625 23.390625
z
" id="DejaVuSans-45"/>
<path d="M 18.109375 8.203125
L 18.109375 -20.796875
L 9.078125 -20.796875
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.390625
Q 20.953125 51.265625 25.265625 53.625
Q 29.59375 56 35.59375 56
Q 45.5625 56 51.78125 48.09375
Q 58.015625 40.1875 58.015625 27.296875
Q 58.015625 14.40625 51.78125 6.484375
Q 45.5625 -1.421875 35.59375 -1.421875
Q 29.59375 -1.421875 25.265625 0.953125
Q 20.953125 3.328125 18.109375 8.203125
z
M 48.6875 27.296875
Q 48.6875 37.203125 44.609375 42.84375
Q 40.53125 48.484375 33.40625 48.484375
Q 26.265625 48.484375 22.1875 42.84375
Q 18.109375 37.203125 18.109375 27.296875
Q 18.109375 17.390625 22.1875 11.75
Q 26.265625 6.109375 33.40625 6.109375
Q 40.53125 6.109375 44.609375 11.75
Q 48.6875 17.390625 48.6875 27.296875
z
" id="DejaVuSans-112"/>
<path d="M 9.421875 75.984375
L 18.40625 75.984375
L 18.40625 0
L 9.421875 0
z
" id="DejaVuSans-108"/>
<path d="M 8.5 21.578125
L 8.5 54.6875
L 17.484375 54.6875
L 17.484375 21.921875
Q 17.484375 14.15625 20.5 10.265625
Q 23.53125 6.390625 29.59375 6.390625
Q 36.859375 6.390625 41.078125 11.03125
Q 45.3125 15.671875 45.3125 23.6875
L 45.3125 54.6875
L 54.296875 54.6875
L 54.296875 0
L 45.3125 0
L 45.3125 8.40625
Q 42.046875 3.421875 37.71875 1
Q 33.40625 -1.421875 27.6875 -1.421875
Q 18.265625 -1.421875 13.375 4.4375
Q 8.5 10.296875 8.5 21.578125
z
M 31.109375 56
z
" id="DejaVuSans-117"/>
</defs>
<use xlink:href="#DejaVuSans-48"/>
<use x="63.623047" xlink:href="#DejaVuSans-48"/>
<use x="127.246094" xlink:href="#DejaVuSans-49"/>
<use x="190.869141" xlink:href="#DejaVuSans-105"/>
<use x="218.652344" xlink:href="#DejaVuSans-110"/>
<use x="282.03125" xlink:href="#DejaVuSans-100"/>
<use x="345.507812" xlink:href="#DejaVuSans-105"/>
<use x="373.291016" xlink:href="#DejaVuSans-110"/>
<use x="436.669922" xlink:href="#DejaVuSans-118"/>
<use x="493.224609" xlink:href="#DejaVuSans-45"/>
<use x="529.308594" xlink:href="#DejaVuSans-97"/>
<use x="590.587891" xlink:href="#DejaVuSans-112"/>
<use x="654.064453" xlink:href="#DejaVuSans-97"/>
<use x="715.34375" xlink:href="#DejaVuSans-108"/>
<use x="743.126953" xlink:href="#DejaVuSans-97"/>
<use x="804.40625" xlink:href="#DejaVuSans-99"/>
<use x="859.386719" xlink:href="#DejaVuSans-104"/>
<use x="922.765625" xlink:href="#DejaVuSans-101"/>
<use x="984.289062" xlink:href="#DejaVuSans-45"/>
<use x="1020.373047" xlink:href="#DejaVuSans-117"/>
<use x="1083.751953" xlink:href="#DejaVuSans-110"/>
<use x="1147.130859" xlink:href="#DejaVuSans-115"/>
<use x="1199.230469" xlink:href="#DejaVuSans-116"/>
<use x="1238.439453" xlink:href="#DejaVuSans-97"/>
<use x="1299.71875" xlink:href="#DejaVuSans-98"/>
<use x="1363.195312" xlink:href="#DejaVuSans-108"/>
<use x="1390.978516" xlink:href="#DejaVuSans-101"/>
</g>
</g>
</g>
</g>
</g>
<defs>
<clipPath id="p902cfd873e">
<rect height="266.112" width="357.12" x="57.6" y="41.472"/>
</clipPath>
</defs>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 31 KiB

13
spec/light-client/accountability/results/001indinv-apalache-unstable.csv Normal file
View File

@@ -0,0 +1,13 @@
01:no,02:tool,03:status,04:time_sec,05:depth,05:mem_kb,10:ninit_trans,11:ninit_trans,12:ncells,13:nclauses,14:navg_clause_len
1,apalache,NoError,704,1,3215424,0,0,217385,1305718,89
2,apalache,NoError,699,1,3195020,0,0,207969,1341979,88
3,apalache,NoError,1018,1,4277060,0,0,311798,2028544,101
4,apalache,NoError,889,1,4080012,0,0,290989,1951616,103
5,apalache,NoError,9,0,577100,0,0,2045,14655,42
6,apalache,NoError,10,0,673772,0,0,2913,28213,43
7,apalache,NoError,8,0,651008,0,0,2214,17077,44
8,apalache,NoError,10,0,683188,0,0,3082,32651,45
9,apalache,NoError,340,0,3053848,0,0,196943,889859,108
10,apalache,NoError,517,0,6424536,0,0,2856378,3802779,34
11,apalache,NoError,587,0,4028516,0,0,284369,1343296,128
12,apalache,NoError,880,0,7881148,0,0,4382556,5778072,38
1 01:no 02:tool 03:status 04:time_sec 05:depth 05:mem_kb 10:ninit_trans 11:ninit_trans 12:ncells 13:nclauses 14:navg_clause_len
2 1 apalache NoError 704 1 3215424 0 0 217385 1305718 89
3 2 apalache NoError 699 1 3195020 0 0 207969 1341979 88
4 3 apalache NoError 1018 1 4277060 0 0 311798 2028544 101
5 4 apalache NoError 889 1 4080012 0 0 290989 1951616 103
6 5 apalache NoError 9 0 577100 0 0 2045 14655 42
7 6 apalache NoError 10 0 673772 0 0 2913 28213 43
8 7 apalache NoError 8 0 651008 0 0 2214 17077 44
9 8 apalache NoError 10 0 683188 0 0 3082 32651 45
10 9 apalache NoError 340 0 3053848 0 0 196943 889859 108
11 10 apalache NoError 517 0 6424536 0 0 2856378 3802779 34
12 11 apalache NoError 587 0 4028516 0 0 284369 1343296 128
13 12 apalache NoError 880 0 7881148 0 0 4382556 5778072 38

9
spec/light-client/accountability/run.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/bin/sh
#
# The script to run all experiments at once
export SCRIPTS_DIR=~/devl/apalache-tests/scripts
export BUILDS="unstable"
export BENCHMARK=001indinv-apalache
export RUN_SCRIPT=./run-all.sh # alternatively, use ./run-parallel.sh
make -e -f ~/devl/apalache-tests/Makefile.common

BIN
spec/light-client/assets/light-node-image.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 119 KiB

166
spec/light-client/attacks/Blockchain_003_draft.tla Normal file
View File

@@ -0,0 +1,166 @@
------------------------ MODULE Blockchain_003_draft -----------------------------
(*
This is a high-level specification of Tendermint blockchain
that is designed specifically for the light client.
Validators have the voting power of one. If you like to model various
voting powers, introduce multiple copies of the same validator
(do not forget to give them unique names though).
*)
EXTENDS Integers, FiniteSets, Apalache
Min(a, b) == IF a < b THEN a ELSE b
CONSTANT
AllNodes,
(* a set of all nodes that can act as validators (correct and faulty) *)
ULTIMATE_HEIGHT,
(* a maximal height that can be ever reached (modelling artifact) *)
TRUSTING_PERIOD
(* the period within which the validators are trusted *)
Heights == 1..ULTIMATE_HEIGHT (* possible heights *)
(* A commit is just a set of nodes who have committed the block *)
Commits == SUBSET AllNodes
(* The set of all block headers that can be on the blockchain.
This is a simplified version of the Block data structure in the actual implementation. *)
BlockHeaders == [
height: Heights,
\* the block height
time: Int,
\* the block timestamp in some integer units
lastCommit: Commits,
\* the nodes who have voted on the previous block, the set itself instead of a hash
(* in the implementation, only the hashes of V and NextV are stored in a block,
as V and NextV are stored in the application state *)
VS: SUBSET AllNodes,
\* the validators of this bloc. We store the validators instead of the hash.
NextVS: SUBSET AllNodes
\* the validators of the next block. We store the next validators instead of the hash.
]
(* A signed header is just a header together with a set of commits *)
LightBlocks == [header: BlockHeaders, Commits: Commits]
VARIABLES
refClock,
(* the current global time in integer units as perceived by the reference chain *)
blockchain,
(* A sequence of BlockHeaders, which gives us a bird view of the blockchain. *)
Faulty
(* A set of faulty nodes, which can act as validators. We assume that the set
of faulty processes is non-decreasing. If a process has recovered, it should
connect using a different id. *)
(* all variables, to be used with UNCHANGED *)
vars == <<refClock, blockchain, Faulty>>
(* The set of all correct nodes in a state *)
Corr == AllNodes \ Faulty
(* APALACHE annotations *)
a <: b == a \* type annotation
NT == STRING
NodeSet(S) == S <: {NT}
EmptyNodeSet == NodeSet({})
BT == [height |-> Int, time |-> Int, lastCommit |-> {NT}, VS |-> {NT}, NextVS |-> {NT}]
LBT == [header |-> BT, Commits |-> {NT}]
(* end of APALACHE annotations *)
(****************************** BLOCKCHAIN ************************************)
(* the header is still within the trusting period *)
InTrustingPeriod(header) ==
refClock < header.time + TRUSTING_PERIOD
(*
Given a function pVotingPower \in D -> Powers for some D \subseteq AllNodes
and pNodes \subseteq D, test whether the set pNodes \subseteq AllNodes has
more than 2/3 of voting power among the nodes in D.
*)
TwoThirds(pVS, pNodes) ==
LET TP == Cardinality(pVS)
SP == Cardinality(pVS \intersect pNodes)
IN
3 * SP > 2 * TP \* when thinking in real numbers, not integers: SP > 2.0 / 3.0 * TP
(*
Given a set of FaultyNodes, test whether the voting power of the correct nodes in D
is more than 2/3 of the voting power of the faulty nodes in D.
Parameters:
- pFaultyNodes is a set of nodes that are considered faulty
- pVS is a set of all validators, maybe including Faulty, intersecting with it, etc.
- pMaxFaultRatio is a pair <<a, b>> that limits the ratio a / b of the faulty
validators from above (exclusive)
*)
FaultyValidatorsFewerThan(pFaultyNodes, pVS, maxRatio) ==
LET FN == pFaultyNodes \intersect pVS \* faulty nodes in pNodes
CN == pVS \ pFaultyNodes \* correct nodes in pNodes
CP == Cardinality(CN) \* power of the correct nodes
FP == Cardinality(FN) \* power of the faulty nodes
IN
\* CP + FP = TP is the total voting power
LET TP == CP + FP IN
FP * maxRatio[2] < TP * maxRatio[1]
(* Can a block be produced by a correct peer, or an authenticated Byzantine peer *)
IsLightBlockAllowedByDigitalSignatures(ht, block) ==
\/ block.header = blockchain[ht] \* signed by correct and faulty (maybe)
\/ /\ block.Commits \subseteq Faulty
/\ block.header.height = ht
/\ block.header.time >= 0 \* signed only by faulty
(*
Initialize the blockchain to the ultimate height right in the initial states.
We pick the faulty validators statically, but that should not affect the light client.
Parameters:
- pMaxFaultyRatioExclusive is a pair <<a, b>> that bound the number of
faulty validators in each block by the ratio a / b (exclusive)
*)
InitToHeight(pMaxFaultyRatioExclusive) ==
/\ \E Nodes \in SUBSET AllNodes:
Faulty := Nodes \* pick a subset of nodes to be faulty
\* pick the validator sets and last commits
/\ \E vs, lastCommit \in [Heights -> SUBSET AllNodes]:
\E timestamp \in [Heights -> Int]:
\* refClock is at least as early as the timestamp in the last block
/\ \E tm \in Int:
refClock := tm /\ tm >= timestamp[ULTIMATE_HEIGHT]
\* the genesis starts on day 1
/\ timestamp[1] = 1
/\ vs[1] = AllNodes
/\ lastCommit[1] = EmptyNodeSet
/\ \A h \in Heights \ {1}:
/\ lastCommit[h] \subseteq vs[h - 1] \* the non-validators cannot commit
/\ TwoThirds(vs[h - 1], lastCommit[h]) \* the commit has >2/3 of validator votes
\* the faulty validators have the power below the threshold
/\ FaultyValidatorsFewerThan(Faulty, vs[h], pMaxFaultyRatioExclusive)
/\ timestamp[h] > timestamp[h - 1] \* the time grows monotonically
/\ timestamp[h] < timestamp[h - 1] + TRUSTING_PERIOD \* but not too fast
\* form the block chain out of validator sets and commits (this makes apalache faster)
/\ blockchain := [h \in Heights |->
[height |-> h,
time |-> timestamp[h],
VS |-> vs[h],
NextVS |-> IF h < ULTIMATE_HEIGHT THEN vs[h + 1] ELSE AllNodes,
lastCommit |-> lastCommit[h]]
] \******
(********************* BLOCKCHAIN ACTIONS ********************************)
(*
Advance the clock by zero or more time units.
*)
AdvanceTime ==
/\ \E tm \in Int: tm >= refClock /\ refClock' = tm
/\ UNCHANGED <<blockchain, Faulty>>
=============================================================================
\* Modification History
\* Last modified Wed Jun 10 14:10:54 CEST 2020 by igor
\* Created Fri Oct 11 15:45:11 CEST 2019 by igor

159
spec/light-client/attacks/Isolation_001_draft.tla Normal file
View File

@@ -0,0 +1,159 @@
----------------------- MODULE Isolation_001_draft ----------------------------
(**
* The specification of the attackers isolation at full node,
* when it has received an evidence from the light client.
* We check that the isolation spec produces a set of validators
* that have more than 1/3 of the voting power.
*
* It follows the English specification:
*
* https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/attacks/isolate-attackers_001_draft.md
*
* The assumptions made in this specification:
*
* - the voting power of every validator is 1
* (add more validators, if you need more validators)
*
* - Tendermint security model is violated
* (there are Byzantine validators who signed a conflicting block)
*
* Igor Konnov, Zarko Milosevic, Josef Widder, Informal Systems, 2020
*)
EXTENDS Integers, FiniteSets, Apalache
\* algorithm parameters
CONSTANTS
AllNodes,
(* a set of all nodes that can act as validators (correct and faulty) *)
COMMON_HEIGHT,
(* an index of the block header that two peers agree upon *)
CONFLICT_HEIGHT,
(* an index of the block header that two peers disagree upon *)
TRUSTING_PERIOD,
(* the period within which the validators are trusted *)
FAULTY_RATIO
(* a pair <<a, b>> that limits that ratio of faulty validator in the blockchain
from above (exclusive). Tendermint security model prescribes 1 / 3. *)
VARIABLES
blockchain, (* the chain at the full node *)
refClock, (* the reference clock at the full node *)
Faulty, (* the set of faulty validators *)
conflictingBlock, (* an evidence that two peers reported conflicting blocks *)
state, (* the state of the attack isolation machine at the full node *)
attackers (* the set of the identified attackers *)
vars == <<blockchain, refClock, Faulty, conflictingBlock, state>>
\* instantiate the chain at the full node
ULTIMATE_HEIGHT == CONFLICT_HEIGHT + 1
BC == INSTANCE Blockchain_003_draft
\* use the light client API
TRUSTING_HEIGHT == COMMON_HEIGHT
TARGET_HEIGHT == CONFLICT_HEIGHT
LC == INSTANCE LCVerificationApi_003_draft
WITH localClock <- refClock, REAL_CLOCK_DRIFT <- 0, CLOCK_DRIFT <- 0
\* old-style type annotations in apalache
a <: b == a
\* [LCAI-NONVALID-OUTPUT.1::TLA.1]
ViolatesValidity(header1, header2) ==
\/ header1.VS /= header2.VS
\/ header1.NextVS /= header2.NextVS
\/ header1.height /= header2.height
\/ header1.time /= header2.time
(* The English specification also checks the fields that we do not have
at this level of abstraction:
- header1.ConsensusHash != header2.ConsensusHash or
- header1.AppHash != header2.AppHash or
- header1.LastResultsHash header2 != ev.LastResultsHash
*)
Init ==
/\ state := "init"
\* Pick an arbitrary blockchain from 1 to COMMON_HEIGHT + 1.
/\ BC!InitToHeight(FAULTY_RATIO) \* initializes blockchain, Faulty, and refClock
/\ attackers := {} <: {STRING} \* attackers are unknown
\* Receive an arbitrary evidence.
\* Instantiate the light block fields one by one,
\* to avoid combinatorial explosion of records.
/\ \E time \in Int:
\E VS, NextVS, lastCommit, Commits \in SUBSET AllNodes:
LET conflicting ==
[ Commits |-> Commits,
header |->
[height |-> CONFLICT_HEIGHT,
time |-> time,
VS |-> VS,
NextVS |-> NextVS,
lastCommit |-> lastCommit] ]
IN
LET refBlock == [ header |-> blockchain[COMMON_HEIGHT],
Commits |-> blockchain[COMMON_HEIGHT + 1].lastCommit ]
IN
/\ "SUCCESS" = LC!ValidAndVerifiedUntimed(refBlock, conflicting)
\* More than third of next validators in the common reference block
\* is faulty. That is a precondition for a fork.
/\ 3 * Cardinality(Faulty \intersect refBlock.header.NextVS)
> Cardinality(refBlock.header.NextVS)
\* correct validators cannot sign an invalid block
/\ ViolatesValidity(conflicting.header, refBlock.header)
=> conflicting.Commits \subseteq Faulty
/\ conflictingBlock := conflicting
\* This is a specification of isolateMisbehavingProcesses.
\*
\* [LCAI-FUNC-MAIN.1::TLA.1]
Next ==
/\ state = "init"
\* Extract the rounds from the reference block and the conflicting block.
\* In this specification, we just pick rounds non-deterministically.
\* The English specification calls RoundOf on the blocks.
/\ \E referenceRound, evidenceRound \in Int:
/\ referenceRound >= 0 /\ evidenceRound >= 0
/\ LET reference == blockchain[CONFLICT_HEIGHT]
referenceCommit == blockchain[CONFLICT_HEIGHT + 1].lastCommit
evidenceHeader == conflictingBlock.header
evidenceCommit == conflictingBlock.Commits
IN
IF ViolatesValidity(reference, evidenceHeader)
THEN /\ attackers' := blockchain[COMMON_HEIGHT].NextVS \intersect evidenceCommit
/\ state' := "Lunatic"
ELSE IF referenceRound = evidenceRound
THEN /\ attackers' := referenceCommit \intersect evidenceCommit
/\ state' := "Equivocation"
ELSE
\* This property is shown in property
\* Accountability of TendermintAcc3.tla
/\ state' := "Amnesia"
/\ \E Attackers \in SUBSET (Faulty \intersect reference.VS):
/\ 3 * Cardinality(Attackers) > Cardinality(reference.VS)
/\ attackers' := Attackers
/\ blockchain' := blockchain
/\ refClock' := refClock
/\ Faulty' := Faulty
/\ conflictingBlock' := conflictingBlock
(********************************** INVARIANTS *******************************)
\* This invariant ensure that the attackers have
\* more than 1/3 of the voting power
\*
\* [LCAI-INV-Output.1::TLA-DETECTION-COMPLETENESS.1]
DetectionCompleteness ==
state /= "init" =>
3 * Cardinality(attackers) > Cardinality(blockchain[CONFLICT_HEIGHT].VS)
\* This invariant ensures that only the faulty validators are detected
\*
\* [LCAI-INV-Output.1::TLA-DETECTION-ACCURACY.1]
DetectionAccuracy ==
attackers \subseteq Faulty
==============================================================================

192
spec/light-client/attacks/LCVerificationApi_003_draft.tla Normal file
View File

@@ -0,0 +1,192 @@
-------------------- MODULE LCVerificationApi_003_draft --------------------------
(**
* The common interface of the light client verification and detection.
*)
EXTENDS Integers, FiniteSets
\* the parameters of Light Client
CONSTANTS
TRUSTING_PERIOD,
(* the period within which the validators are trusted *)
CLOCK_DRIFT,
(* the assumed precision of the clock *)
REAL_CLOCK_DRIFT,
(* the actual clock drift, which under normal circumstances should not
be larger than CLOCK_DRIFT (otherwise, there will be a bug) *)
FAULTY_RATIO
(* a pair <<a, b>> that limits that ratio of faulty validator in the blockchain
from above (exclusive). Tendermint security model prescribes 1 / 3. *)
VARIABLES
localClock (* current time as measured by the light client *)
(* the header is still within the trusting period *)
InTrustingPeriodLocal(header) ==
\* note that the assumption about the drift reduces the period of trust
localClock < header.time + TRUSTING_PERIOD - CLOCK_DRIFT
(* the header is still within the trusting period, even if the clock can go backwards *)
InTrustingPeriodLocalSurely(header) ==
\* note that the assumption about the drift reduces the period of trust
localClock < header.time + TRUSTING_PERIOD - 2 * CLOCK_DRIFT
(* ensure that the local clock does not drift far away from the global clock *)
IsLocalClockWithinDrift(local, global) ==
/\ global - REAL_CLOCK_DRIFT <= local
/\ local <= global + REAL_CLOCK_DRIFT
(**
* Check that the commits in an untrusted block form 1/3 of the next validators
* in a trusted header.
*)
SignedByOneThirdOfTrusted(trusted, untrusted) ==
LET TP == Cardinality(trusted.header.NextVS)
SP == Cardinality(untrusted.Commits \intersect trusted.header.NextVS)
IN
3 * SP > TP
(**
The first part of the precondition of ValidAndVerified, which does not take
the current time into account.
[LCV-FUNC-VALID.1::TLA-PRE-UNTIMED.1]
*)
ValidAndVerifiedPreUntimed(trusted, untrusted) ==
LET thdr == trusted.header
uhdr == untrusted.header
IN
/\ thdr.height < uhdr.height
\* the trusted block has been created earlier
/\ thdr.time < uhdr.time
/\ untrusted.Commits \subseteq uhdr.VS
/\ LET TP == Cardinality(uhdr.VS)
SP == Cardinality(untrusted.Commits)
IN
3 * SP > 2 * TP
/\ thdr.height + 1 = uhdr.height => thdr.NextVS = uhdr.VS
(* As we do not have explicit hashes we ignore these three checks of the English spec:
1. "trusted.Commit is a commit is for the header trusted.Header,
i.e. it contains the correct hash of the header".
2. untrusted.Validators = hash(untrusted.Header.Validators)
3. untrusted.NextValidators = hash(untrusted.Header.NextValidators)
*)
(**
Check the precondition of ValidAndVerified, including the time checks.
[LCV-FUNC-VALID.1::TLA-PRE.1]
*)
ValidAndVerifiedPre(trusted, untrusted, checkFuture) ==
LET thdr == trusted.header
uhdr == untrusted.header
IN
/\ InTrustingPeriodLocal(thdr)
\* The untrusted block is not from the future (modulo clock drift).
\* Do the check, if it is required.
/\ checkFuture => uhdr.time < localClock + CLOCK_DRIFT
/\ ValidAndVerifiedPreUntimed(trusted, untrusted)
(**
Check, whether an untrusted block is valid and verifiable w.r.t. a trusted header.
This test does take current time into account, but only looks at the block structure.
[LCV-FUNC-VALID.1::TLA-UNTIMED.1]
*)
ValidAndVerifiedUntimed(trusted, untrusted) ==
IF ~ValidAndVerifiedPreUntimed(trusted, untrusted)
THEN "INVALID"
ELSE IF untrusted.header.height = trusted.header.height + 1
\/ SignedByOneThirdOfTrusted(trusted, untrusted)
THEN "SUCCESS"
ELSE "NOT_ENOUGH_TRUST"
(**
Check, whether an untrusted block is valid and verifiable w.r.t. a trusted header.
[LCV-FUNC-VALID.1::TLA.1]
*)
ValidAndVerified(trusted, untrusted, checkFuture) ==
IF ~ValidAndVerifiedPre(trusted, untrusted, checkFuture)
THEN "INVALID"
ELSE IF ~InTrustingPeriodLocal(untrusted.header)
(* We leave the following test for the documentation purposes.
The implementation should do this test, as signature verification may be slow.
In the TLA+ specification, ValidAndVerified happens in no time.
*)
THEN "FAILED_TRUSTING_PERIOD"
ELSE IF untrusted.header.height = trusted.header.height + 1
\/ SignedByOneThirdOfTrusted(trusted, untrusted)
THEN "SUCCESS"
ELSE "NOT_ENOUGH_TRUST"
(**
The invariant of the light store that is not related to the blockchain
*)
LightStoreInv(fetchedLightBlocks, lightBlockStatus) ==
\A lh, rh \in DOMAIN fetchedLightBlocks:
\* for every pair of stored headers that have been verified
\/ lh >= rh
\/ lightBlockStatus[lh] /= "StateVerified"
\/ lightBlockStatus[rh] /= "StateVerified"
\* either there is a header between them
\/ \E mh \in DOMAIN fetchedLightBlocks:
lh < mh /\ mh < rh /\ lightBlockStatus[mh] = "StateVerified"
\* or the left header is outside the trusting period, so no guarantees
\/ LET lhdr == fetchedLightBlocks[lh]
rhdr == fetchedLightBlocks[rh]
IN
\* we can verify the right one using the left one
"SUCCESS" = ValidAndVerifiedUntimed(lhdr, rhdr)
(**
Correctness states that all the obtained headers are exactly like in the blockchain.
It is always the case that every verified header in LightStore was generated by
an instance of Tendermint consensus.
[LCV-DIST-SAFE.1::CORRECTNESS-INV.1]
*)
CorrectnessInv(blockchain, fetchedLightBlocks, lightBlockStatus) ==
\A h \in DOMAIN fetchedLightBlocks:
lightBlockStatus[h] = "StateVerified" =>
fetchedLightBlocks[h].header = blockchain[h]
(**
* When the light client terminates, there are no failed blocks.
* (Otherwise, someone lied to us.)
*)
NoFailedBlocksOnSuccessInv(fetchedLightBlocks, lightBlockStatus) ==
\A h \in DOMAIN fetchedLightBlocks:
lightBlockStatus[h] /= "StateFailed"
(**
The expected post-condition of VerifyToTarget.
*)
VerifyToTargetPost(blockchain, isPeerCorrect,
fetchedLightBlocks, lightBlockStatus,
trustedHeight, targetHeight, finalState) ==
LET trustedHeader == fetchedLightBlocks[trustedHeight].header IN
\* The light client is not lying us on the trusted block.
\* It is straightforward to detect.
/\ lightBlockStatus[trustedHeight] = "StateVerified"
/\ trustedHeight \in DOMAIN fetchedLightBlocks
/\ trustedHeader = blockchain[trustedHeight]
\* the invariants we have found in the light client verification
\* there is a problem with trusting period
/\ isPeerCorrect
=> CorrectnessInv(blockchain, fetchedLightBlocks, lightBlockStatus)
\* a correct peer should fail the light client,
\* if the trusted block is in the trusting period
/\ isPeerCorrect /\ InTrustingPeriodLocalSurely(trustedHeader)
=> finalState = "finishedSuccess"
/\ finalState = "finishedSuccess" =>
/\ lightBlockStatus[targetHeight] = "StateVerified"
/\ targetHeight \in DOMAIN fetchedLightBlocks
/\ NoFailedBlocksOnSuccessInv(fetchedLightBlocks, lightBlockStatus)
/\ LightStoreInv(fetchedLightBlocks, lightBlockStatus)
==================================================================================

18
spec/light-client/attacks/MC_5_3.tla Normal file
View File

@@ -0,0 +1,18 @@
------------------------- MODULE MC_5_3 -------------------------------------
AllNodes == {"n1", "n2", "n3", "n4", "n5"}
COMMON_HEIGHT == 1
CONFLICT_HEIGHT == 3
TRUSTING_PERIOD == 1400 \* two weeks, one day is 100 time units :-)
FAULTY_RATIO == <<1, 2>> \* < 1 / 2 faulty validators
VARIABLES
blockchain, \* the reference blockchain
refClock, \* current time in the reference blockchain
Faulty, \* the set of faulty validators
state, \* the state of the light client detector
conflictingBlock, \* an evidence that two peers reported conflicting blocks
attackers
INSTANCE Isolation_001_draft
============================================================================

221
spec/light-client/attacks/isolate-attackers_001_draft.md Normal file
View File

@@ -0,0 +1,221 @@
# Lightclient Attackers Isolation
> Warning: This is the beginning of an unfinished draft. Don't continue reading!
Adversarial nodes may have the incentive to lie to a lightclient about the state of a Tendermint blockchain. An attempt to do so is called attack. Light client [verification][verification] checks incoming data by checking a so-called "commit", which is a forwarded set of signed messages that is (supposedly) produced during executing Tendermint consensus. Thus, an attack boils down to creating and signing Tendermint consensus messages in deviation from the Tendermint consensus algorithm rules.
As Tendermint consensus and light client verification is safe under the assumption of more than 2/3 of correct voting power per block [[TMBC-FM-2THIRDS]][TMBC-FM-2THIRDS-link], this implies that if there was an attack then [[TMBC-FM-2THIRDS]][TMBC-FM-2THIRDS-link] was violated, that is, there is a block such that
- validators deviated from the protocol, and
- these validators represent more than 1/3 of the voting power in that block.
In the case of an [attack][node-based-attack-characterization], the lightclient [attack detection mechanism][detection] computes data, so called evidence [[LC-DATA-EVIDENCE.1]][LC-DATA-EVIDENCE-link], that can be used
- to proof that there has been attack [[TMBC-LC-EVIDENCE-DATA.1]][TMBC-LC-EVIDENCE-DATA-link] and
- as basis to find the actual nodes that deviated from the Tendermint protocol.
This specification considers how a full node in a Tendermint blockchain can isolate a set of attackers that launched the attack. The set should satisfy
- the set does not contain a correct validator
- the set contains validators that represent more than 1/3 of the voting power of a block that is still within the unbonding period
# Outline
**TODO** when preparing a version for broader review.
# Part I - Basics
For definitions of data structures used here, in particular LightBlocks [[LCV-DATA-LIGHTBLOCK.1]](https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md#lcv-data-lightblock1), cf. [Light Client Verification][verification].
# Part II - Definition of the Problem
The specification of the [detection mechanism][detection] describes
- what is a light client attack,
- conditions under which the detector will detect a light client attack,
- and the format of the output data, called evidence, in the case an attack is detected. The format is defined in
[[LC-DATA-EVIDENCE.1]][LC-DATA-EVIDENCE-link] and looks as follows
```go
type LightClientAttackEvidence struct {
ConflictingBlock LightBlock
CommonHeight int64
}
```
The isolator is a function that gets as input evidence `ev`
and a prefix of the blockchain `bc` at least up to height `ev.ConflictingBlock.Header.Height + 1`. The output is a set of *peerIDs* of validators.
We assume that the full node is synchronized with the blockchain and has reached the height `ev.ConflictingBlock.Header.Height + 1`.
#### **[FN-INV-Output.1]**
When an output is generated it satisfies the following properties:
- If
- `bc[CommonHeight].bfttime` is within the unbonding period w.r.t. the time at the full node,
- `ev.ConflictingBlock.Header != bc[ev.ConflictingBlock.Header.Height]`
- Validators in `ev.ConflictingBlock.Commit` represent more than 1/3 of the voting power in `bc[ev.CommonHeight].NextValidators`
- Then: A set of validators in `bc[CommonHeight].NextValidators` that
- represent more than 1/3 of the voting power in `bc[ev.commonHeight].NextValidators`
- signed Tendermint consensus messages for height `ev.ConflictingBlock.Header.Height` by violating the Tendermint consensus protocol.
- Else: the empty set.
# Part IV - Protocol
Here we discuss how to solve the problem of isolating misbehaving processes. We describe the function `isolateMisbehavingProcesses` as well as all the helping functions below. In [Part V](#part-v---Completeness), we discuss why the solution is complete based on result from analysis with automated tools.
## Isolation
### Outline
> Describe solution (in English), decomposition into functions, where communication to other components happens.
#### **[LCAI-FUNC-MAIN.1]**
```go
func isolateMisbehavingProcesses(ev LightClientAttackEvidence, bc Blockchain) []ValidatorAddress {
reference := bc[ev.conflictingBlock.Header.Height].Header
ev_header := ev.conflictingBlock.Header
ref_commit := bc[ev.conflictingBlock.Header.Height + 1].Header.LastCommit // + 1 !!
ev_commit := ev.conflictingBlock.Commit
if violatesTMValidity(reference, ev_header) {
// lunatic light client attack
signatories := Signers(ev.ConflictingBlock.Commit)
bonded_vals := Addresses(bc[ev.CommonHeight].NextValidators)
return intersection(signatories,bonded_vals)
}
// If this point is reached the validator sets in reference and ev_header are identical
else if RoundOf(ref_commit) == RoundOf(ev_commit) {
// equivocation light client attack
return intersection(Signers(ref_commit), Signers(ev_commit))
}
else {
// amnesia light client attack
return IsolateAmnesiaAttacker(ev, bc)
}
}
```
- Implementation comment
- If the full node has only reached height `ev.conflictingBlock.Header.Height` then `bc[ev.conflictingBlock.Header.Height + 1].Header.LastCommit` refers to the locally stored commit for this height. (This commit must be present by the precondition on `length(bc)`.)
- We check in the precondition that the unbonding period is not expired. However, since time moves on, before handing the validators over Cosmos SDK, the time needs to be checked again to satisfy the contract which requires that only bonded validators are reported. This passing of validators to the SDK is out of scope of this specification.
- Expected precondition
- `length(bc) >= ev.conflictingBlock.Header.Height`
- `ValidAndVerifiedUnbonding(bc[ev.CommonHeight], ev.ConflictingBlock) == SUCCESS`
- `ev.ConflictingBlock.Header != bc[ev.ConflictingBlock.Header.Height]`
- TODO: input light blocks pass basic validation
- Expected postcondition
- [[FN-INV-Output.1]](#FN-INV-Output1) holds
- Error condition
- returns an error if precondition is violated.
### Details of the Functions
#### **[LCAI-FUNC-VVU.1]**
```go
func ValidAndVerifiedUnbonding(trusted LightBlock, untrusted LightBlock) Result
```
- Conditions are identical to [[LCV-FUNC-VALID.2]][LCV-FUNC-VALID.link] except the precondition "*trusted.Header.Time > now - trustingPeriod*" is substituted with
- `trusted.Header.Time > now - UnbondingPeriod`
#### **[LCAI-FUNC-NONVALID.1]**
```go
func violatesTMValidity(ref Header, ev Header) boolean
```
- Implementation remarks
- checks whether the evidence header `ev` violates the validity property of Tendermint Consensus, by checking agains a reference header
- Expected precondition
- `ref.Height == ev.Height`
- Expected postcondition
- returns evaluation of the following disjunction
**[[LCAI-NONVALID-OUTPUT.1]]** ==
`ref.ValidatorsHash != ev.ValidatorsHash` or
`ref.NextValidatorsHash != ev.NextValidatorsHash` or
`ref.ConsensusHash != ev.ConsensusHash` or
`ref.AppHash != ev.AppHash` or
`ref.LastResultsHash != ev.LastResultsHash`
```go
func IsolateAmnesiaAttacker(ev LightClientAttackEvidence, bc Blockchain) []ValidatorAddress
```
- Implementation remarks
**TODO:** What should we do here? Refer to the accountability doc?
- Expected postcondition
**TODO:** What should we do here? Refer to the accountability doc?
```go
func RoundOf(commit Commit) []ValidatorAddress
```
- Expected precondition
- `commit` is well-formed. In particular all votes are from the same round `r`.
- Expected postcondition
- returns round `r` that is encoded in all the votes of the commit
```go
func Signers(commit Commit) []ValidatorAddress
```
- Expected postcondition
- returns all validator addresses in `commit`
```go
func Addresses(vals Validator[]) ValidatorAddress[]
```
- Expected postcondition
- returns all validator addresses in `vals`
# Part V - Completeness
As discussed in the beginning of this document, an attack boils down to creating and signing Tendermint consensus messages in deviation from the Tendermint consensus algorithm rules.
The main function `isolateMisbehavingProcesses` distinguishes three kinds of wrongly signing messages, namely,
- lunatic: signing invalid blocks
- equivocation: double-signing valid blocks in the same consensus round
- amnesia: signing conflicting blocks in different consensus rounds, without having seen a quorum of messages that would have allowed to do so.
The question is whether this captures all attacks.
First observe that the first checking in `isolateMisbehavingProcesses` is `violatesTMValidity`. It takes care of lunatic attacks. If this check passes, that is, if `violatesTMValidity` returns `FALSE` this means that [FN-NONVALID-OUTPUT] evaluates to false, which implies that `ref.ValidatorsHash = ev.ValidatorsHash`. Hence after `violatesTMValidity`, all the involved validators are the ones from the blockchain. It is thus sufficient to analyze one instance of Tendermint consensus with a fixed group membership (set of validators). Also it is sufficient to consider two different valid consensus values, that is, binary consensus.
**TODO** we have analyzed Tendermint consensus with TLA+ and have accompanied Galois in an independent study of the protocol based on [Ivy proofs](https://github.com/tendermint/spec/tree/master/ivy-proofs).
# References
[[supervisor]] The specification of the light client supervisor.
[[verification]] The specification of the light client verification protocol
[[detection]] The specification of the light client attack detection mechanism.
[supervisor]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/supervisor/supervisor_001_draft.md
[verification]: https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md
[detection]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md
[LC-DATA-EVIDENCE-link]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md#lc-data-evidence1
[TMBC-LC-EVIDENCE-DATA-link]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md#tmbc-lc-evidence-data1
[node-based-attack-characterization]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md#node-based-characterization-of-attacks
[TMBC-FM-2THIRDS-link]: https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md#tmbc-fm-2thirds1
[LCV-FUNC-VALID.link]: https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md#lcv-func-valid2

223
spec/light-client/attacks/isolate-attackers_002_reviewed.md Normal file
View File

@@ -0,0 +1,223 @@
# Lightclient Attackers Isolation
Adversarial nodes may have the incentive to lie to a lightclient about the state of a Tendermint blockchain. An attempt to do so is called attack. Light client [verification][verification] checks incoming data by checking a so-called "commit", which is a forwarded set of signed messages that is (supposedly) produced during executing Tendermint consensus. Thus, an attack boils down to creating and signing Tendermint consensus messages in deviation from the Tendermint consensus algorithm rules.
As Tendermint consensus and light client verification is safe under the assumption of more than 2/3 of correct voting power per block [[TMBC-FM-2THIRDS]][TMBC-FM-2THIRDS-link], this implies that if there was an attack then [[TMBC-FM-2THIRDS]][TMBC-FM-2THIRDS-link] was violated, that is, there is a block such that
- validators deviated from the protocol, and
- these validators represent more than 1/3 of the voting power in that block.
In the case of an [attack][node-based-attack-characterization], the lightclient [attack detection mechanism][detection] computes data, so called evidence [[LC-DATA-EVIDENCE.1]][LC-DATA-EVIDENCE-link], that can be used
- to proof that there has been attack [[TMBC-LC-EVIDENCE-DATA.1]][TMBC-LC-EVIDENCE-DATA-link] and
- as basis to find the actual nodes that deviated from the Tendermint protocol.
This specification considers how a full node in a Tendermint blockchain can isolate a set of attackers that launched the attack. The set should satisfy
- the set does not contain a correct validator
- the set contains validators that represent more than 1/3 of the voting power of a block that is still within the unbonding period
# Outline
After providing the [problem statement](#Part-I---Basics-and-Definition-of-the-Problem), we specify the [isolator function](#Part-II---Protocol) and close with the discussion about its [correctness](#Part-III---Completeness) which is based on computer-aided analysis of Tendermint Consensus.
# Part I - Basics and Definition of the Problem
For definitions of data structures used here, in particular LightBlocks [[LCV-DATA-LIGHTBLOCK.1]](https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md#lcv-data-lightblock1), we refer to the specification of [Light Client Verification][verification].
The specification of the [detection mechanism][detection] describes
- what is a light client attack,
- conditions under which the detector will detect a light client attack,
- and the format of the output data, called evidence, in the case an attack is detected. The format is defined in
[[LC-DATA-EVIDENCE.1]][LC-DATA-EVIDENCE-link] and looks as follows
```go
type LightClientAttackEvidence struct {
ConflictingBlock LightBlock
CommonHeight int64
}
```
The isolator is a function that gets as input evidence `ev`
and a prefix of the blockchain `bc` at least up to height `ev.ConflictingBlock.Header.Height + 1`. The output is a set of *peerIDs* of validators.
We assume that the full node is synchronized with the blockchain and has reached the height `ev.ConflictingBlock.Header.Height + 1`.
#### **[LCAI-INV-Output.1]**
When an output is generated it satisfies the following properties:
- If
- `bc[CommonHeight].bfttime` is within the unbonding period w.r.t. the time at the full node,
- `ev.ConflictingBlock.Header != bc[ev.ConflictingBlock.Header.Height]`
- Validators in `ev.ConflictingBlock.Commit` represent more than 1/3 of the voting power in `bc[ev.CommonHeight].NextValidators`
- Then: The output is a set of validators in `bc[CommonHeight].NextValidators` that
- represent more than 1/3 of the voting power in `bc[ev.commonHeight].NextValidators`
- signed Tendermint consensus messages for height `ev.ConflictingBlock.Header.Height` by violating the Tendermint consensus protocol.
- Else: the empty set.
# Part II - Protocol
Here we discuss how to solve the problem of isolating misbehaving processes. We describe the function `isolateMisbehavingProcesses` as well as all the helping functions below. In [Part III](#part-III---Completeness), we discuss why the solution is complete based on result from analysis with automated tools.
## Isolation
### Outline
We first check whether the conflicting block can indeed be verified from the common height. We then first check whether it was a lunatic attack (violating validity). If this is not the case, we check for equivocation. If this also is not the case, we start the on-chain [accountability protocol](https://docs.google.com/document/d/11ZhMsCj3y7zIZz4udO9l25xqb0kl7gmWqNpGVRzOeyY/edit).
#### **[LCAI-FUNC-MAIN.1]**
```go
func isolateMisbehavingProcesses(ev LightClientAttackEvidence, bc Blockchain) []ValidatorAddress {
reference := bc[ev.conflictingBlock.Header.Height].Header
ev_header := ev.conflictingBlock.Header
ref_commit := bc[ev.conflictingBlock.Header.Height + 1].Header.LastCommit // + 1 !!
ev_commit := ev.conflictingBlock.Commit
if violatesTMValidity(reference, ev_header) {
// lunatic light client attack
signatories := Signers(ev.ConflictingBlock.Commit)
bonded_vals := Addresses(bc[ev.CommonHeight].NextValidators)
return intersection(signatories,bonded_vals)
}
// If this point is reached the validator sets in reference and ev_header are identical
else if RoundOf(ref_commit) == RoundOf(ev_commit) {
// equivocation light client attack
return intersection(Signers(ref_commit), Signers(ev_commit))
}
else {
// amnesia light client attack
return IsolateAmnesiaAttacker(ev, bc)
}
}
```
- Implementation comment
- If the full node has only reached height `ev.conflictingBlock.Header.Height` then `bc[ev.conflictingBlock.Header.Height + 1].Header.LastCommit` refers to the locally stored commit for this height. (This commit must be present by the precondition on `length(bc)`.)
- We check in the precondition that the unbonding period is not expired. However, since time moves on, before handing the validators over Cosmos SDK, the time needs to be checked again to satisfy the contract which requires that only bonded validators are reported. This passing of validators to the SDK is out of scope of this specification.
- Expected precondition
- `length(bc) >= ev.conflictingBlock.Header.Height`
- `ValidAndVerifiedUnbonding(bc[ev.CommonHeight], ev.ConflictingBlock) == SUCCESS`
- `ev.ConflictingBlock.Header != bc[ev.ConflictingBlock.Header.Height]`
- `ev.conflictingBlock` satisfies basic validation (in particular all signed messages in the Commit are from the same round)
- Expected postcondition
- [[FN-INV-Output.1]](#FN-INV-Output1) holds
- Error condition
- returns an error if precondition is violated.
### Details of the Functions
#### **[LCAI-FUNC-VVU.1]**
```go
func ValidAndVerifiedUnbonding(trusted LightBlock, untrusted LightBlock) Result
```
- Conditions are identical to [[LCV-FUNC-VALID.2]][LCV-FUNC-VALID.link] except the precondition "*trusted.Header.Time > now - trustingPeriod*" is substituted with
- `trusted.Header.Time > now - UnbondingPeriod`
#### **[LCAI-FUNC-NONVALID.1]**
```go
func violatesTMValidity(ref Header, ev Header) boolean
```
- Implementation remarks
- checks whether the evidence header `ev` violates the validity property of Tendermint Consensus, by checking against a reference header
- Expected precondition
- `ref.Height == ev.Height`
- Expected postcondition
- returns evaluation of the following disjunction
**[LCAI-NONVALID-OUTPUT.1]** ==
`ref.ValidatorsHash != ev.ValidatorsHash` or
`ref.NextValidatorsHash != ev.NextValidatorsHash` or
`ref.ConsensusHash != ev.ConsensusHash` or
`ref.AppHash != ev.AppHash` or
`ref.LastResultsHash != ev.LastResultsHash`
```go
func IsolateAmnesiaAttacker(ev LightClientAttackEvidence, bc Blockchain) []ValidatorAddress
```
- Implementation remarks
- This triggers the [query/response protocol](https://docs.google.com/document/d/11ZhMsCj3y7zIZz4udO9l25xqb0kl7gmWqNpGVRzOeyY/edit).
- Expected postcondition
- returns attackers according to [LCAI-INV-Output.1].
```go
func RoundOf(commit Commit) []ValidatorAddress
```
- Expected precondition
- `commit` is well-formed. In particular all votes are from the same round `r`.
- Expected postcondition
- returns round `r` that is encoded in all the votes of the commit
- Error condition
- reports error if precondition is violated
```go
func Signers(commit Commit) []ValidatorAddress
```
- Expected postcondition
- returns all validator addresses in `commit`
```go
func Addresses(vals Validator[]) ValidatorAddress[]
```
- Expected postcondition
- returns all validator addresses in `vals`
# Part III - Completeness
As discussed in the beginning of this document, an attack boils down to creating and signing Tendermint consensus messages in deviation from the Tendermint consensus algorithm rules.
The main function `isolateMisbehavingProcesses` distinguishes three kinds of wrongly signed messages, namely,
- lunatic: signing invalid blocks
- equivocation: double-signing valid blocks in the same consensus round
- amnesia: signing conflicting blocks in different consensus rounds, without having seen a quorum of messages that would have allowed to do so.
The question is whether this captures all attacks.
First observe that the first check in `isolateMisbehavingProcesses` is `violatesTMValidity`. It takes care of lunatic attacks. If this check passes, that is, if `violatesTMValidity` returns `FALSE` this means that [[LCAI-NONVALID-OUTPUT.1]](#LCAI-FUNC-NONVALID1]) evaluates to false, which implies that `ref.ValidatorsHash = ev.ValidatorsHash`. Hence, after `violatesTMValidity`, all the involved validators are the ones from the blockchain. It is thus sufficient to analyze one instance of Tendermint consensus with a fixed group membership (set of validators). Also, as we have two different blocks for the same height, it is sufficient to consider two different valid consensus values, that is, binary consensus.
For this fixed group membership, we have analyzed the attacks using the TLA+ specification of [Tendermint Consensus in TLA+][tendermint-accountability]. We checked that indeed the only possible scenarios that can lead to violation of agreement are **equivocation** and **amnesia**. An independent study by Galois of the protocol based on [Ivy proofs](https://github.com/tendermint/spec/tree/master/ivy-proofs) led to the same conclusion.
# References
[[supervisor]] The specification of the light client supervisor.
[[verification]] The specification of the light client verification protocol.
[[detection]] The specification of the light client attack detection mechanism.
[[tendermint-accountability]]: TLA+ specification to check the types of attacks
[tendermint-accountability]:
https://github.com/tendermint/spec/blob/master/rust-spec/tendermint-accountability/README.md
[supervisor]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/supervisor/supervisor_001_draft.md
[verification]: https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md
[detection]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md
[LC-DATA-EVIDENCE-link]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md#lc-data-evidence1
[TMBC-LC-EVIDENCE-DATA-link]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md#tmbc-lc-evidence-data1
[node-based-attack-characterization]:
https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/detection/detection_003_reviewed.md#node-based-characterization-of-attacks
[TMBC-FM-2THIRDS-link]: https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md#tmbc-fm-2thirds1
[LCV-FUNC-VALID.link]: https://github.com/tendermint/spec/blob/master/rust-spec/lightclient/verification/verification_002_draft.md#lcv-func-valid2

219
spec/light-client/attacks/notes-on-evidence-handling.md Normal file
View File

@@ -0,0 +1,219 @@
# Light client attacks
We define a light client attack as detection of conflicting headers for a given height that can be verified
starting from the trusted light block. A light client attack is defined in the context of interactions of
light client with two peers. One of the peers (called primary) defines a trace of verified light blocks
(primary trace) that are being checked against trace of the other peer (called witness) that we call
witness trace.
A light client attack is defined by the primary and witness traces
that have a common root (the same trusted light block for a common height) but forms
conflicting branches (end of traces is for the same height but with different headers).
Note that conflicting branches could be arbitrarily big as branches continue to diverge after
a bifurcation point. We propose an approach that allows us to define a valid light client attack
only with a common light block and a single conflicting light block. We rely on the fact that
we assume that the primary is under suspicion (therefore not trusted) and that the witness plays
support role to detect and process an attack (therefore trusted). Therefore, once a light client
detects an attack, it needs to send to a witness only missing data (common height
and conflicting light block) as it has its trace. Keeping light client attack data of constant size
saves bandwidth and reduces an attack surface. As we will explain below, although in the context of
light client core
[verification](https://github.com/informalsystems/tendermint-rs/tree/master/docs/spec/lightclient/verification)
the roles of primary and witness are clearly defined,
in case of the attack, we run the same attack detection procedure twice where the roles are swapped.
The rationale is that the light client does not know what peer is correct (on a right main branch)
so it tries to create and submit an attack evidence to both peers.
Light client attack evidence consists of a conflicting light block and a common height.
```go
type LightClientAttackEvidence struct {
ConflictingBlock LightBlock
CommonHeight int64
}
```
Full node can validate a light client attack evidence by executing the following procedure:
```go
func IsValid(lcaEvidence LightClientAttackEvidence, bc Blockchain) boolean {
commonBlock = GetLightBlock(bc, lcaEvidence.CommonHeight)
if commonBlock == nil return false
// Note that trustingPeriod in ValidAndVerified is set to UNBONDING_PERIOD
verdict = ValidAndVerified(commonBlock, lcaEvidence.ConflictingBlock)
conflictingHeight = lcaEvidence.ConflictingBlock.Header.Height
return verdict == OK and bc[conflictingHeight].Header != lcaEvidence.ConflictingBlock.Header
}
```
## Light client attack creation
Given a trusted light block `trusted`, a light node executes the bisection algorithm to verify header
`untrusted` at some height `h`. If the bisection algorithm succeeds, then the header `untrusted` is verified.
Headers that are downloaded as part of the bisection algorithm are stored in a store and they are also in
the verified state. Therefore, after the bisection algorithm successfully terminates we have a trace of
the light blocks ([] LightBlock) we obtained from the primary that we call primary trace.
### Primary trace
The following invariant holds for the primary trace:
- Given a `trusted` light block, target height `h`, and `primary_trace` ([] LightBlock):
*primary_trace[0] == trusted* and *primary_trace[len(primary_trace)-1].Height == h* and
successive light blocks are passing light client verification logic.
### Witness with a conflicting header
The verified header at height `h` is cross-checked with every witness as part of
[detection](https://github.com/informalsystems/tendermint-rs/tree/master/docs/spec/lightclient/detection).
If a witness returns the conflicting header at the height `h` the following procedure is executed to verify
if the conflicting header comes from the valid trace and if that's the case to create an attack evidence:
#### Helper functions
We assume the following helper functions:
```go
// Returns trace of verified light blocks starting from rootHeight and ending with targetHeight.
Trace(lightStore LightStore, rootHeight int64, targetHeight int64) LightBlock[]
// Returns validator set for the given height
GetValidators(bc Blockchain, height int64) Validator[]
// Returns validator set for the given height
GetValidators(bc Blockchain, height int64) Validator[]
// Return validator addresses for the given validators
GetAddresses(vals Validator[]) ValidatorAddress[]
```
```go
func DetectLightClientAttacks(primary PeerID,
primary_trace []LightBlock,
witness PeerID) (LightClientAttackEvidence, LightClientAttackEvidence) {
primary_lca_evidence, witness_trace = DetectLightClientAttack(primary_trace, witness)
witness_lca_evidence = nil
if witness_trace != nil {
witness_lca_evidence, _ = DetectLightClientAttack(witness_trace, primary)
}
return primary_lca_evidence, witness_lca_evidence
}
func DetectLightClientAttack(trace []LightBlock, peer PeerID) (LightClientAttackEvidence, []LightBlock) {
lightStore = new LightStore().Update(trace[0], StateTrusted)
for i in 1..len(trace)-1 {
lightStore, result = VerifyToTarget(peer, lightStore, trace[i].Header.Height)
if result == ResultFailure then return (nil, nil)
current = lightStore.Get(trace[i].Header.Height)
// if obtained header is the same as in the trace we continue with a next height
if current.Header == trace[i].Header continue
// we have identified a conflicting header
commonBlock = trace[i-1]
conflictingBlock = trace[i]
return (LightClientAttackEvidence { conflictingBlock, commonBlock.Header.Height },
Trace(lightStore, trace[i-1].Header.Height, trace[i].Header.Height))
}
return (nil, nil)
}
```
## Evidence handling
As part of on chain evidence handling, full nodes identifies misbehaving processes and informs
the application, so they can be slashed. Note that only bonded validators should
be reported to the application. There are three types of attacks that can be executed against
Tendermint light client:
- lunatic attack
- equivocation attack and
- amnesia attack.
We now specify the evidence handling logic.
```go
func detectMisbehavingProcesses(lcAttackEvidence LightClientAttackEvidence, bc Blockchain) []ValidatorAddress {
assume IsValid(lcaEvidence, bc)
// lunatic light client attack
if !isValidBlock(current.Header, conflictingBlock.Header) {
conflictingCommit = lcAttackEvidence.ConflictingBlock.Commit
bondedValidators = GetNextValidators(bc, lcAttackEvidence.CommonHeight)
return getSigners(conflictingCommit) intersection GetAddresses(bondedValidators)
// equivocation light client attack
} else if current.Header.Round == conflictingBlock.Header.Round {
conflictingCommit = lcAttackEvidence.ConflictingBlock.Commit
trustedCommit = bc[conflictingBlock.Header.Height+1].LastCommit
return getSigners(trustedCommit) intersection getSigners(conflictingCommit)
// amnesia light client attack
} else {
HandleAmnesiaAttackEvidence(lcAttackEvidence, bc)
}
}
// Block validity in this context is defined by the trusted header.
func isValidBlock(trusted Header, conflicting Header) boolean {
return trusted.ValidatorsHash == conflicting.ValidatorsHash and
trusted.NextValidatorsHash == conflicting.NextValidatorsHash and
trusted.ConsensusHash == conflicting.ConsensusHash and
trusted.AppHash == conflicting.AppHash and
trusted.LastResultsHash == conflicting.LastResultsHash
}
func getSigners(commit Commit) []ValidatorAddress {
signers = []ValidatorAddress
for (i, commitSig) in commit.Signatures {
if commitSig.BlockIDFlag == BlockIDFlagCommit {
signers.append(commitSig.ValidatorAddress)
}
}
return signers
}
```
Note that amnesia attack evidence handling involves more complex processing, i.e., cannot be
defined simply on amnesia attack evidence. We explain in the following section a protocol
for handling amnesia attack evidence.
### Amnesia attack evidence handling
Detecting faulty processes in case of the amnesia attack is more complex and cannot be inferred
purely based on attack evidence data. In this case, in order to detect misbehaving processes we need
access to votes processes sent/received during the conflicting height. Therefore, amnesia handling assumes that
validators persist all votes received and sent during multi-round heights (as amnesia attack
is only possible in heights that executes over multiple rounds, i.e., commit round > 0).
To simplify description of the algorithm we assume existence of the trusted oracle called monitor that will
drive the algorithm and output faulty processes at the end. Monitor can be implemented in a
distributed setting as on-chain module. The algorithm works as follows:
1) Monitor sends votesets request to validators of the conflicting height. Validators
are expected to send their votesets within predefined timeout.
2) Upon receiving votesets request, validators send their votesets to a monitor.
2) Validators which have not sent its votesets within timeout are considered faulty.
3) The preprocessing of the votesets is done. That means that the received votesets are analyzed
and each vote (valid) sent by process p is added to the voteset of the sender p. This phase ensures that
votes sent by faulty processes observed by at least one correct validator cannot be excluded from the analysis.
4) Votesets of every validator are analyzed independently to decide whether the validator is correct or faulty.
A faulty validators is the one where at least one of those invalid transitions is found:
- More than one PREVOTE message is sent in a round
- More than one PRECOMMIT message is sent in a round
- PRECOMMIT message is sent without receiving +2/3 of voting-power equivalent
appropriate PREVOTE messages
- PREVOTE message is sent for the value V’ in round r’ and the PRECOMMIT message had
been sent for the value V in round r by the same process (r’ > r) and there are no
+2/3 of voting-power equivalent PREVOTE(vr, V’) messages (vr ≥ 0 and vr > r and vr < r’)
as the justification for sending PREVOTE(r’, V’)

10
spec/light-client/detection/004bmc-apalache-ok.csv Normal file
View File

@@ -0,0 +1,10 @@
no;filename;tool;timeout;init;inv;next;args
1;LCD_MC3_3_faulty.tla;apalache;1h;;CommonHeightOnEvidenceInv;;--length=10
2;LCD_MC3_3_faulty.tla;apalache;1h;;AccuracyInv;;--length=10
3;LCD_MC3_3_faulty.tla;apalache;1h;;PrecisionInvLocal;;--length=10
4;LCD_MC3_4_faulty.tla;apalache;1h;;CommonHeightOnEvidenceInv;;--length=10
5;LCD_MC3_4_faulty.tla;apalache;1h;;AccuracyInv;;--length=10
6;LCD_MC3_4_faulty.tla;apalache;1h;;PrecisionInvLocal;;--length=10
7;LCD_MC4_4_faulty.tla;apalache;1h;;CommonHeightOnEvidenceInv;;--length=10
8;LCD_MC4_4_faulty.tla;apalache;1h;;AccuracyInv;;--length=10
9;LCD_MC4_4_faulty.tla;apalache;1h;;PrecisionInvLocal;;--length=10
1 no filename tool timeout init inv next args
2 1 LCD_MC3_3_faulty.tla apalache 1h CommonHeightOnEvidenceInv --length=10
3 2 LCD_MC3_3_faulty.tla apalache 1h AccuracyInv --length=10
4 3 LCD_MC3_3_faulty.tla apalache 1h PrecisionInvLocal --length=10
5 4 LCD_MC3_4_faulty.tla apalache 1h CommonHeightOnEvidenceInv --length=10
6 5 LCD_MC3_4_faulty.tla apalache 1h AccuracyInv --length=10
7 6 LCD_MC3_4_faulty.tla apalache 1h PrecisionInvLocal --length=10
8 7 LCD_MC4_4_faulty.tla apalache 1h CommonHeightOnEvidenceInv --length=10
9 8 LCD_MC4_4_faulty.tla apalache 1h AccuracyInv --length=10
10 9 LCD_MC4_4_faulty.tla apalache 1h PrecisionInvLocal --length=10

Some files were not shown because too many files have changed in this diff Show More