mirrors/tendermint
1
0
Fork 0
mirror of https://github.com/tendermint/tendermint.git synced 2026-09-04 07:07:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Deprecate/refactor content in docs/specification (#1934)

Browse Source 
* docs: deprecate specification dir, closes #1814

* update genesis

* old spec dir, deprecation complete

* rm a file
This commit is contained in:
Zach
2018-07-11 23:45:10 +04:00
committed by Anton Kaliaev
parent 4f7fac8076
commit bbf2bd1d81
17 changed files with 771 additions and 1064 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tendermint-core/block-structure.md
+206
View File
@@ -0,0 +1,206 @@
# Block Structure
The tendermint consensus engine records all agreements by a
supermajority of nodes into a blockchain, which is replicated among all
nodes. This blockchain is accessible via various rpc endpoints, mainly
`/block?height=` to get the full block, as well as
`/blockchain?minHeight=_&maxHeight=_` to get a list of headers. But what
exactly is stored in these blocks?
## Block
A
[Block](https://godoc.org/github.com/tendermint/tendermint/types#Block)
contains:
- a [Header](#header) contains merkle hashes for various chain states
- the
[Data](https://godoc.org/github.com/tendermint/tendermint/types#Data)
is all transactions which are to be processed
- the [LastCommit](#commit) > 2/3 signatures for the last block
The signatures returned along with block `H` are those validating block
`H-1`. This can be a little confusing, but we must also consider that
the `Header` also contains the `LastCommitHash`. It would be impossible
for a Header to include the commits that sign it, as it would cause an
infinite loop here. But when we get block `H`, we find
`Header.LastCommitHash`, which must match the hash of `LastCommit`.
## Header
The
[Header](https://godoc.org/github.com/tendermint/tendermint/types#Header)
contains lots of information (follow link for up-to-date info). Notably,
it maintains the `Height`, the `LastBlockID` (to make it a chain), and
hashes of the data, the app state, and the validator set. This is
important as the only item that is signed by the validators is the
`Header`, and all other data must be validated against one of the merkle
hashes in the `Header`.
The `DataHash` can provide a nice check on the
[Data](https://godoc.org/github.com/tendermint/tendermint/types#Data)
returned in this same block. If you are subscribed to new blocks, via
tendermint RPC, in order to display or process the new transactions you
should at least validate that the `DataHash` is valid. If it is
important to verify autheniticity, you must wait for the `LastCommit`
from the next block to make sure the block header (including `DataHash`)
was properly signed.
The `ValidatorHash` contains a hash of the current
[Validators](https://godoc.org/github.com/tendermint/tendermint/types#Validator).
Tracking all changes in the validator set is complex, but a client can
quickly compare this hash with the [hash of the currently known
validators](https://godoc.org/github.com/tendermint/tendermint/types#ValidatorSet.Hash)
to see if there have been changes.
The `AppHash` serves as the basis for validating any merkle proofs that
come from the ABCI application. It represents the state of the actual
application, rather that the state of the blockchain itself. This means
it's necessary in order to perform any business logic, such as verifying
an account balance.
**Note** After the transactions are committed to a block, they still
need to be processed in a separate step, which happens between the
blocks. If you find a given transaction in the block at height `H`, the
effects of running that transaction will be first visible in the
`AppHash` from the block header at height `H+1`.
Like the `LastCommit` issue, this is a requirement of the immutability
of the block chain, as the application only applies transactions *after*
they are commited to the chain.
## Commit
The
[Commit](https://godoc.org/github.com/tendermint/tendermint/types#Commit)
contains a set of
[Votes](https://godoc.org/github.com/tendermint/tendermint/types#Vote)
that were made by the validator set to reach consensus on this block.
This is the key to the security in any PoS system, and actually no data
that cannot be traced back to a block header with a valid set of Votes
can be trusted. Thus, getting the Commit data and verifying the votes is
extremely important.
As mentioned above, in order to find the `precommit votes` for block
header `H`, we need to query block `H+1`. Then we need to check the
votes, make sure they really are for that block, and properly formatted.
Much of this code is implemented in Go in the
[light-client](https://github.com/tendermint/light-client) package. If
you look at the code, you will notice that we need to provide the
`chainID` of the blockchain in order to properly calculate the votes.
This is to protect anyone from swapping votes between chains to fake (or
frame) a validator. Also note that this `chainID` is in the
`genesis.json` from *Tendermint*, not the `genesis.json` from the
basecoin app ([that is a different
chainID...](https://github.com/cosmos/cosmos-sdk/issues/32)).
Once we have those votes, and we calculated the proper [sign
bytes](https://godoc.org/github.com/tendermint/tendermint/types#Vote.WriteSignBytes)
using the chainID and a [nice helper
function](https://godoc.org/github.com/tendermint/tendermint/types#SignBytes),
we can verify them. The light client is responsible for maintaining a
set of validators that we trust. Each vote only stores the validators
`Address`, as well as the `Signature`. Assuming we have a local copy of
the trusted validator set, we can look up the `Public Key` of the
validator given its `Address`, then verify that the `Signature` matches
the `SignBytes` and `Public Key`. Then we sum up the total voting power
of all validators, whose votes fulfilled all these stringent
requirements. If the total number of voting power for a single block is
greater than 2/3 of all voting power, then we can finally trust the
block header, the AppHash, and the proof we got from the ABCI
application.
### Vote Sign Bytes
The `sign-bytes` of a vote is produced by taking a
[stable-json](https://github.com/substack/json-stable-stringify)-like
deterministic JSON [wire](./wire-protocol.html) encoding of the vote
(excluding the `Signature` field), and wrapping it with
`{"chain_id":"my_chain","vote":...}`.
For example, a precommit vote might have the following `sign-bytes`:
```
{"chain_id":"my_chain","vote":{"block_hash":"611801F57B4CE378DF1A3FFF1216656E89209A99","block_parts_header":{"hash":"B46697379DBE0774CC2C3B656083F07CA7E0F9CE","total":123},"height":1234,"round":1,"type":2}}
```
## Block Hash
The [block
hash](https://godoc.org/github.com/tendermint/tendermint/types#Block.Hash)
is the [Simple Tree hash](./merkle.html#simple-tree-with-dictionaries)
of the fields of the block `Header` encoded as a list of `KVPair`s.
## Transaction
A transaction is any sequence of bytes. It is up to your ABCI
application to accept or reject transactions.
## BlockID
Many of these data structures refer to the
[BlockID](https://godoc.org/github.com/tendermint/tendermint/types#BlockID),
which is the `BlockHash` (hash of the block header, also referred to by
the next block) along with the `PartSetHeader`. The `PartSetHeader` is
explained below and is used internally to orchestrate the p2p
propogation. For clients, it is basically opaque bytes, but they must
match for all votes.
## PartSetHeader
The
[PartSetHeader](https://godoc.org/github.com/tendermint/tendermint/types#PartSetHeader)
contains the total number of pieces in a
[PartSet](https://godoc.org/github.com/tendermint/tendermint/types#PartSet),
and the Merkle root hash of those pieces.
## PartSet
PartSet is used to split a byteslice of data into parts (pieces) for
transmission. By splitting data into smaller parts and computing a
Merkle root hash on the list, you can verify that a part is legitimately
part of the complete data, and the part can be forwarded to other peers
before all the parts are known. In short, it's a fast way to securely
propagate a large chunk of data (like a block) over a gossip network.
PartSet was inspired by the LibSwift project.
Usage:
```
data := RandBytes(2 << 20) // Something large
partSet := NewPartSetFromData(data)
partSet.Total() // Total number of 4KB parts
partSet.Count() // Equal to the Total, since we already have all the parts
partSet.Hash() // The Merkle root hash
partSet.BitArray() // A BitArray of partSet.Total() 1's
header := partSet.Header() // Send this to the peer
header.Total // Total number of parts
header.Hash // The merkle root hash
// Now we'll reconstruct the data from the parts
partSet2 := NewPartSetFromHeader(header)
partSet2.Total() // Same total as partSet.Total()
partSet2.Count() // Zero, since this PartSet doesn't have any parts yet.
partSet2.Hash() // Same hash as in partSet.Hash()
partSet2.BitArray() // A BitArray of partSet.Total() 0's
// In a gossip network the parts would arrive in arbitrary order, perhaps
// in response to explicit requests for parts, or optimistically in response
// to the receiving peer's partSet.BitArray().
for !partSet2.IsComplete() {
part := receivePartFromGossipNetwork()
added, err := partSet2.AddPart(part)
if err != nil {
// A wrong part,
// the merkle trail does not hash to partSet2.Hash()
} else if !added {
// A duplicate part already received
}
}
data2, _ := ioutil.ReadAll(partSet2.GetReader())
bytes.Equal(data, data2) // true
```
docs/tendermint-core/light-client-protocol.md
+30
View File
@@ -0,0 +1,30 @@
# Light Client Protocol
Light clients are an important part of the complete blockchain system
for most applications. Tendermint provides unique speed and security
properties for light client applications.
See our [lite
package](https://godoc.org/github.com/tendermint/tendermint/lite).
## Overview
The objective of the light client protocol is to get a
[commit](./validators.md#committing-a-block) for a recent [block
hash](../spec/consensus/consensus.md.md#block-hash) where the commit includes a
majority of signatures from the last known validator set. From there,
all the application state is verifiable with [merkle
proofs](./merkle.md#iavl-tree).
## Properties
- You get the full collateralized security benefits of Tendermint; No
need to wait for confirmations.
- You get the full speed benefits of Tendermint; transactions
commit instantly.
- You can get the most recent version of the application state
non-interactively (without committing anything to the blockchain).
For example, this means that you can get the most recent value of a
name from the name-registry without worrying about fork censorship
attacks, without posting a commit and waiting for confirmations.
It's fast, secure, and free!
docs/tendermint-core/running-in-production.md
+63
View File
@@ -104,6 +104,69 @@ signals we use the default behaviour in Go: [Default behavior of signals
in Go
programs](https://golang.org/pkg/os/signal/#hdr-Default_behavior_of_signals_in_Go_programs).
## Corruption
**NOTE:** Make sure you have a backup of the Tendermint data directory.
### Possible causes
Remember that most corruption is caused by hardware issues:
- RAID controllers with faulty / worn out battery backup, and an unexpected power loss
- Hard disk drives with write-back cache enabled, and an unexpected power loss
- Cheap SSDs with insufficient power-loss protection, and an unexpected power-loss
- Defective RAM
- Defective or overheating CPU(s)
Other causes can be:
- Database systems configured with fsync=off and an OS crash or power loss
- Filesystems configured to use write barriers plus a storage layer that ignores write barriers. LVM is a particular culprit.
- Tendermint bugs
- Operating system bugs
- Admin error (e.g., directly modifying Tendermint data-directory contents)
(Source: https://wiki.postgresql.org/wiki/Corruption)
### WAL Corruption
If consensus WAL is corrupted at the lastest height and you are trying to start
Tendermint, replay will fail with panic.
Recovering from data corruption can be hard and time-consuming. Here are two approaches you can take:
1) Delete the WAL file and restart Tendermint. It will attempt to sync with other peers.
2) Try to repair the WAL file manually:
1. Create a backup of the corrupted WAL file:
```
cp "$TMHOME/data/cs.wal/wal" > /tmp/corrupted_wal_backup
```
2. Use `./scripts/wal2json` to create a human-readable version
```
./scripts/wal2json/wal2json "$TMHOME/data/cs.wal/wal" > /tmp/corrupted_wal
```
3. Search for a "CORRUPTED MESSAGE" line.
4. By looking at the previous message and the message after the corrupted one
and looking at the logs, try to rebuild the message. If the consequent
messages are marked as corrupted too (this may happen if length header
got corrupted or some writes did not make it to the WAL ~ truncation),
then remove all the lines starting from the corrupted one and restart
Tendermint.
```
$EDITOR /tmp/corrupted_wal
```
5. After editing, convert this file back into binary form by running:
```
./scripts/json2wal/json2wal /tmp/corrupted_wal > "$TMHOME/data/cs.wal/wal"
```
## Hardware
### Processor and Memory
docs/tendermint-core/secure-p2p.md
+73
View File
@@ -0,0 +1,73 @@
# Secure P2P
The Tendermint p2p protocol uses an authenticated encryption scheme
based on the [Station-to-Station
Protocol](https://en.wikipedia.org/wiki/Station-to-Station_protocol).
The implementation uses
[golang's](https://godoc.org/golang.org/x/crypto/nacl/box) [nacl
box](http://nacl.cr.yp.to/box.html) for the actual authenticated
encryption algorithm.
Each peer generates an ED25519 key-pair to use as a persistent
(long-term) id.
When two peers establish a TCP connection, they first each generate an
ephemeral ED25519 key-pair to use for this session, and send each other
their respective ephemeral public keys. This happens in the clear.
They then each compute the shared secret. The shared secret is the
multiplication of the peer's ephemeral private key by the other peer's
ephemeral public key. The result is the same for both peers by the magic
of [elliptic
curves](https://en.wikipedia.org/wiki/Elliptic_curve_cryptography). The
shared secret is used as the symmetric key for the encryption algorithm.
The two ephemeral public keys are sorted to establish a canonical order.
Then a 24-byte nonce is generated by concatenating the public keys and
hashing them with Ripemd160. Note Ripemd160 produces 20byte hashes, so
the nonce ends with four 0s.
The nonce is used to seed the encryption - it is critical that the same
nonce never be used twice with the same private key. For convenience,
the last bit of the nonce is flipped, giving us two nonces: one for
encrypting our own messages, one for decrypting our peer's. Which ever
peer has the higher public key uses the "bit-flipped" nonce for
encryption.
Now, a challenge is generated by concatenating the ephemeral public keys
and taking the SHA256 hash.
Each peer signs the challenge with their persistent private key, and
sends the other peer an AuthSigMsg, containing their persistent public
key and the signature. On receiving an AuthSigMsg, the peer verifies the
signature.
The peers are now authenticated.
All future communications can now be encrypted using the shared secret
and the generated nonces, where each nonce is incremented by one each
time it is used. The communications maintain Perfect Forward Secrecy, as
the persistent key pair was not used for generating secrets - only for
authenticating.
## Caveat
This system is still vulnerable to a Man-In-The-Middle attack if the
persistent public key of the remote node is not known in advance. The
only way to mitigate this is with a public key authentication system,
such as the Web-of-Trust or Certificate Authorities. In our case, we can
use the blockchain itself as a certificate authority to ensure that we
are connected to at least one validator.
## Config
Authenticated encryption is enabled by default.
## Additional Reading
- [Implementation](https://github.com/tendermint/tendermint/blob/64bae01d007b5bee0d0827ab53259ffd5910b4e6/p2p/conn/secret_connection.go#L47)
- [Original STS paper by Whitfield Diffie, Paul C. van Oorschot and
Michael J.
Wiener](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.216.6107&rep=rep1&type=pdf)
- [Further work on secret
handshakes](https://dominictarr.github.io/secret-handshake-paper/shs.pdf)
docs/tendermint-core/using-tendermint.md
+67
View File
@@ -31,6 +31,73 @@ For more elaborate initialization, see the tesnet command:
tendermint testnet --help
```
### Genesis
The `genesis.json` file in `$TMHOME/config/` defines the initial
TendermintCore state upon genesis of the blockchain ([see
definition](https://github.com/tendermint/tendermint/blob/master/types/genesis.go)).
#### Fields
- `genesis_time`: Official time of blockchain start.
- `chain_id`: ID of the blockchain. This must be unique for
every blockchain. If your testnet blockchains do not have unique
chain IDs, you will have a bad time.
- `validators`:
- `pub_key`: The first element specifies the `pub_key` type. 1
== Ed25519. The second element are the pubkey bytes.
- `power`: The validator's voting power.
- `name`: Name of the validator (optional).
- `app_hash`: The expected application hash (as returned by the
`ResponseInfo` ABCI message) upon genesis. If the app's hash does
not match, Tendermint will panic.
- `app_state`: The application state (e.g. initial distribution
of tokens).
#### Sample genesis.json
```
{
"genesis_time": "2018-07-09T22:43:06.255718641Z",
"chain_id": "chain-IAkWsK",
"validators": [
{
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "oX8HhKsErMluxI0QWNSR8djQMSupDvHdAYrHwP7n73k="
},
"power": "1",
"name": "node0"
},
{
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "UZNSJA9zmeFQj36Rs296lY+WFQ4Rt6s7snPpuKypl5I="
},
"power": "1",
"name": "node1"
},
{
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "i9GrM6/MHB4zjCelMZBUYHNXYIzl4n0RkDCVmmLhS/o="
},
"power": "1",
"name": "node2"
},
{
"pub_key": {
"type": "tendermint/PubKeyEd25519",
"value": "0qq7954l87trEqbQV9c7d1gurnjTGMxreXc848ZZ5aw="
},
"power": "1",
"name": "node3"
}
],
"app_hash": ""
}
```
## Run
To run a Tendermint node, use
docs/tendermint-core/validators.md
+39
View File
@@ -0,0 +1,39 @@
# Validators
Validators are responsible for committing new blocks in the blockchain.
These validators participate in the consensus protocol by broadcasting
*votes* which contain cryptographic signatures signed by each
validator's private key.
Some Proof-of-Stake consensus algorithms aim to create a "completely"
decentralized system where all stakeholders (even those who are not
always available online) participate in the committing of blocks.
Tendermint has a different approach to block creation. Validators are
expected to be online, and the set of validators is permissioned/curated
by some external process. Proof-of-stake is not required, but can be
implemented on top of Tendermint consensus. That is, validators may be
required to post collateral on-chain, off-chain, or may not be required
to post any collateral at all.
Validators have a cryptographic key-pair and an associated amount of
"voting power". Voting power need not be the same.
## Becoming a Validator
There are two ways to become validator.
1. They can be pre-established in the [genesis state](../../tendermint-core/using-tendermint.md#genesis)
2. The ABCI app responds to the EndBlock message with changes to the
existing validator set.
## Committing a Block
*+2/3 is short for "more than 2/3"*
A block is committed when +2/3 of the validator set sign [precommit
votes](../spec/blockchain/blockchain.md#vote) for that block at the same `round`.
The +2/3 set of precommit votes is called a
[*commit*](../spec/blockchain/blockchain.md#commit). While any +2/3 set of
precommits for the same block at the same height&round can serve as
validation, the canonical commit is included in the next block (see
[LastCommit](../spec/blockchain/blockchain.md#last-commit)).