mirrors/tendermint
1
0
Fork 0
mirror of https://github.com/tendermint/tendermint.git synced 2026-09-20 06:54:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'develop' into 1428-remove-wal-light

Browse Source
This commit is contained in:
Ethan Buchman
2018-04-27 12:06:08 -04:00
committed by GitHub
parent e88f74bb9b 65ebbccb74
commit 25cee8827a
108 changed files with 4021 additions and 1082 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/getting-started.rst
+32 -36
View File
@@ -81,9 +81,8 @@ Tendermint node as follows:
curl -s localhost:46657/status
The ``-s`` just silences ``curl``. For nicer output, pipe the result
into a tool like `jq <https://stedolan.github.io/jq/>`__ or
`jsonpp <https://github.com/jmhodges/jsonpp>`__.
The ``-s`` just silences ``curl``. For nicer output, pipe the result into a
tool like `jq <https://stedolan.github.io/jq/>`__ or ``json_pp``.
Now let's send some transactions to the kvstore.
@@ -104,17 +103,23 @@ like:
"id": "",
"result": {
"check_tx": {
"code": 0,
"data": "",
"log": ""
"fee": {}
},
"deliver_tx": {
"code": 0,
"data": "",
"log": ""
"tags": [
{
"key": "YXBwLmNyZWF0b3I=",
"value": "amFl"
},
{
"key": "YXBwLmtleQ==",
"value": "YWJjZA=="
}
],
"fee": {}
},
"hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
"height": 154
"hash": "9DF66553F98DE3C26E3C3317A3E4CED54F714E39",
"height": 14
}
}
@@ -134,20 +139,17 @@ The result should look like:
"id": "",
"result": {
"response": {
"code": 0,
"index": 0,
"key": "",
"value": "61626364",
"proof": "",
"height": 0,
"log": "exists"
"log": "exists",
"index": "-1",
"key": "YWJjZA==",
"value": "YWJjZA=="
}
}
}
Note the ``value`` in the result (``61626364``); this is the
hex-encoding of the ASCII of ``abcd``. You can verify this in
a python 2 shell by running ``"61626364".decode('hex')`` or in python 3 shell by running ``import codecs; codecs.decode("61626364", 'hex').decode('ascii')``. Stay
Note the ``value`` in the result (``YWJjZA==``); this is the
base64-encoding of the ASCII of ``abcd``. You can verify this in
a python 2 shell by running ``"61626364".decode('base64')`` or in python 3 shell by running ``import codecs; codecs.decode("61626364", 'base64').decode('ascii')``. Stay
tuned for a future release that `makes this output more human-readable <https://github.com/tendermint/abci/issues/32>`__.
Now let's try setting a different key and value:
@@ -157,7 +159,7 @@ Now let's try setting a different key and value:
curl -s 'localhost:46657/broadcast_tx_commit?tx="name=satoshi"'
Now if we query for ``name``, we should get ``satoshi``, or
``7361746F736869`` in hex:
``c2F0b3NoaQ==`` in base64:
::
@@ -226,17 +228,15 @@ the number ``1``. If instead, we try to send a ``5``, we get an error:
"id": "",
"result": {
"check_tx": {
"code": 0,
"data": "",
"log": ""
"fee": {}
},
"deliver_tx": {
"code": 3,
"data": "",
"log": "Invalid nonce. Expected 1, got 5"
"code": 2,
"log": "Invalid nonce. Expected 1, got 5",
"fee": {}
},
"hash": "33B93DFF98749B0D6996A70F64071347060DC19C",
"height": 38
"height": 34
}
}
@@ -250,17 +250,13 @@ But if we send a ``1``, it works again:
"id": "",
"result": {
"check_tx": {
"code": 0,
"data": "",
"log": ""
"fee": {}
},
"deliver_tx": {
"code": 0,
"data": "",
"log": ""
"fee": {}
},
"hash": "F17854A977F6FA7EEA1BD758E296710B86F72F3D",
"height": 87
"height": 60
}
}
docs/how-to-read-logs.rst
+1 -1
View File
@@ -59,7 +59,7 @@ Next we replay all the messages from the WAL.
::
I[10-04|13:54:30.391] Starting RPC HTTP server on tcp socket 0.0.0.0:46657 module=rpc-server
I[10-04|13:54:30.392] Started node module=main nodeInfo="NodeInfo{pk: PubKeyEd25519{DF22D7C92C91082324A1312F092AA1DA197FA598DBBFB6526E177003C4D6FD66}, moniker: anonymous, network: test-chain-3MNw2N [remote , listen 10.0.2.15:46656], version: 0.11.0-10f361fc ([wire_version=0.6.2 p2p_version=0.5.0 consensus_version=v1/0.2.2 rpc_version=0.7.0/3 tx_index=on rpc_addr=tcp://0.0.0.0:46657])}"
I[10-04|13:54:30.392] Started node module=main nodeInfo="NodeInfo{id: DF22D7C92C91082324A1312F092AA1DA197FA598DBBFB6526E, moniker: anonymous, network: test-chain-3MNw2N [remote , listen 10.0.2.15:46656], version: 0.11.0-10f361fc ([wire_version=0.6.2 p2p_version=0.5.0 consensus_version=v1/0.2.2 rpc_version=0.7.0/3 tx_index=on rpc_addr=tcp://0.0.0.0:46657])}"
Next follows a standard block creation cycle, where we enter a new round,
propose a block, receive more than 2/3 of prevotes, then precommits and finally
docs/install.rst
+3 -2
View File
@@ -4,7 +4,7 @@ Install Tendermint
From Binary
-----------
To download pre-built binaries, see the `Download page <https://tendermint.com/download>`__.
To download pre-built binaries, see the `Download page <https://tendermint.com/downloads>`__.
From Source
-----------
@@ -37,13 +37,13 @@ First, install ``dep``:
::
cd $GOPATH/src/github.com/tendermint/tendermint
make get_tools
Now we can fetch the correct versions of each dependency by running:
::
cd $GOPATH/src/github.com/tendermint/tendermint
make get_vendor_deps
make install
@@ -96,6 +96,7 @@ If ``go get`` failing bothers you, fetch the code using ``git``:
mkdir -p $GOPATH/src/github.com/tendermint
git clone https://github.com/tendermint/tendermint $GOPATH/src/github.com/tendermint/tendermint
cd $GOPATH/src/github.com/tendermint/tendermint
make get_tools
make get_vendor_deps
make install
docs/specification/genesis.rst
+2 -2
View File
@@ -18,8 +18,8 @@ Fields
- ``power``: The validator's voting power.
- ``name``: Name of the validator (optional).
- ``app_hash``: The expected application hash (as returned by the
``Commit`` ABCI message) upon genesis. If the app's hash does not
match, a warning message is printed.
``ResponseInfo`` ABCI message) upon genesis. If the app's hash does not
match, Tendermint will panic.
- ``app_state``: The application state (e.g. initial distribution of tokens).
Sample genesis.json
docs/specification/new-spec/abci.md
+69
View File
@@ -0,0 +1,69 @@
# Application Blockchain Interface (ABCI)
ABCI is the interface between Tendermint (a state-machine replication engine)
and an application (the actual state machine).
The ABCI message types are defined in a [protobuf
file](https://github.com/tendermint/abci/blob/master/types/types.proto).
For full details on the ABCI message types and protocol, see the [ABCI
specificaiton](https://github.com/tendermint/abci/blob/master/specification.rst).
For additional details on server implementation, see the [ABCI
readme](https://github.com/tendermint/abci#implementation).
Here we provide some more details around the use of ABCI by Tendermint and
clarify common "gotchas".
## Validator Updates
Updates to the Tendermint validator set can be made by returning `Validator`
objects in the `ResponseBeginBlock`:
```
message Validator {
bytes pub_key = 1;
int64 power = 2;
}
```
The `pub_key` is the Amino encoded public key for the validator. For details on
Amino encoded public keys, see the [section of the encoding spec](./encoding.md#public-key-cryptography).
For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 32-byte Ed25519 pubkey
`76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
Amino encoded as
`1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`
(Note: in old versions of Tendermint (pre-v0.19.0), the pubkey is just prefixed with a
single type byte, so for ED25519 we'd have `pub_key = 0x1 | pub`)
The `power` is the new voting power for the validator, with the
following rules:
- power must be non-negative
- if power is 0, the validator must already exist, and will be removed from the
validator set
- if power is non-0:
- if the validator does not already exist, it will be added to the validator
set with the given power
- if the validator does already exist, its power will be adjusted to the given power
## Query
Query is a generic message type with lots of flexibility to enable diverse sets
of queries from applications. Tendermint has no requirements from the Query
message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too.
That said, Tendermint makes a number of queries to support some optional
features. These are:
### Peer Filtering
When Tendermint connects to a peer, it sends two queries to the ABCI application
using the following paths, with no additional data:
- `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and
the port of the connection
- `p2p/filter/pubkey/<ID>`, where `<ID>` is the peer node ID (ie. the
pubkey.Address() for the peer's PubKey)
If either of these queries return a non-zero ABCI code, Tendermint will refuse
to connect to the peer.
docs/specification/new-spec/encoding.md
+230 -116
View File
@@ -1,106 +1,163 @@
# Tendermint Encoding
## Binary Serialization (TMBIN)
## Amino
Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs
are laid out in memory.
Variable length items are length-prefixed.
While the encoding was inspired by Go, it is easily implemented in other languages as well, given its intuitive design.
Tendermint uses the Protobuf3 derrivative [Amino]() for all data structures.
Think of Amino as an object-oriented Protobuf3 with native JSON support.
The goal of the Amino encoding protocol is to bring parity between application
logic objects and persistence objects.
XXX: This is changing to use real varints and 4-byte-prefixes.
See https://github.com/tendermint/go-wire/tree/sdk2.
Please see the [Amino
specification](https://github.com/tendermint/go-amino#amino-encoding-for-go) for
more details.
### Fixed Length Integers
Notably, every object that satisfies an interface (eg. a particular kind of p2p message,
or a particular kind of pubkey) is registered with a global name, the hash of
which is included in the object's encoding as the so-called "prefix bytes".
Fixed length integers are encoded in Big-Endian using the specified number of bytes.
So `uint8` and `int8` use one byte, `uint16` and `int16` use two bytes,
`uint32` and `int32` use 3 bytes, and `uint64` and `int64` use 4 bytes.
We define the `func AminoEncode(obj interface{}) []byte` function to take an
arbitrary object and return the Amino encoded bytes.
Negative integers are encoded via twos-complement.
## Byte Arrays
Examples:
The encoding of a byte array is simply the raw-bytes prefixed with the length of
the array as a `UVarint` (what Protobuf calls a `Varint`).
```go
encode(uint8(6)) == [0x06]
encode(uint32(6)) == [0x00, 0x00, 0x00, 0x06]
For details on varints, see the [protobuf
spec](https://developers.google.com/protocol-buffers/docs/encoding#varints).
encode(int8(-6)) == [0xFA]
encode(int32(-6)) == [0xFF, 0xFF, 0xFF, 0xFA]
For example, the byte-array `[0xA, 0xB]` would be encoded as `0x020A0B`,
while a byte-array containing 300 entires beginning with `[0xA, 0xB, ...]` would
be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300.
## Public Key Cryptography
Tendermint uses Amino to distinguish between different types of private keys,
public keys, and signatures. Additionally, for each public key, Tendermint
defines an Address function that can be used as a more compact identifier in
place of the public key. Here we list the concrete types, their names,
and prefix bytes for public keys and signatures, as well as the address schemes
for each PubKey. Note for brevity we don't
include details of the private keys beyond their type and name, as they can be
derrived the same way as the others using Amino.
All registered objects are encoded by Amino using a 4-byte PrefixBytes that
uniquely identifies the object and includes information about its underlying
type. For details on how PrefixBytes are computed, see the [Amino
spec](https://github.com/tendermint/go-amino#computing-the-prefix-and-disambiguation-bytes).
In what follows, we provide the type names and prefix bytes directly.
Notice that when encoding byte-arrays, the length of the byte-array is appended
to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes>
<Length> <ByteArray>`
(NOTE: the remainder of this section on Public Key Cryptography can be generated
from [this script](./scripts/crypto.go))
### PubKeyEd25519
```
// Name: tendermint/PubKeyEd25519
// PrefixBytes: 0x1624DE62
// Length: 0x20
// Notes: raw 32-byte Ed25519 pubkey
type PubKeyEd25519 [32]byte
func (pubkey PubKeyEd25519) Address() []byte {
// NOTE: hash of the Amino encoded bytes!
return RIPEMD160(AminoEncode(pubkey))
}
```
### Variable Length Integers
For example, the 32-byte Ed25519 pubkey
`CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E` would be
encoded as
`1624DE6220CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E`.
Variable length integers are encoded as length-prefixed Big-Endian integers.
The length-prefix consists of a single byte and corresponds to the length of the encoded integer.
The address would then be
`RIPEMD160(0x1624DE6220CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E)`
or `430FF75BAF1EC4B0D51BB3EEC2955479D0071605`
Negative integers are encoded by flipping the leading bit of the length-prefix to a `1`.
### SignatureEd25519
Zero is encoded as `0x00`. It is not length-prefixed.
Examples:
```go
encode(uint(6)) == [0x01, 0x06]
encode(uint(70000)) == [0x03, 0x01, 0x11, 0x70]
encode(int(-6)) == [0xF1, 0x06]
encode(int(-70000)) == [0xF3, 0x01, 0x11, 0x70]
encode(int(0)) == [0x00]
```
// Name: tendermint/SignatureKeyEd25519
// PrefixBytes: 0x3DA1DB2A
// Length: 0x40
// Notes: raw 64-byte Ed25519 signature
type SignatureEd25519 [64]byte
```
### Strings
For example, the 64-byte Ed25519 signature
`1B6034A8ED149D3C94FDA13EC03B26CC0FB264D9B0E47D3FA3DEF9FCDE658E49C80B35F9BE74949356401B15B18FB817D6E54495AD1C4A8401B248466CB0DB0B`
would be encoded as
`3DA1DB2A401B6034A8ED149D3C94FDA13EC03B26CC0FB264D9B0E47D3FA3DEF9FCDE658E49C80B35F9BE74949356401B15B18FB817D6E54495AD1C4A8401B248466CB0DB0B`
An encoded string is length-prefixed followed by the underlying bytes of the string.
The length-prefix is itself encoded as an `int`.
### PrivKeyEd25519
The empty string is encoded as `0x00`. It is not length-prefixed.
Examples:
```go
encode("") == [0x00]
encode("a") == [0x01, 0x01, 0x61]
encode("hello") == [0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F]
encode("¥") == [0x01, 0x02, 0xC2, 0xA5]
```
// Name: tendermint/PrivKeyEd25519
// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key
type PrivKeyEd25519 [64]byte
```
### Arrays (fixed length)
### PubKeySecp256k1
An encoded fix-lengthed array is the concatenation of the encoding of its elements.
There is no length-prefix.
```
// Name: tendermint/PubKeySecp256k1
// PrefixBytes: 0xEB5AE982
// Length: 0x21
// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03
type PubKeySecp256k1 [33]byte
Examples:
```go
encode([4]int8{1, 2, 3, 4}) == [0x01, 0x02, 0x03, 0x04]
encode([4]int16{1, 2, 3, 4}) == [0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
encode([4]int{1, 2, 3, 4}) == [0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x04]
encode([2]string{"abc", "efg"}) == [0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
func (pubkey PubKeySecp256k1) Address() []byte {
// NOTE: hash of the raw pubkey bytes (not Amino encoded!).
// Compatible with Bitcoin addresses.
return RIPEMD160(SHA256(pubkey[:]))
}
```
### Slices (variable length)
For example, the 33-byte Secp256k1 pubkey
`020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9` would be
encoded as
`EB5AE98221020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
An encoded variable-length array is length-prefixed followed by the concatenation of the encoding of
its elements.
The length-prefix is itself encoded as an `int`.
The address would then be
`RIPEMD160(SHA256(0x020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9))`
or `0AE5BEE929ABE51BAD345DB925EEA652680783FC`
An empty slice is encoded as `0x00`. It is not length-prefixed.
### SignatureSecp256k1
Examples:
```go
encode([]int8{}) == [0x00]
encode([]int8{1, 2, 3, 4}) == [0x01, 0x04, 0x01, 0x02, 0x03, 0x04]
encode([]int16{1, 2, 3, 4}) == [0x01, 0x04, 0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
encode([]int{1, 2, 3, 4}) == [0x01, 0x04, 0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x4]
encode([]string{"abc", "efg"}) == [0x01, 0x02, 0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
```
// Name: tendermint/SignatureKeySecp256k1
// PrefixBytes: 0x16E1FEEA
// Length: Variable
// Encoding prefix: Variable
// Notes: raw bytes of the Secp256k1 signature
type SignatureSecp256k1 []byte
```
For example, the Secp256k1 signature
`304402201CD4B8C764D2FD8AF23ECFE6666CA8A53886D47754D951295D2D311E1FEA33BF02201E0F906BB1CF2C30EAACFFB032A7129358AFF96B9F79B06ACFFB18AC90C2ADD7`
would be encoded as
`16E1FEEA46304402201CD4B8C764D2FD8AF23ECFE6666CA8A53886D47754D951295D2D311E1FEA33BF02201E0F906BB1CF2C30EAACFFB032A7129358AFF96B9F79B06ACFFB18AC90C2ADD7`
### PrivKeySecp256k1
```
// Name: tendermint/PrivKeySecp256k1
// Notes: raw 32-byte priv key
type PrivKeySecp256k1 [32]byte
```
## Other Common Types
### BitArray
BitArray is encoded as an `int` of the number of bits, and with an array of `uint64` to encode
value of each array element.
The BitArray is used in block headers and some consensus messages to signal
whether or not something was done by each validator. BitArray is represented
with a struct containing the number of bits (`Bits`) and the bit-array itself
encoded in base64 (`Elems`).
```go
type BitArray struct {
@@ -109,36 +166,35 @@ type BitArray struct {
}
```
### Time
This type is easily encoded directly by Amino.
Time is encoded as an `int64` of the number of nanoseconds since January 1, 1970,
rounded to the nearest millisecond.
Note BitArray receives a special JSON encoding in the form of `x` and `_`
representing `1` and `0`. Ie. the BitArray `10110` would be JSON encoded as
`"x_xx_"`
Times before then are invalid.
### Part
Examples:
Part is used to break up blocks into pieces that can be gossiped in parallel
and securely verified using a Merkle tree of the parts.
Part contains the index of the part in the larger set (`Index`), the actual
underlying data of the part (`Bytes`), and a simple Merkle proof that the part is contained in
the larger set (`Proof`).
```go
encode(time.Time("Jan 1 00:00:00 UTC 1970")) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
encode(time.Time("Jan 1 00:00:01 UTC 1970")) == [0x00, 0x00, 0x00, 0x00, 0x3B, 0x9A, 0xCA, 0x00] // 1,000,000,000 ns
encode(time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")) == [0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
type Part struct {
Index int
Bytes byte[]
Proof byte[]
}
```
### Structs
### MakeParts
An encoded struct is the concatenation of the encoding of its elements.
There is no length-prefix.
Examples:
Encode an object using Amino and slice it into parts.
```go
type MyStruct struct{
A int
B string
C time.Time
}
encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) ==
[0x01, 0x04, 0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F, 0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
func MakeParts(obj interface{}, partSize int) []Part
```
## Merkle Trees
@@ -147,6 +203,8 @@ Simple Merkle trees are used in numerous places in Tendermint to compute a crypt
RIPEMD160 is always used as the hashing function.
### Simple Merkle Root
The function `SimpleMerkleRoot` is a simple recursive function defined as follows:
```go
@@ -159,20 +217,96 @@ func SimpleMerkleRoot(hashes [][]byte) []byte{
default:
left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2])
right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:])
return RIPEMD160(append(left, right))
return SimpleConcatHash(left, right)
}
}
func SimpleConcatHash(left, right []byte) []byte{
left = encodeByteSlice(left)
right = encodeByteSlice(right)
return RIPEMD160 (append(left, right))
}
```
Note: we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
Note that the leaves are Amino encoded as byte-arrays (ie. simple Uvarint length
prefix) before being concatenated together and hashed.
Note: we will abuse notion and invoke `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to
field name and then hashing them.
For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements.
## JSON (TMJSON)
### Simple Merkle Proof
Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN.
TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64.
Proof that a leaf is in a Merkle tree consists of a simple structure:
```
type SimpleProof struct {
Aunts [][]byte
}
```
Which is verified using the following:
```
func (proof SimpleProof) Verify(index, total int, leafHash, rootHash []byte) bool {
computedHash := computeHashFromAunts(index, total, leafHash, proof.Aunts)
return computedHash == rootHash
}
func computeHashFromAunts(index, total int, leafHash []byte, innerHashes [][]byte) []byte{
assert(index < total && index >= 0 && total > 0)
if total == 1{
assert(len(proof.Aunts) == 0)
return leafHash
}
assert(len(innerHashes) > 0)
numLeft := (total + 1) / 2
if index < numLeft {
leftHash := computeHashFromAunts(index, numLeft, leafHash, innerHashes[:len(innerHashes)-1])
assert(leftHash != nil)
return SimpleHashFromTwoHashes(leftHash, innerHashes[len(innerHashes)-1])
}
rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1])
assert(rightHash != nil)
return SimpleHashFromTwoHashes(innerHashes[len(innerHashes)-1], rightHash)
}
```
## JSON
### Amino
TODO: improve this
Amino also supports JSON encoding - registered types are simply encoded as:
```
{
"type": "<DisfixBytes>",
"value": <JSON>
}
For instance, an ED25519 PubKey would look like:
```
{
"type": "AC26791624DE60",
"value": "uZ4h63OFWuQ36ZZ4Bd6NF+/w9fWUwrOncrQsackrsTk="
}
```
Where the `"value"` is the base64 encoding of the raw pubkey bytes, and the
`"type"` is the full disfix bytes for Ed25519 pubkeys.
### Signed Messages
Signed messages (eg. votes, proposals) in the consensus are encoded using Amino-JSON, rather than in the standard binary format.
When signing, the elements of a message are sorted by key and the sorted message is embedded in an
outer JSON that includes a `chain_id` field.
@@ -184,23 +318,3 @@ like:
```
Note how the fields within each level are sorted.
## Other
### MakeParts
Encode an object using TMBIN and slice it into parts.
```go
MakeParts(object, partSize)
```
### Part
```go
type Part struct {
Index int
Bytes byte[]
Proof byte[]
}
```
docs/specification/new-spec/p2p/peer.md
+2 -2
View File
@@ -83,7 +83,7 @@ The Tendermint Version Handshake allows the peers to exchange their NodeInfo:
```golang
type NodeInfo struct {
PubKey crypto.PubKey
ID p2p.ID
Moniker string
Network string
RemoteAddr string
@@ -95,7 +95,7 @@ type NodeInfo struct {
```
The connection is disconnected if:
- `peer.NodeInfo.PubKey != peer.PubKey`
- `peer.NodeInfo.ID` is not equal `peerConn.ID`
- `peer.NodeInfo.Version` is not formatted as `X.X.X` where X are integers known as Major, Minor, and Revision
- `peer.NodeInfo.Version` Major is not the same as ours
- `peer.NodeInfo.Version` Minor is not the same as ours
docs/specification/new-spec/pre-amino.md
+246
View File
@@ -0,0 +1,246 @@
# Tendermint Encoding (Pre-Amino)
## PubKeys and Addresses
PubKeys are prefixed with a type-byte, followed by the raw bytes of the public
key.
Two keys are supported with the following type bytes:
```
TypeByteEd25519 = 0x1
TypeByteSecp256k1 = 0x2
```
```
// TypeByte: 0x1
type PubKeyEd25519 [32]byte
func (pub PubKeyEd25519) Encode() []byte {
return 0x1 | pub
}
func (pub PubKeyEd25519) Address() []byte {
// NOTE: the length (0x0120) is also included
return RIPEMD160(0x1 | 0x0120 | pub)
}
// TypeByte: 0x2
// NOTE: OpenSSL compressed pubkey (x-cord with 0x2 or 0x3)
type PubKeySecp256k1 [33]byte
func (pub PubKeySecp256k1) Encode() []byte {
return 0x2 | pub
}
func (pub PubKeySecp256k1) Address() []byte {
return RIPEMD160(SHA256(pub))
}
```
See https://github.com/tendermint/go-crypto/blob/v0.5.0/pub_key.go for more.
## Binary Serialization (go-wire)
Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs
are laid out in memory.
Variable length items are length-prefixed.
While the encoding was inspired by Go, it is easily implemented in other languages as well, given its intuitive design.
XXX: This is changing to use real varints and 4-byte-prefixes.
See https://github.com/tendermint/go-wire/tree/sdk2.
### Fixed Length Integers
Fixed length integers are encoded in Big-Endian using the specified number of bytes.
So `uint8` and `int8` use one byte, `uint16` and `int16` use two bytes,
`uint32` and `int32` use 3 bytes, and `uint64` and `int64` use 4 bytes.
Negative integers are encoded via twos-complement.
Examples:
```go
encode(uint8(6)) == [0x06]
encode(uint32(6)) == [0x00, 0x00, 0x00, 0x06]
encode(int8(-6)) == [0xFA]
encode(int32(-6)) == [0xFF, 0xFF, 0xFF, 0xFA]
```
### Variable Length Integers
Variable length integers are encoded as length-prefixed Big-Endian integers.
The length-prefix consists of a single byte and corresponds to the length of the encoded integer.
Negative integers are encoded by flipping the leading bit of the length-prefix to a `1`.
Zero is encoded as `0x00`. It is not length-prefixed.
Examples:
```go
encode(uint(6)) == [0x01, 0x06]
encode(uint(70000)) == [0x03, 0x01, 0x11, 0x70]
encode(int(-6)) == [0xF1, 0x06]
encode(int(-70000)) == [0xF3, 0x01, 0x11, 0x70]
encode(int(0)) == [0x00]
```
### Strings
An encoded string is length-prefixed followed by the underlying bytes of the string.
The length-prefix is itself encoded as an `int`.
The empty string is encoded as `0x00`. It is not length-prefixed.
Examples:
```go
encode("") == [0x00]
encode("a") == [0x01, 0x01, 0x61]
encode("hello") == [0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F]
encode("¥") == [0x01, 0x02, 0xC2, 0xA5]
```
### Arrays (fixed length)
An encoded fix-lengthed array is the concatenation of the encoding of its elements.
There is no length-prefix.
Examples:
```go
encode([4]int8{1, 2, 3, 4}) == [0x01, 0x02, 0x03, 0x04]
encode([4]int16{1, 2, 3, 4}) == [0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
encode([4]int{1, 2, 3, 4}) == [0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x04]
encode([2]string{"abc", "efg"}) == [0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
```
### Slices (variable length)
An encoded variable-length array is length-prefixed followed by the concatenation of the encoding of
its elements.
The length-prefix is itself encoded as an `int`.
An empty slice is encoded as `0x00`. It is not length-prefixed.
Examples:
```go
encode([]int8{}) == [0x00]
encode([]int8{1, 2, 3, 4}) == [0x01, 0x04, 0x01, 0x02, 0x03, 0x04]
encode([]int16{1, 2, 3, 4}) == [0x01, 0x04, 0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
encode([]int{1, 2, 3, 4}) == [0x01, 0x04, 0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x4]
encode([]string{"abc", "efg"}) == [0x01, 0x02, 0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
```
### BitArray
BitArray is encoded as an `int` of the number of bits, and with an array of `uint64` to encode
value of each array element.
```go
type BitArray struct {
Bits int
Elems []uint64
}
```
### Time
Time is encoded as an `int64` of the number of nanoseconds since January 1, 1970,
rounded to the nearest millisecond.
Times before then are invalid.
Examples:
```go
encode(time.Time("Jan 1 00:00:00 UTC 1970")) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
encode(time.Time("Jan 1 00:00:01 UTC 1970")) == [0x00, 0x00, 0x00, 0x00, 0x3B, 0x9A, 0xCA, 0x00] // 1,000,000,000 ns
encode(time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")) == [0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
```
### Structs
An encoded struct is the concatenation of the encoding of its elements.
There is no length-prefix.
Examples:
```go
type MyStruct struct{
A int
B string
C time.Time
}
encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) ==
[0x01, 0x04, 0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F, 0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
```
## Merkle Trees
Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
RIPEMD160 is always used as the hashing function.
The function `SimpleMerkleRoot` is a simple recursive function defined as follows:
```go
func SimpleMerkleRoot(hashes [][]byte) []byte{
switch len(hashes) {
case 0:
return nil
case 1:
return hashes[0]
default:
left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2])
right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:])
return RIPEMD160(append(left, right))
}
}
```
Note: we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to
field name and then hashing them.
For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements.
## JSON (TMJSON)
Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN.
TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64.
When signing, the elements of a message are sorted by key and the sorted message is embedded in an
outer JSON that includes a `chain_id` field.
We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look
like:
```json
{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2}
```
Note how the fields within each level are sorted.
## Other
### MakeParts
Encode an object using TMBIN and slice it into parts.
```go
MakeParts(object, partSize)
```
### Part
```go
type Part struct {
Index int
Bytes byte[]
Proof byte[]
}
```
docs/specification/new-spec/scripts/crypto.go
+125
View File
@@ -0,0 +1,125 @@
package main
import (
"fmt"
crypto "github.com/tendermint/go-crypto"
)
// SECRET
var SECRET = []byte("some secret")
func printEd() {
priv := crypto.GenPrivKeyEd25519FromSecret(SECRET)
pub := priv.PubKey().(crypto.PubKeyEd25519)
sig := priv.Sign([]byte("hello")).(crypto.SignatureEd25519)
name := "tendermint/PubKeyEd25519"
length := len(pub[:])
fmt.Println("### PubKeyEd25519")
fmt.Println("")
fmt.Println("```")
fmt.Printf("// Name: %s\n", name)
fmt.Printf("// PrefixBytes: 0x%X \n", pub.Bytes()[:4])
fmt.Printf("// Length: 0x%X \n", length)
fmt.Println("// Notes: raw 32-byte Ed25519 pubkey")
fmt.Println("type PubKeyEd25519 [32]byte")
fmt.Println("")
fmt.Println(`func (pubkey PubKeyEd25519) Address() []byte {
// NOTE: hash of the Amino encoded bytes!
return RIPEMD160(AminoEncode(pubkey))
}`)
fmt.Println("```")
fmt.Println("")
fmt.Printf("For example, the 32-byte Ed25519 pubkey `%X` would be encoded as `%X`.\n\n", pub[:], pub.Bytes())
fmt.Printf("The address would then be `RIPEMD160(0x%X)` or `%X`\n", pub.Bytes(), pub.Address())
fmt.Println("")
name = "tendermint/SignatureKeyEd25519"
length = len(sig[:])
fmt.Println("### SignatureEd25519")
fmt.Println("")
fmt.Println("```")
fmt.Printf("// Name: %s\n", name)
fmt.Printf("// PrefixBytes: 0x%X \n", sig.Bytes()[:4])
fmt.Printf("// Length: 0x%X \n", length)
fmt.Println("// Notes: raw 64-byte Ed25519 signature")
fmt.Println("type SignatureEd25519 [64]byte")
fmt.Println("```")
fmt.Println("")
fmt.Printf("For example, the 64-byte Ed25519 signature `%X` would be encoded as `%X`\n", sig[:], sig.Bytes())
fmt.Println("")
name = "tendermint/PrivKeyEd25519"
fmt.Println("### PrivKeyEd25519")
fmt.Println("")
fmt.Println("```")
fmt.Println("// Name:", name)
fmt.Println("// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key")
fmt.Println("type PrivKeyEd25519 [64]byte")
fmt.Println("```")
}
func printSecp() {
priv := crypto.GenPrivKeySecp256k1FromSecret(SECRET)
pub := priv.PubKey().(crypto.PubKeySecp256k1)
sig := priv.Sign([]byte("hello")).(crypto.SignatureSecp256k1)
name := "tendermint/PubKeySecp256k1"
length := len(pub[:])
fmt.Println("### PubKeySecp256k1")
fmt.Println("")
fmt.Println("```")
fmt.Printf("// Name: %s\n", name)
fmt.Printf("// PrefixBytes: 0x%X \n", pub.Bytes()[:4])
fmt.Printf("// Length: 0x%X \n", length)
fmt.Println("// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03")
fmt.Println("type PubKeySecp256k1 [33]byte")
fmt.Println("")
fmt.Println(`func (pubkey PubKeySecp256k1) Address() []byte {
// NOTE: hash of the raw pubkey bytes (not Amino encoded!).
// Compatible with Bitcoin addresses.
return RIPEMD160(SHA256(pubkey[:]))
}`)
fmt.Println("```")
fmt.Println("")
fmt.Printf("For example, the 33-byte Secp256k1 pubkey `%X` would be encoded as `%X`\n\n", pub[:], pub.Bytes())
fmt.Printf("The address would then be `RIPEMD160(SHA256(0x%X))` or `%X`\n", pub[:], pub.Address())
fmt.Println("")
name = "tendermint/SignatureKeySecp256k1"
fmt.Println("### SignatureSecp256k1")
fmt.Println("")
fmt.Println("```")
fmt.Printf("// Name: %s\n", name)
fmt.Printf("// PrefixBytes: 0x%X \n", sig.Bytes()[:4])
fmt.Printf("// Length: Variable\n")
fmt.Printf("// Encoding prefix: Variable\n")
fmt.Println("// Notes: raw bytes of the Secp256k1 signature")
fmt.Println("type SignatureSecp256k1 []byte")
fmt.Println("```")
fmt.Println("")
fmt.Printf("For example, the Secp256k1 signature `%X` would be encoded as `%X`\n", []byte(sig[:]), sig.Bytes())
fmt.Println("")
name = "tendermint/PrivKeySecp256k1"
fmt.Println("### PrivKeySecp256k1")
fmt.Println("")
fmt.Println("```")
fmt.Println("// Name:", name)
fmt.Println("// Notes: raw 32-byte priv key")
fmt.Println("type PrivKeySecp256k1 [32]byte")
fmt.Println("```")
}
func main() {
printEd()
fmt.Println("")
printSecp()
}
docs/specification/new-spec/state.md
-3
View File
@@ -74,9 +74,6 @@ func TotalVotingPower(vals []Validators) int64{
}
```
### PubKey
TODO:
### ConsensusParams
docs/specification/new-spec/wire.go
-80
View File
@@ -1,80 +0,0 @@
package main
import (
"fmt"
"time"
wire "github.com/tendermint/go-wire"
)
func main() {
encode(uint8(6))
encode(uint32(6))
encode(int8(-6))
encode(int32(-6))
Break()
encode(uint(6))
encode(uint(70000))
encode(int(0))
encode(int(-6))
encode(int(-70000))
Break()
encode("")
encode("a")
encode("hello")
encode("¥")
Break()
encode([4]int8{1, 2, 3, 4})
encode([4]int16{1, 2, 3, 4})
encode([4]int{1, 2, 3, 4})
encode([2]string{"abc", "efg"})
Break()
encode([]int8{})
encode([]int8{1, 2, 3, 4})
encode([]int16{1, 2, 3, 4})
encode([]int{1, 2, 3, 4})
encode([]string{"abc", "efg"})
Break()
timeFmt := "Mon Jan 2 15:04:05 -0700 MST 2006"
t1, _ := time.Parse(timeFmt, timeFmt)
n := (t1.UnixNano() / 1000000.) * 1000000
encode(n)
encode(t1)
t2, _ := time.Parse(timeFmt, "Thu Jan 1 00:00:00 -0000 UTC 1970")
encode(t2)
t2, _ = time.Parse(timeFmt, "Thu Jan 1 00:00:01 -0000 UTC 1970")
fmt.Println("N", t2.UnixNano())
encode(t2)
Break()
encode(struct {
A int
B string
C time.Time
}{
4,
"hello",
t1,
})
}
func encode(i interface{}) {
Println(wire.BinaryBytes(i))
}
func Println(b []byte) {
s := "["
for _, x := range b {
s += fmt.Sprintf("0x%.2X, ", x)
}
s = s[:len(s)-2] + "]"
fmt.Println(s)
}
func Break() {
fmt.Println("------")
}
docs/using-tendermint.rst
+60 -67
View File
@@ -74,20 +74,17 @@ RPC server, for example:
curl http://localhost:46657/broadcast_tx_commit?tx=\"abcd\"
For handling responses, we recommend you `install the jsonpp
tool <http://jmhodges.github.io/jsonpp/>`__ to pretty print the JSON.
We can see the chain's status at the ``/status`` end-point:
::
curl http://localhost:46657/status | jsonpp
curl http://localhost:46657/status | json_pp
and the ``latest_app_hash`` in particular:
::
curl http://localhost:46657/status | jsonpp | grep app_hash
curl http://localhost:46657/status | json_pp | grep latest_app_hash
Visit http://localhost:46657 in your browser to see the list of other
endpoints. Some take no arguments (like ``/status``), while others
@@ -260,19 +257,19 @@ When ``tendermint init`` is run, both a ``genesis.json`` and
::
{
"app_hash": "",
"chain_id": "test-chain-HZw6TB",
"genesis_time": "0001-01-01T00:00:00.000Z",
"validators": [
{
"power": 10,
"name": "",
"pub_key": [
1,
"5770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
]
}
]
"validators" : [
{
"pub_key" : {
"value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
"type" : "AC26791624DE60"
},
"power" : 10,
"name" : ""
}
],
"app_hash" : "",
"chain_id" : "test-chain-rDlYSN",
"genesis_time" : "0001-01-01T00:00:00Z"
}
And the ``priv_validator.json``:
@@ -280,20 +277,18 @@ And the ``priv_validator.json``:
::
{
"address": "4F4D895F882A18E1D1FC608D102601DA8D3570E5",
"last_height": 0,
"last_round": 0,
"last_signature": null,
"last_signbytes": "",
"last_step": 0,
"priv_key": [
1,
"F9FA3CD435BDAE54D0BCA8F1BC289D718C23D855C6DB21E8543F5E4F457E62805770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
],
"pub_key": [
1,
"5770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
]
"last_step" : 0,
"last_round" : 0,
"address" : "B788DEDE4F50AD8BC9462DE76741CCAFF87D51E2",
"pub_key" : {
"value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
"type" : "AC26791624DE60"
},
"last_height" : 0,
"priv_key" : {
"value" : "JPivl82x+LfVkp8i3ztoTjY6c6GJ4pBxQexErOCyhwqHeGT5ATxzpAtPJKnxNx/NyUnD8Ebv3OIYH+kgD4N88Q==",
"type" : "954568A3288910"
}
}
The ``priv_validator.json`` actually contains a private key, and should
@@ -387,20 +382,18 @@ Now we can update our genesis file. For instance, if the new
::
{
"address": "AC379688105901436A34A65F185C115B8BB277A1",
"last_height": 0,
"last_round": 0,
"last_signature": null,
"last_signbytes": "",
"last_step": 0,
"priv_key": [
1,
"0D2ED337D748ADF79BE28559B9E59EBE1ABBA0BAFE6D65FCB9797985329B950C8F2B5AACAACC9FCE41881349743B0CFDE190DF0177744568D4E82A18F0B7DF94"
],
"pub_key": [
1,
"8F2B5AACAACC9FCE41881349743B0CFDE190DF0177744568D4E82A18F0B7DF94"
]
"address" : "5AF49D2A2D4F5AD4C7C8C4CC2FB020131E9C4902",
"pub_key" : {
"value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=",
"type" : "AC26791624DE60"
},
"priv_key" : {
"value" : "EDJY9W6zlAw+su6ITgTKg2nTZcHAH1NMTW5iwlgmNDuX1f35+OR4HMN88ZtQzsAwhETq4k3vzM3n6WTk5ii16Q==",
"type" : "954568A3288910"
},
"last_step" : 0,
"last_round" : 0,
"last_height" : 0
}
then the new ``genesis.json`` will be:
@@ -408,27 +401,27 @@ then the new ``genesis.json`` will be:
::
{
"app_hash": "",
"chain_id": "test-chain-HZw6TB",
"genesis_time": "0001-01-01T00:00:00.000Z",
"validators": [
{
"power": 10,
"name": "",
"pub_key": [
1,
"5770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
]
},
{
"power": 10,
"name": "",
"pub_key": [
1,
"8F2B5AACAACC9FCE41881349743B0CFDE190DF0177744568D4E82A18F0B7DF94"
]
}
]
"validators" : [
{
"pub_key" : {
"value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
"type" : "AC26791624DE60"
},
"power" : 10,
"name" : ""
},
{
"pub_key" : {
"value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=",
"type" : "AC26791624DE60"
},
"power" : 10,
"name" : ""
}
],
"app_hash" : "",
"chain_id" : "test-chain-rDlYSN",
"genesis_time" : "0001-01-01T00:00:00Z"
}
Update the ``genesis.json`` in ``~/.tendermint/config``. Copy the genesis file