mirrors/tendermint
1
0
Fork 0
mirror of https://github.com/tendermint/tendermint.git synced 2026-01-08 14:21:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

spec: protobuf changes (#156)

Browse Source 
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
This commit is contained in:
Marko
2020-10-06 12:40:25 +02:00
committed by GitHub
parent 04fb20e33d
commit c5576dfa69
8 changed files with 137 additions and 177 deletions
Download Patch File Download Diff File Expand all files Collapse all files

145
spec/core/encoding.md
View File

@@ -1,22 +1,10 @@
# Encoding
## Amino
## Protocol Buffers
Tendermint uses the proto3 derivative [Amino](https://github.com/tendermint/go-amino) for all data structures.
Think of Amino as an object-oriented proto3 with native JSON support.
The goal of the Amino encoding protocol is to bring parity between application
logic objects and persistence objects.
Tendermint uses [Protocol Buffers](https://developers.google.com/protocol-buffers), specifically proto3, for all data structures.
Please see the [Amino
specification](https://github.com/tendermint/go-amino#amino-encoding-for-go) for
more details.
Notably, every object that satisfies an interface (eg. a particular kind of p2p message,
or a particular kind of pubkey) is registered with a global name, the hash of
which is included in the object's encoding as the so-called "prefix bytes".
We define the `func AminoEncode(obj interface{}) []byte` function to take an
arbitrary object and return the Amino encoded bytes.
Please see the [Proto3 language guide](https://developers.google.com/protocol-buffers/docs/proto3) for more details.
## Byte Arrays
@@ -34,47 +22,18 @@ be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300.
Tendermint uses `SHA256` as its hash function.
Objects are always Amino encoded before being hashed.
So `SHA256(obj)` is short for `SHA256(AminoEncode(obj))`.
So `SHA256(obj)` is short for `SHA256(ProtoEncoding(obj))`.
## Public Key Cryptography
Tendermint uses Amino to distinguish between different types of private keys,
public keys, and signatures. Additionally, for each public key, Tendermint
Tendermint uses Protobuf [Oneof](https://developers.google.com/protocol-buffers/docs/proto3#oneof)
to distinguish between different types public keys, and signatures.
Additionally, for each public key, Tendermint
defines an Address function that can be used as a more compact identifier in
place of the public key. Here we list the concrete types, their names,
and prefix bytes for public keys and signatures, as well as the address schemes
for each PubKey. Note for brevity we don't
include details of the private keys beyond their type and name, as they can be
derived the same way as the others using Amino.
All registered objects are encoded by Amino using a 4-byte PrefixBytes that
uniquely identifies the object and includes information about its underlying
type. For details on how PrefixBytes are computed, see the [Amino
spec](https://github.com/tendermint/go-amino#computing-the-prefix-and-disambiguation-bytes).
In what follows, we provide the type names and prefix bytes directly.
Notice that when encoding byte-arrays, the length of the byte-array is appended
to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes> <Length> <ByteArray>`. In other words, to encode any type listed below you do not need to be
familiar with amino encoding.
You can simply use below table and concatenate Prefix || Length (of raw bytes) || raw bytes
( while || stands for byte concatenation here).
| Type | Name | Prefix | Length | Notes |
| ----------------------- | ---------------------------------- | ---------- | -------- | ----- |
| PubKeyEd25519 | tendermint/PubKeyEd25519 | 0x1624DE64 | 0x20 | |
| PubKeySr25519 | tendermint/PubKeySr25519 | 0x0DFB1005 | 0x20 | |
| PubKeySecp256k1 | tendermint/PubKeySecp256k1 | 0xEB5AE987 | 0x21 | |
| PrivKeyEd25519 | tendermint/PrivKeyEd25519 | 0xA3288910 | 0x40 | |
| PrivKeySr25519 | tendermint/PrivKeySr25519 | 0x2F82D78B | 0x20 | |
| PrivKeySecp256k1 | tendermint/PrivKeySecp256k1 | 0xE1B0F79B | 0x20 | |
| PubKeyMultisigThreshold | tendermint/PubKeyMultisigThreshold | 0x22C1F7E2 | variable | |
### Example
For example, the 33-byte (or 0x21-byte in hex) Secp256k1 pubkey
`020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
would be encoded as
`EB5AE98721020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
include details of the private keys beyond their type and name.
### Key Types
@@ -92,39 +51,6 @@ address = SHA256(pubkey)[:20]
The signature is the raw 64-byte ED25519 signature.
#### Sr25519
TODO: pubkey
The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key:
```go
address = SHA256(pubkey)[:20]
```
The signature is the raw 64-byte ED25519 signature.
#### Secp256k1
TODO: pubkey
The address is the RIPEMD160 hash of the SHA256 hash of the OpenSSL compressed public key:
```go
address = RIPEMD160(SHA256(pubkey))
```
This is the same as Bitcoin.
The signature is the 64-byte concatenation of ECDSA `r` and `s` (ie. `r || s`),
where `s` is lexicographically less than its inverse, to prevent malleability.
This is like Ethereum, but without the extra byte for pubkey recovery, since
Tendermint assumes the pubkey is always provided anyway.
#### Multisig
TODO
## Other Common Types
### BitArray
@@ -136,7 +62,7 @@ encoded in base64 (`Elems`).
```go
type BitArray struct {
Bits int
Bits int64
Elems []uint64
}
```
@@ -158,7 +84,7 @@ the set (`Proof`).
```go
type Part struct {
Index int
Index uint32
Bytes []byte
Proof SimpleProof
}
@@ -168,7 +94,7 @@ See details of SimpleProof, below.
### MakeParts
Encode an object using Amino and slice it into parts.
Encode an object using Protobuf and slice it into parts.
Tendermint uses a part size of 65536 bytes, and allows a maximum of 1601 parts
(see `types.MaxBlockPartsCount`). This corresponds to the hard-coded block size
limit of 100MB.
@@ -260,27 +186,27 @@ func Hashes(items [][]byte) [][]byte {
```
Note: we will abuse notion and invoke `MerkleRoot` with arguments of type `struct` or type `[]struct`.
For `struct` arguments, we compute a `[][]byte` containing the amino encoding of each
For `struct` arguments, we compute a `[][]byte` containing the protobuf encoding of each
field in the struct, in the same order the fields appear in the struct.
For `[]struct` arguments, we compute a `[][]byte` by amino encoding the individual `struct` elements.
For `[]struct` arguments, we compute a `[][]byte` by protobuf encoding the individual `struct` elements.
### Simple Merkle Proof
### Merkle Proof
Proof that a leaf is in a Merkle tree is composed as follows:
```golang
type SimpleProof struct {
Total int
Index int
LeafHash []byte
Aunts [][]byte
type Proof struct {
Total int
Index int
LeafHash []byte
Aunts [][]byte
}
```
Which is verified as follows:
```golang
func (proof SimpleProof) Verify(rootHash []byte, leaf []byte) bool {
func (proof Proof) Verify(rootHash []byte, leaf []byte) bool {
assert(proof.LeafHash, leafHash(leaf)
computedHash := computeHashFromAunts(proof.Index, proof.Total, proof.LeafHash, proof.Aunts)
@@ -319,13 +245,13 @@ Because Tendermint only uses a Simple Merkle Tree, application developers are ex
## JSON
### Amino
Tendermint has its own JSON encoding in order to keep backwards compatibility with the prvious RPC layer.
Amino also supports JSON encoding - registered types are simply encoded as:
Registered types are encoded as:
```json
{
"type": "<amino type name>",
"type": "<type name>",
"value": <JSON>
}
```
@@ -340,30 +266,33 @@ For instance, an ED25519 PubKey would look like:
```
Where the `"value"` is the base64 encoding of the raw pubkey bytes, and the
`"type"` is the amino name for Ed25519 pubkeys.
`"type"` is the type name for Ed25519 pubkeys.
### Signed Messages
Signed messages (eg. votes, proposals) in the consensus are encoded using Amino.
Signed messages (eg. votes, proposals) in the consensus are encoded using protobuf.
When signing, the elements of a message are re-ordered so the fixed-length fields
are first, making it easy to quickly check the type, height, and round.
The `ChainID` is also appended to the end.
We call this encoding the SignBytes. For instance, SignBytes for a vote is the Amino encoding of the following struct:
We call this encoding the SignBytes. For instance, SignBytes for a vote is the protobuf encoding of the following struct:
```go
type CanonicalVote struct {
Type byte
Height int64 `binary:"fixed64"`
Round int64 `binary:"fixed64"`
BlockID CanonicalBlockID
Timestamp time.Time
ChainID string
```protobuf
message CanonicalVote {
SignedMsgType type = 1;
sfixed64 height = 2; // canonicalization requires fixed size encoding here
sfixed64 round = 3; // canonicalization requires fixed size encoding here
CanonicalBlockID block_id = 4;
google.protobuf.Timestamp timestamp = 5;
string chain_id = 6;
}
```
The field ordering and the fixed sized encoding for the first three fields is optimized to ease parsing of SignBytes
in HSMs. It creates fixed offsets for relevant fields that need to be read in this context.
> Note: All canonical messages are length prefixed.
For more details, see the [signing spec](../consensus/signing.md).
Also, see the motivating discussion in
[#1622](https://github.com/tendermint/tendermint/issues/1622).