mirrors/tendermint
1
0
Fork 0
mirror of https://github.com/tendermint/tendermint.git synced 2026-01-03 11:45:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

spec: protobuf changes (#156)

Browse Source 
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
This commit is contained in:
Marko
2020-10-06 12:40:25 +02:00
committed by GitHub
parent 04fb20e33d
commit c5576dfa69
8 changed files with 137 additions and 177 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
spec/abci/abci.md
View File

@@ -61,6 +61,23 @@ Each event has a `type` which is meant to categorize the event for a particular
particular event. Every key and value in an event's attributes must be UTF-8
encoded strings along with the event type itself.
```protobuf
message Event {
string type = 1;
repeated EventAttribute attributes = 2;
}
```
The attributes of an `Event` consist of a `key`, `value` and a `index`. The index field notifies the indexer within Tendermint to index the event. This field is non-deterministic and will vary across different nodes in the network.
```protobuf
message EventAttribute {
bytes key = 1;
bytes value = 2;
bool index = 3; // nondeterministic
}
```
Example:
```go
@@ -563,12 +580,7 @@ via light client.
### PubKey
- **Fields**:
- `Type (string)`: Type of the public key. A simple string like `"ed25519"`.
In the future, may indicate a serialization algorithm to parse the `Data`,
for instance `"amino"`.
- `Data ([]byte)`: Public key data. For a simple public key, it's just the
raw bytes. If the `Type` indicates an encoding algorithm, this is the
encoded public key.
- `Sum (oneof PublicKey)`: This field is a Protobuf [`oneof`](https://developers.google.com/protocol-buffers/docs/proto#oneof)
- **Usage**:
- A generic and extensible typed public key
@@ -626,8 +638,8 @@ via light client.
### ValidatorParams
- **Fields**:
- `PubKeyTypes ([]string)`: List of accepted pubkey types. Uses same
naming as `PubKey.Type`.
- `PubKeyTypes ([]string)`: List of accepted public key types.
- Uses same naming as `PubKey.Type`.
### VersionParams

30
spec/abci/apps.md
View File

@@ -250,21 +250,21 @@ blockchain.
Updates to the Tendermint validator set can be made by returning
`ValidatorUpdate` objects in the `ResponseEndBlock`:
```proto
```protobuf
message ValidatorUpdate {
PubKey pub_key
tendermint.crypto.keys.PublicKey pub_key
int64 power
}
message PubKey {
string type
bytes data
}
message PublicKey {
oneof {
ed25519 bytes = 1;
}
```
The `pub_key` currently supports only one type:
- `type = "ed25519"` and `data = <raw 32-byte public key>`
- `type = "ed25519"`
The `power` is the new voting power for the validator, with the
following rules:
@@ -288,7 +288,7 @@ evidence. They can be set in InitChain and updated in EndBlock.
### BlockParams.MaxBytes
The maximum size of a complete Amino encoded block.
The maximum size of a complete Protobuf encoded block.
This is enforced by Tendermint consensus.
This implies a maximum tx size that is this MaxBytes, less the expected size of
@@ -314,6 +314,8 @@ This is enforced by Tendermint consensus.
Must have `TimeIotaMs > 0` to ensure time monotonicity.
> *Note: This is not exposed to the application*
### EvidenceParams.MaxAgeDuration
This is the maximum age of evidence in time units.
@@ -383,7 +385,7 @@ local node's state - hence they may return stale reads. For reads that require
consensus, use a transaction.
The most important use of Query is to return Merkle proofs of the application state at some height
that can be used for efficient application-specific lite-clients.
that can be used for efficient application-specific light-clients.
Note Tendermint has technically no requirements from the Query
message for normal operation - that is, the ABCI app developer need not implement
@@ -402,9 +404,9 @@ While some applications keep all relevant state in the transactions themselves
(like Bitcoin and its UTXOs), others maintain a separated state that is
computed deterministically *from* transactions, but is not contained directly in
the transactions themselves (like Ethereum contracts and accounts).
For such applications, the `AppHash` provides a much more efficient way to verify lite-client proofs.
For such applications, the `AppHash` provides a much more efficient way to verify light-client proofs.
ABCI applications can take advantage of more efficient lite-client proofs for
ABCI applications can take advantage of more efficient light-client proofs for
their state as follows:
- return the Merkle root of the deterministic application state in
@@ -413,15 +415,15 @@ their state as follows:
- return efficient Merkle proofs about that application state in `ResponseQuery.Proof`
that can be verified using the `AppHash` of the corresponding block.
For instance, this allows an application's lite-client to verify proofs of
For instance, this allows an application's light-client to verify proofs of
absence in the application state, something which is much less efficient to do using the block hash.
Some applications (eg. Ethereum, Cosmos-SDK) have multiple "levels" of Merkle trees,
where the leaves of one tree are the root hashes of others. To support this, and
the general variability in Merkle proofs, the `ResponseQuery.Proof` has some minimal structure:
```proto
message Proof {
```protobuf
message ProofOps {
repeated ProofOp ops
}

2
spec/abci/client-server.md
View File

@@ -41,7 +41,7 @@ See examples, in various stages of maintenance, in
### In Process
The simplest implementation uses function calls within Golang.
This means ABCI applications written in Golang can be compiled with TendermintCore and run as a single binary.
This means ABCI applications written in Golang can be compiled with Tendermint Core and run as a single binary.
### GRPC

31
spec/core/data_structures.md
View File

@@ -89,11 +89,11 @@ parts (ie. `len(MakeParts(block))`)
```go
type BlockID struct {
Hash []byte
PartsHeader PartSetHeader
PartSetHeader PartSetHeader
}
type PartSetHeader struct {
Total int32
Total uint32
Hash []byte
}
```
@@ -125,7 +125,7 @@ validator. It also contains the relevant BlockID, height and round:
```go
type Commit struct {
Height int64
Round int
Round int32
BlockID BlockID
Signatures []CommitSig
}
@@ -168,17 +168,28 @@ The vote includes information about the validator signing it.
```go
type Vote struct {
Type byte
Type SignedMsgType
Height int64
Round int
Round int32
BlockID BlockID
Timestamp Time
ValidatorAddress []byte
ValidatorIndex int
ValidatorIndex int32
Signature []byte
}
```
```protobuf
enum SignedMsgType {
SIGNED_MSG_TYPE_UNKNOWN = 0;
// Votes
PREVOTE_TYPE = 1;
PRECOMMIT_TYPE = 2;
// Proposals
PROPOSAL_TYPE = 32;
}
```
There are two types of votes:
a _prevote_ has `vote.Type == 1` and
a _precommit_ has `vote.Type == 2`.
@@ -412,7 +423,7 @@ first by voting power (descending), then by address (ascending).
block.ConsensusHash == state.ConsensusParams.Hash()
```
Hash of the amino-encoding of a subset of the consensus parameters.
Hash of the protobuf-encoding of a subset of the consensus parameters.
### AppHash
@@ -497,9 +508,9 @@ The number of votes in a commit is limited to 10000 (see `types.MaxVotesCount`).
### Vote
A vote is a signed message broadcast in the consensus for a particular block at a particular height and round.
When stored in the blockchain or propagated over the network, votes are encoded in Amino.
For signing, votes are represented via `CanonicalVote` and also encoded using amino (protobuf compatible) via
`Vote.SignBytes` which includes the `ChainID`, and uses a different ordering of
When stored in the blockchain or propagated over the network, votes are encoded in Protobuf.
For signing, votes are represented via `CanonicalVote` and also encoded using Protobuf via
`VoteSignBytes` which includes the `ChainID`, and uses a different ordering of
the fields.
We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the `SignBytes`

145
spec/core/encoding.md
View File

@@ -1,22 +1,10 @@
# Encoding
## Amino
## Protocol Buffers
Tendermint uses the proto3 derivative [Amino](https://github.com/tendermint/go-amino) for all data structures.
Think of Amino as an object-oriented proto3 with native JSON support.
The goal of the Amino encoding protocol is to bring parity between application
logic objects and persistence objects.
Tendermint uses [Protocol Buffers](https://developers.google.com/protocol-buffers), specifically proto3, for all data structures.
Please see the [Amino
specification](https://github.com/tendermint/go-amino#amino-encoding-for-go) for
more details.
Notably, every object that satisfies an interface (eg. a particular kind of p2p message,
or a particular kind of pubkey) is registered with a global name, the hash of
which is included in the object's encoding as the so-called "prefix bytes".
We define the `func AminoEncode(obj interface{}) []byte` function to take an
arbitrary object and return the Amino encoded bytes.
Please see the [Proto3 language guide](https://developers.google.com/protocol-buffers/docs/proto3) for more details.
## Byte Arrays
@@ -34,47 +22,18 @@ be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300.
Tendermint uses `SHA256` as its hash function.
Objects are always Amino encoded before being hashed.
So `SHA256(obj)` is short for `SHA256(AminoEncode(obj))`.
So `SHA256(obj)` is short for `SHA256(ProtoEncoding(obj))`.
## Public Key Cryptography
Tendermint uses Amino to distinguish between different types of private keys,
public keys, and signatures. Additionally, for each public key, Tendermint
Tendermint uses Protobuf [Oneof](https://developers.google.com/protocol-buffers/docs/proto3#oneof)
to distinguish between different types public keys, and signatures.
Additionally, for each public key, Tendermint
defines an Address function that can be used as a more compact identifier in
place of the public key. Here we list the concrete types, their names,
and prefix bytes for public keys and signatures, as well as the address schemes
for each PubKey. Note for brevity we don't
include details of the private keys beyond their type and name, as they can be
derived the same way as the others using Amino.
All registered objects are encoded by Amino using a 4-byte PrefixBytes that
uniquely identifies the object and includes information about its underlying
type. For details on how PrefixBytes are computed, see the [Amino
spec](https://github.com/tendermint/go-amino#computing-the-prefix-and-disambiguation-bytes).
In what follows, we provide the type names and prefix bytes directly.
Notice that when encoding byte-arrays, the length of the byte-array is appended
to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes> <Length> <ByteArray>`. In other words, to encode any type listed below you do not need to be
familiar with amino encoding.
You can simply use below table and concatenate Prefix || Length (of raw bytes) || raw bytes
( while || stands for byte concatenation here).
| Type | Name | Prefix | Length | Notes |
| ----------------------- | ---------------------------------- | ---------- | -------- | ----- |
| PubKeyEd25519 | tendermint/PubKeyEd25519 | 0x1624DE64 | 0x20 | |
| PubKeySr25519 | tendermint/PubKeySr25519 | 0x0DFB1005 | 0x20 | |
| PubKeySecp256k1 | tendermint/PubKeySecp256k1 | 0xEB5AE987 | 0x21 | |
| PrivKeyEd25519 | tendermint/PrivKeyEd25519 | 0xA3288910 | 0x40 | |
| PrivKeySr25519 | tendermint/PrivKeySr25519 | 0x2F82D78B | 0x20 | |
| PrivKeySecp256k1 | tendermint/PrivKeySecp256k1 | 0xE1B0F79B | 0x20 | |
| PubKeyMultisigThreshold | tendermint/PubKeyMultisigThreshold | 0x22C1F7E2 | variable | |
### Example
For example, the 33-byte (or 0x21-byte in hex) Secp256k1 pubkey
`020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
would be encoded as
`EB5AE98721020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
include details of the private keys beyond their type and name.
### Key Types
@@ -92,39 +51,6 @@ address = SHA256(pubkey)[:20]
The signature is the raw 64-byte ED25519 signature.
#### Sr25519
TODO: pubkey
The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key:
```go
address = SHA256(pubkey)[:20]
```
The signature is the raw 64-byte ED25519 signature.
#### Secp256k1
TODO: pubkey
The address is the RIPEMD160 hash of the SHA256 hash of the OpenSSL compressed public key:
```go
address = RIPEMD160(SHA256(pubkey))
```
This is the same as Bitcoin.
The signature is the 64-byte concatenation of ECDSA `r` and `s` (ie. `r || s`),
where `s` is lexicographically less than its inverse, to prevent malleability.
This is like Ethereum, but without the extra byte for pubkey recovery, since
Tendermint assumes the pubkey is always provided anyway.
#### Multisig
TODO
## Other Common Types
### BitArray
@@ -136,7 +62,7 @@ encoded in base64 (`Elems`).
```go
type BitArray struct {
Bits int
Bits int64
Elems []uint64
}
```
@@ -158,7 +84,7 @@ the set (`Proof`).
```go
type Part struct {
Index int
Index uint32
Bytes []byte
Proof SimpleProof
}
@@ -168,7 +94,7 @@ See details of SimpleProof, below.
### MakeParts
Encode an object using Amino and slice it into parts.
Encode an object using Protobuf and slice it into parts.
Tendermint uses a part size of 65536 bytes, and allows a maximum of 1601 parts
(see `types.MaxBlockPartsCount`). This corresponds to the hard-coded block size
limit of 100MB.
@@ -260,27 +186,27 @@ func Hashes(items [][]byte) [][]byte {
```
Note: we will abuse notion and invoke `MerkleRoot` with arguments of type `struct` or type `[]struct`.
For `struct` arguments, we compute a `[][]byte` containing the amino encoding of each
For `struct` arguments, we compute a `[][]byte` containing the protobuf encoding of each
field in the struct, in the same order the fields appear in the struct.
For `[]struct` arguments, we compute a `[][]byte` by amino encoding the individual `struct` elements.
For `[]struct` arguments, we compute a `[][]byte` by protobuf encoding the individual `struct` elements.
### Simple Merkle Proof
### Merkle Proof
Proof that a leaf is in a Merkle tree is composed as follows:
```golang
type SimpleProof struct {
Total int
Index int
LeafHash []byte
Aunts [][]byte
type Proof struct {
Total int
Index int
LeafHash []byte
Aunts [][]byte
}
```
Which is verified as follows:
```golang
func (proof SimpleProof) Verify(rootHash []byte, leaf []byte) bool {
func (proof Proof) Verify(rootHash []byte, leaf []byte) bool {
assert(proof.LeafHash, leafHash(leaf)
computedHash := computeHashFromAunts(proof.Index, proof.Total, proof.LeafHash, proof.Aunts)
@@ -319,13 +245,13 @@ Because Tendermint only uses a Simple Merkle Tree, application developers are ex
## JSON
### Amino
Tendermint has its own JSON encoding in order to keep backwards compatibility with the prvious RPC layer.
Amino also supports JSON encoding - registered types are simply encoded as:
Registered types are encoded as:
```json
{
"type": "<amino type name>",
"type": "<type name>",
"value": <JSON>
}
```
@@ -340,30 +266,33 @@ For instance, an ED25519 PubKey would look like:
```
Where the `"value"` is the base64 encoding of the raw pubkey bytes, and the
`"type"` is the amino name for Ed25519 pubkeys.
`"type"` is the type name for Ed25519 pubkeys.
### Signed Messages
Signed messages (eg. votes, proposals) in the consensus are encoded using Amino.
Signed messages (eg. votes, proposals) in the consensus are encoded using protobuf.
When signing, the elements of a message are re-ordered so the fixed-length fields
are first, making it easy to quickly check the type, height, and round.
The `ChainID` is also appended to the end.
We call this encoding the SignBytes. For instance, SignBytes for a vote is the Amino encoding of the following struct:
We call this encoding the SignBytes. For instance, SignBytes for a vote is the protobuf encoding of the following struct:
```go
type CanonicalVote struct {
Type byte
Height int64 `binary:"fixed64"`
Round int64 `binary:"fixed64"`
BlockID CanonicalBlockID
Timestamp time.Time
ChainID string
```protobuf
message CanonicalVote {
SignedMsgType type = 1;
sfixed64 height = 2; // canonicalization requires fixed size encoding here
sfixed64 round = 3; // canonicalization requires fixed size encoding here
CanonicalBlockID block_id = 4;
google.protobuf.Timestamp timestamp = 5;
string chain_id = 6;
}
```
The field ordering and the fixed sized encoding for the first three fields is optimized to ease parsing of SignBytes
in HSMs. It creates fixed offsets for relevant fields that need to be read in this context.
> Note: All canonical messages are length prefixed.
For more details, see the [signing spec](../consensus/signing.md).
Also, see the motivating discussion in
[#1622](https://github.com/tendermint/tendermint/issues/1622).

70
spec/core/state.md
View File

@@ -7,7 +7,7 @@ necessary for validating new blocks. For instance, the validators set and the re
transactions are never included in blocks, but their Merkle roots are - the state keeps track of them.
Note that the `State` object itself is an implementation detail, since it is never
included in a block or gossipped over the network, and we never compute
included in a block or gossiped over the network, and we never compute
its hash. Thus we do not include here details of how the `State` object is
persisted or queried. That said, the types it contains are part of the specification, since
their Merkle roots are included in blocks and their values are used in
@@ -55,21 +55,24 @@ type Consensus struct {
}
```
### Result
### ResponseDeliverTx
```go
type Result struct {
Code uint32
Data []byte
```protobuf
message ResponseDeliverTx {
uint32 code = 1;
bytes data = 2;
string log = 3; // nondeterministic
string info = 4; // nondeterministic
int64 gas_wanted = 5;
int64 gas_used = 6;
repeated Event events = 7
[(gogoproto.nullable) = false, (gogoproto.jsontag) = "events,omitempty"];
string codespace = 8;
}
```
`Result` is the result of executing a transaction against the application.
It returns a result code and an arbitrary byte array (ie. a return value).
NOTE: the Result needs to be updated to include more fields returned from
processing transactions, like gas variables and events - see
[issue 1007](https://github.com/tendermint/tendermint/issues/1007).
`ResponseDeliverTx` is the result of executing a transaction against the application.
It returns a result code (`uint32`), an arbitrary byte array (`[]byte`) (ie. a return value), Log (`string`), Info (`string`), GasWanted (`int64`), GasUsed (`int64`), Events (`[]Events`) and a Codespace (`string`).
### Validator
@@ -110,45 +113,48 @@ Like validator sets, they are set during genesis and can be updated by the appli
When hashed, only a subset of the params are included, to allow the params to
evolve without breaking the header.
```go
type ConsensusParams struct {
Block
Evidence
Validator
Version
```protobuf
message ConsensusParams {
BlockParams block = 1;
EvidenceParams evidence = 2;
ValidatorParams validator = 3;
VersionParams version = 4;
}
```
```go
type hashedParams struct {
BlockMaxBytes int64
BlockMaxGas int64
}
func (params ConsensusParams) Hash() []byte {
func HashConsensusParams() []byte {
SHA256(hashedParams{
BlockMaxBytes: params.Block.MaxBytes,
BlockMaxGas: params.Block.MaxGas,
})
}
```
type BlockParams struct {
MaxBytes int64
MaxGas int64
TimeIotaMs int64
```protobuf
message BlockParams {
int64 max_bytes = 1;
int64 max_gas = 2;
int64 time_iota_ms = 3; // not exposed to the application
}
type EvidenceParams struct {
MaxAgeNumBlocks int64
MaxAgeDuration time.Duration
MaxNum uint32
ProofTrialPeriod int64
message EvidenceParams {
int64 max_age_num_blocks = 1;
google.protobuf.Duration max_age_duration = 2;
uint32 max_num = 3;
}
type ValidatorParams struct {
PubKeyTypes []string
message ValidatorParams {
repeated string pub_key_types = 1;
}
type VersionParams struct {
AppVersion uint64
message VersionParams {
uint64 AppVersion = 1;
}
```

2
spec/p2p/connection.md
View File

@@ -38,7 +38,7 @@ type msgPacket struct {
}
```
The `msgPacket` is serialized using [go-amino](https://github.com/tendermint/go-amino) and prefixed with 0x3.
The `msgPacket` is serialized using [Proto3](https://developers.google.com/protocol-buffers/docs/proto3).
The received `Bytes` of a sequential set of packets are appended together
until a packet with `EOF=1` is received, then the complete serialized message
is returned for processing by the `onReceive` function of the corresponding channel.

6
spec/p2p/peer.md
View File

@@ -21,7 +21,7 @@ corresponding to `<ID>`. This prevents man-in-the-middle attacks on the peer lay
All p2p connections use TCP.
Upon establishing a successful TCP connection with a peer,
two handhsakes are performed: one for authenticated encryption, and one for Tendermint versioning.
two handshakes are performed: one for authenticated encryption, and one for Tendermint versioning.
Both handshakes have configurable timeouts (they should complete quickly).
### Authenticated Encryption Handshake
@@ -29,8 +29,8 @@ Both handshakes have configurable timeouts (they should complete quickly).
Tendermint implements the Station-to-Station protocol
using X25519 keys for Diffie-Helman key-exchange and chacha20poly1305 for encryption.
Previous versions of this protocol suffered from malleability attacks whereas an active man
in the middle attacker could compromise confidentiality as decribed in [Prime, Order Please!
Previous versions of this protocol (0.32 and below) suffered from malleability attacks whereas an active man
in the middle attacker could compromise confidentiality as described in [Prime, Order Please!
Revisiting Small Subgroup and Invalid Curve Attacks on
Protocols using Diffie-Hellman](https://eprint.iacr.org/2019/526.pdf).