mirror of
https://github.com/tendermint/tendermint.git
synced 2026-09-20 15:04:22 +00:00
disable markdown link checker and migrate spec RFCs
This commit is contained in:
Callum Waters
Binary file not shown.
|
Before Width: | Height: | Size: 52 KiB |
@@ -1,4 +1,4 @@
|
||||
# RFC 014: ABCI++
|
||||
# RFC 011: ABCI++
|
||||
|
||||
## Changelog
|
||||
|
||||
@@ -1,109 +0,0 @@
|
||||
# RFC 011: Configurable Block Retention
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2020-03-23: Initial draft (@erikgrinaker)
|
||||
- 2020-03-25: Use local config for snapshot interval (@erikgrinaker)
|
||||
- 2020-03-31: Use ABCI commit response for block retention hint
|
||||
- 2020-04-02: Resolved open questions
|
||||
- 2021-02-11: Migrate RFC to tendermint repo (Originally [RFC 001](https://github.com/tendermint/spec/pull/84))
|
||||
|
||||
## Author(s)
|
||||
|
||||
- Erik Grinaker (@erikgrinaker)
|
||||
|
||||
## Context
|
||||
|
||||
Currently, all Tendermint nodes contain the complete sequence of blocks from genesis up to some height (typically the latest chain height). This will no longer be true when the following features are released:
|
||||
|
||||
- [Block pruning](https://github.com/tendermint/tendermint/issues/3652): removes historical blocks and associated data (e.g. validator sets) up to some height, keeping only the most recent blocks.
|
||||
|
||||
- [State sync](https://github.com/tendermint/tendermint/issues/828): bootstraps a new node by syncing state machine snapshots at a given height, but not historical blocks and associated data.
|
||||
|
||||
To maintain the integrity of the chain, the use of these features must be coordinated such that necessary historical blocks will not become unavailable or lost forever. In particular:
|
||||
|
||||
- Some nodes should have complete block histories, for auditability, querying, and bootstrapping.
|
||||
|
||||
- The majority of nodes should retain blocks longer than the Cosmos SDK unbonding period, for light client verification.
|
||||
|
||||
- Some nodes must take and serve state sync snapshots with snapshot intervals less than the block retention periods, to allow new nodes to state sync and then replay blocks to catch up.
|
||||
|
||||
- Applications may not persist their state on commit, and require block replay on restart.
|
||||
|
||||
- Only a minority of nodes can be state synced within the unbonding period, for light client verification and to serve block histories for catch-up.
|
||||
|
||||
However, it is unclear if and how we should enforce this. It may not be possible to technically enforce all of these without knowing the state of the entire network, but it may also be unrealistic to expect this to be enforced entirely through social coordination. This is especially unfortunate since the consequences of misconfiguration can be permanent chain-wide data loss.
|
||||
|
||||
## Proposal
|
||||
|
||||
Add a new field `retain_height` to the ABCI `ResponseCommit` message:
|
||||
|
||||
```proto
|
||||
service ABCIApplication {
|
||||
rpc Commit(RequestCommit) returns (ResponseCommit);
|
||||
}
|
||||
|
||||
message RequestCommit {}
|
||||
|
||||
message ResponseCommit {
|
||||
// reserve 1
|
||||
bytes data = 2; // the Merkle root hash
|
||||
uint64 retain_height = 3; // the oldest block height to retain
|
||||
}
|
||||
```
|
||||
|
||||
Upon ABCI `Commit`, which finalizes execution of a block in the state machine, Tendermint removes all data for heights lower than `retain_height`. This allows the state machine to control block retention, which is preferable since only it can determine the significance of historical blocks. By default (i.e. with `retain_height=0`) all historical blocks are retained.
|
||||
|
||||
Removed data includes not only blocks, but also headers, commit info, consensus params, validator sets, and so on. In the first iteration this will be done synchronously, since the number of heights removed for each run is assumed to be small (often 1) in the typical case. It can be made asynchronous at a later time if this is shown to be necessary.
|
||||
|
||||
Since `retain_height` is dynamic, it is possible for it to refer to a height which has already been removed. For example, commit at height 100 may return `retain_height=90` while commit at height 101 may return `retain_height=80`. This is allowed, and will be ignored - it is the application's responsibility to return appropriate values.
|
||||
|
||||
State sync will eventually support backfilling heights, via e.g. a snapshot metadata field `backfill_height`, but in the initial version it will have a fully truncated block history.
|
||||
|
||||
## Cosmos SDK Example
|
||||
|
||||
As an example, we'll consider how the Cosmos SDK might make use of this. The specific details should be discussed in a separate SDK proposal.
|
||||
|
||||
The returned `retain_height` would be the lowest height that satisfies:
|
||||
|
||||
- Unbonding time: the time interval in which validators can be economically punished for misbehavior. Blocks in this interval must be auditable e.g. by the light client.
|
||||
|
||||
- IAVL snapshot interval: the block interval at which the underlying IAVL database is persisted to disk, e.g. every 10000 heights. Blocks since the last IAVL snapshot must be available for replay on application restart.
|
||||
|
||||
- State sync snapshots: blocks since the _oldest_ available snapshot must be available for state sync nodes to catch up (oldest because a node may be restoring an old snapshot while a new snapshot was taken).
|
||||
|
||||
- Local config: archive nodes may want to retain more or all blocks, e.g. via a local config option `min-retain-blocks`. There may also be a need to vary rentention for other nodes, e.g. sentry nodes which do not need historical blocks.
|
||||
|
||||
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Application-specified block retention allows the application to take all relevant factors into account and prevent necessary blocks from being accidentally removed.
|
||||
|
||||
- Node operators can independently decide whether they want to provide complete block histories (if local configuration for this is provided) and snapshots.
|
||||
|
||||
### Negative
|
||||
|
||||
- Social coordination is required to run archival nodes, failure to do so may lead to permanent loss of historical blocks.
|
||||
|
||||
- Social coordination is required to run snapshot nodes, failure to do so may lead to inability to run state sync, and inability to bootstrap new nodes at all if no archival nodes are online.
|
||||
|
||||
### Neutral
|
||||
|
||||
- Reduced block retention requires application changes, and cannot be controlled directly in Tendermint.
|
||||
|
||||
- Application-specified block retention may set a lower bound on disk space requirements for all nodes.
|
||||
|
||||
## References
|
||||
|
||||
- State sync ADR: <https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-053-state-sync-prototype.md>
|
||||
|
||||
- State sync issue: <https://github.com/tendermint/tendermint/issues/828>
|
||||
|
||||
- Block pruning issue: <https://github.com/tendermint/tendermint/issues/3652>
|
||||
@@ -1,82 +0,0 @@
|
||||
# RFC 012: Non-Zero Genesis
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2020-07-26: Initial draft (@erikgrinaker)
|
||||
- 2020-07-28: Use weak chain linking, i.e. `predecessor` field (@erikgrinaker)
|
||||
- 2020-07-31: Drop chain linking (@erikgrinaker)
|
||||
- 2020-08-03: Add `State.InitialHeight` (@erikgrinaker)
|
||||
- 2021-02-11: Migrate RFC to tendermint repo (Originally [RFC 002](https://github.com/tendermint/spec/pull/119))
|
||||
|
||||
## Author(s)
|
||||
|
||||
- Erik Grinaker (@erikgrinaker)
|
||||
|
||||
## Context
|
||||
|
||||
The recommended upgrade path for block protocol-breaking upgrades is currently to hard fork the
|
||||
chain (see e.g. [`cosmoshub-3` upgrade](https://blog.cosmos.network/cosmos-hub-3-upgrade-announcement-39c9da941aee)).
|
||||
This is done by halting all validators at a predetermined height, exporting the application
|
||||
state via application-specific tooling, and creating an entirely new chain using the exported
|
||||
application state.
|
||||
|
||||
As far as Tendermint is concerned, the upgraded chain is a completely separate chain, with e.g.
|
||||
a new chain ID and genesis file. Notably, the new chain starts at height 1, and has none of the
|
||||
old chain's block history. This causes problems for integrators, e.g. coin exchanges and
|
||||
wallets, that assume a monotonically increasing height for a given blockchain. Users also find
|
||||
it confusing that a given height can now refer to distinct states depending on the chain
|
||||
version.
|
||||
|
||||
An ideal solution would be to always retain block backwards compatibility in such a way that chain
|
||||
history is never lost on upgrades. However, this may require a significant amount of engineering
|
||||
work that is not viable for the planned Stargate release (Tendermint 0.34), and may prove too
|
||||
restrictive for future development.
|
||||
|
||||
As a first step, allowing the new chain to start from an initial height specified in the genesis
|
||||
file would at least provide monotonically increasing heights. There was a proposal to include the
|
||||
last block header of the previous chain as well, but since the genesis file is not verified and
|
||||
hashed (only specific fields are) this would not be trustworthy.
|
||||
|
||||
External tooling will be required to map historical heights onto e.g. archive nodes that contain
|
||||
blocks from previous chain version. Tendermint will not include any such functionality.
|
||||
|
||||
## Proposal
|
||||
|
||||
Tendermint will allow chains to start from an arbitrary initial height:
|
||||
|
||||
- A new field `initial_height` is added to the genesis file, defaulting to `1`. It can be set to any
|
||||
non-negative integer, and `0` is considered equivalent to `1`.
|
||||
|
||||
- A new field `InitialHeight` is added to the ABCI `RequestInitChain` message, with the same value
|
||||
and semantics as the genesis field.
|
||||
|
||||
- A new field `InitialHeight` is added to the `state.State` struct, where `0` is considered invalid.
|
||||
Including the field here simplifies implementation, since the genesis value does not have to be
|
||||
propagated throughout the code base separately, but it is not strictly necessary.
|
||||
|
||||
ABCI applications may have to be updated to handle arbitrary initial heights, otherwise the initial
|
||||
block may fail.
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Heights can be unique throughout the history of a "logical" chain, across hard fork upgrades.
|
||||
|
||||
### Negative
|
||||
|
||||
- Upgrades still cause loss of block history.
|
||||
|
||||
- Integrators will have to map height ranges to specific archive nodes/networks to query history.
|
||||
|
||||
### Neutral
|
||||
|
||||
- There is no explicit link to the last block of the previous chain.
|
||||
|
||||
## References
|
||||
|
||||
- [#2543: Allow genesis file to start from non-zero height w/ prev block header](https://github.com/tendermint/tendermint/issues/2543)
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC 006: Semantic Versioning
|
||||
# RFC 012: Semantic Versioning
|
||||
|
||||
## Changelog
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
# RFC 013: Ed25519 Verification
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2020-08-21: Initial RFC
|
||||
- 2021-02-11: Migrate RFC to tendermint repo (Originally [RFC 003](https://github.com/tendermint/spec/pull/144))
|
||||
|
||||
## Author(s)
|
||||
|
||||
- Marko (@marbar3778)
|
||||
|
||||
## Context
|
||||
|
||||
Ed25519 keys are the only supported key types for Tendermint validators currently. Tendermint-Go wraps the ed25519 key implementation from the go standard library. As more clients are implemented to communicate with the canonical Tendermint implementation (Tendermint-Go) different implementations of ed25519 will be used. Due to [RFC 8032](https://www.rfc-editor.org/rfc/rfc8032.html) not guaranteeing implementation compatibility, Tendermint clients must to come to an agreement of how to guarantee implementation compatibility. [Zcash](https://z.cash/) has multiple implementations of their client and have identified this as a problem as well. The team at Zcash has made a proposal to address this issue, [Zcash improvement proposal 215](https://zips.z.cash/zip-0215).
|
||||
|
||||
## Proposal
|
||||
|
||||
- Tendermint-Go would adopt [hdevalence/ed25519consensus](https://github.com/hdevalence/ed25519consensus).
|
||||
- This library is implements `ed25519.Verify()` in accordance to zip-215. Tendermint-go will continue to use `crypto/ed25519` for signing and key generation.
|
||||
|
||||
- Tendermint-rs would adopt [ed25519-zebra](https://github.com/ZcashFoundation/ed25519-zebra)
|
||||
- related [issue](https://github.com/informalsystems/tendermint-rs/issues/355)
|
||||
|
||||
Signature verification is one of the major bottlenecks of Tendermint-go, batch verification can not be used unless it has the same consensus rules, ZIP 215 makes verification safe in consensus critical areas.
|
||||
|
||||
This change constitutes a breaking changes, therefore must be done in a major release. No changes to validator keys or operations will be needed for this change to be enabled.
|
||||
|
||||
This change has no impact on signature aggregation. To enable this signature aggregation Tendermint will have to use different signature schema (Schnorr, BLS, ...). Secondly, this change will enable safe batch verification for the Tendermint-Go client. Batch verification for the rust client is already supported in the library being used.
|
||||
|
||||
As part of the acceptance of this proposal it would be best to contract or discuss with a third party the process of conducting a security review of the go library.
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Consistent signature verification across implementations
|
||||
- Enable safe batch verification
|
||||
|
||||
### Negative
|
||||
|
||||
#### Tendermint-Go
|
||||
|
||||
- Third_party dependency
|
||||
- library has not gone through a security review.
|
||||
- unclear maintenance schedule
|
||||
- Fragmentation of the ed25519 key for the go implementation, verification is done using a third party library while the rest
|
||||
uses the go standard library
|
||||
|
||||
### Neutral
|
||||
|
||||
## References
|
||||
|
||||
[It’s 255:19AM. Do you know what your validation criteria are?](https://hdevalence.ca/blog/2020-10-04-its-25519am)
|
||||
@@ -1,203 +0,0 @@
|
||||
# RFC 015: ReverseSync - fetching historical data
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2021-02-11: Migrate RFC to tendermint repo (Originally [RFC 005](https://github.com/tendermint/spec/pull/224))
|
||||
- 2021-04-19: Use P2P to gossip necessary data for reverse sync.
|
||||
- 2021-03-03: Simplify proposal to the state sync case.
|
||||
- 2021-02-17: Add notes on asynchronicity of processes.
|
||||
- 2020-12-10: Rename backfill blocks to reverse sync.
|
||||
- 2020-11-25: Initial draft.
|
||||
|
||||
## Author(s)
|
||||
|
||||
- Callum Waters (@cmwaters)
|
||||
|
||||
## Context
|
||||
|
||||
Two new features: [Block pruning](https://github.com/tendermint/tendermint/issues/3652)
|
||||
and [State sync](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-042-state-sync.md)
|
||||
meant nodes no longer needed a complete history of the blockchain. This
|
||||
introduced some challenges of its own which were covered and subsequently
|
||||
tackled with [RFC-001](https://github.com/tendermint/spec/blob/master/rfc/001-block-retention.md).
|
||||
The RFC allowed applications to set a block retention height; an upper bound on
|
||||
what blocks would be pruned. However nodes who state sync past this upper bound
|
||||
(which is necessary as snapshots must be saved within the trusting period for
|
||||
the assisting light client to verify) have no means of backfilling the blocks
|
||||
to meet the retention limit. This could be a problem as nodes who state sync and
|
||||
then eventually switch to consensus (or fast sync) may not have the block and
|
||||
validator history to verify evidence causing them to panic if they see 2/3
|
||||
commit on what the node believes to be an invalid block.
|
||||
|
||||
Thus, this RFC sets out to instil a minimum block history invariant amongst
|
||||
honest nodes.
|
||||
|
||||
## Proposal
|
||||
|
||||
A backfill mechanism can simply be defined as an algorithm for fetching,
|
||||
verifying and storing, headers and validator sets of a height prior to the
|
||||
current base of the node's blockchain. In matching the terminology used for
|
||||
other data retrieving protocols (i.e. fast sync and state sync), we
|
||||
call this method **ReverseSync**.
|
||||
|
||||
We will define the mechanism in four sections:
|
||||
|
||||
- Usage
|
||||
- Design
|
||||
- Verification
|
||||
- Termination
|
||||
|
||||
### Usage
|
||||
|
||||
For now, we focus purely on the case of a state syncing node, whom after
|
||||
syncing to a height will need to verify historical data in order to be capable
|
||||
of processing new blocks. We can denote the earliest height that the node will
|
||||
need to verify and store in order to be able to verify any evidence that might
|
||||
arise as the `max_historical_height`/`time`. Both height and time are necessary
|
||||
as this maps to the BFT time used for evidence expiration. After acquiring
|
||||
`State`, we calculate these parameters as:
|
||||
|
||||
```go
|
||||
max_historical_height = max(state.InitialHeight, state.LastBlockHeight - state.ConsensusParams.EvidenceAgeHeight)
|
||||
max_historical_time = max(GenesisTime, state.LastBlockTime.Sub(state.ConsensusParams.EvidenceAgeTime))
|
||||
```
|
||||
|
||||
Before starting either fast sync or consensus, we then run the following
|
||||
synchronous process:
|
||||
|
||||
```go
|
||||
func ReverseSync(max_historical_height int64, max_historical_time time.Time) error
|
||||
```
|
||||
|
||||
Where we fetch and verify blocks until a block `A` where
|
||||
`A.Height <= max_historical_height` and `A.Time <= max_historical_time`.
|
||||
|
||||
Upon successfully reverse syncing, a node can now safely continue. As this
|
||||
feature is only used as part of state sync, one can think of this as merely an
|
||||
extension to it.
|
||||
|
||||
In the future we may want to extend this functionality to allow nodes to fetch
|
||||
historical blocks for reasons of accountability or data accessibility.
|
||||
|
||||
### Design
|
||||
|
||||
This section will provide a high level overview of some of the more important
|
||||
characteristics of the design, saving the more tedious details as an ADR.
|
||||
|
||||
#### P2P
|
||||
|
||||
Implementation of this RFC will require the addition of a new channel and two
|
||||
new messages.
|
||||
|
||||
```proto
|
||||
message LightBlockRequest {
|
||||
uint64 height = 1;
|
||||
}
|
||||
```
|
||||
|
||||
```proto
|
||||
message LightBlockResponse {
|
||||
Header header = 1;
|
||||
Commit commit = 2;
|
||||
ValidatorSet validator_set = 3;
|
||||
}
|
||||
```
|
||||
|
||||
The P2P path may also enable P2P networked light clients and a state sync that
|
||||
also doesn't need to rely on RPC.
|
||||
|
||||
### Verification
|
||||
|
||||
ReverseSync is used to fetch the following data structures:
|
||||
|
||||
- `Header`
|
||||
- `Commit`
|
||||
- `ValidatorSet`
|
||||
|
||||
Nodes will also need to be able to verify these. This can be achieved by first
|
||||
retrieving the header at the base height from the block store. From this trusted
|
||||
header, the node hashes each of the three data structures and checks that they are correct.
|
||||
|
||||
1. The trusted header's last block ID matches the hash of the new header
|
||||
|
||||
```go
|
||||
header[height].LastBlockID == hash(header[height-1])
|
||||
```
|
||||
|
||||
2. The trusted header's last commit hash matches the hash of the new commit
|
||||
|
||||
```go
|
||||
header[height].LastCommitHash == hash(commit[height-1])
|
||||
```
|
||||
|
||||
3. Given that the node now trusts the new header, check that the header's validator set
|
||||
hash matches the hash of the validator set
|
||||
|
||||
```go
|
||||
header[height-1].ValidatorsHash == hash(validatorSet[height-1])
|
||||
```
|
||||
|
||||
### Termination
|
||||
|
||||
ReverseSync draws a lot of parallels with fast sync. An important consideration
|
||||
for fast sync that also extends to ReverseSync is termination. ReverseSync will
|
||||
finish it's task when one of the following conditions have been met:
|
||||
|
||||
1. It reaches a block `A` where `A.Height <= max_historical_height` and
|
||||
`A.Time <= max_historical_time`.
|
||||
2. None of it's peers reports to have the block at the height below the
|
||||
processes current block.
|
||||
3. A global timeout.
|
||||
|
||||
This implies that we can't guarantee adequate history and thus the term
|
||||
"invariant" can't be used in the strictest sense. In the case that the first
|
||||
condition isn't met, the node will log an error and optimistically attempt
|
||||
to continue with either fast sync or consensus.
|
||||
|
||||
## Alternative Solutions
|
||||
|
||||
The need for a minimum block history invariant stems purely from the need to
|
||||
validate evidence (although there may be some application relevant needs as
|
||||
well). Because of this, an alternative, could be to simply trust whatever the
|
||||
2/3+ majority has agreed upon and in the case where a node is at the head of the
|
||||
blockchain, you simply abstain from voting.
|
||||
|
||||
As it stands, if 2/3+ vote on evidence you can't verify, in the same manner if
|
||||
2/3+ vote on a header that a node sees as invalid (perhaps due to a different
|
||||
app hash), the node will halt.
|
||||
|
||||
Another alternative is the method with which the relevant data is retrieved.
|
||||
Instead of introducing new messages to the P2P layer, RPC could have been used
|
||||
instead.
|
||||
|
||||
The aforementioned data is already available via the following RPC endpoints:
|
||||
`/commit` for `Header`'s' and `/validators` for `ValidatorSet`'s'. It was
|
||||
decided predominantly due to the instability of the current RPC infrastructure
|
||||
that P2P be used instead.
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Ensures a minimum block history invariant for honest nodes. This will allow
|
||||
nodes to verify evidence.
|
||||
|
||||
### Negative
|
||||
|
||||
- Statesync will be slower as more processing is required.
|
||||
|
||||
### Neutral
|
||||
|
||||
- By having validator sets served through p2p, this would make it easier to
|
||||
extend p2p support to light clients and state sync.
|
||||
- In the future, it may also be possible to extend this feature to allow for
|
||||
nodes to freely fetch and verify prior blocks
|
||||
|
||||
## References
|
||||
|
||||
- [RFC-001: Block retention](https://github.com/tendermint/spec/blob/master/rfc/001-block-retention.md)
|
||||
- [Original issue](https://github.com/tendermint/tendermint/issues/4629)
|
||||
Reference in New Issue
Block a user