mirror of
https://github.com/tendermint/tendermint.git
synced 2026-09-20 06:54:41 +00:00
docs: add reactor sections (#6510)
This commit is contained in:
Marko
@@ -0,0 +1,71 @@
|
||||
---
|
||||
order: 1
|
||||
parent:
|
||||
title: Block Sync
|
||||
order: 6
|
||||
---
|
||||
|
||||
|
||||
# Block Sync
|
||||
*Formerly known as Fast Sync*
|
||||
|
||||
In a proof of work blockchain, syncing with the chain is the same
|
||||
process as staying up-to-date with the consensus: download blocks, and
|
||||
look for the one with the most total work. In proof-of-stake, the
|
||||
consensus process is more complex, as it involves rounds of
|
||||
communication between the nodes to determine what block should be
|
||||
committed next. Using this process to sync up with the blockchain from
|
||||
scratch can take a very long time. It's much faster to just download
|
||||
blocks and check the merkle tree of validators than to run the real-time
|
||||
consensus gossip protocol.
|
||||
|
||||
## Using Block Sync
|
||||
|
||||
To support faster syncing, Tendermint offers a `blocksync` mode, which
|
||||
is enabled by default, and can be toggled in the `config.toml` or via
|
||||
`--blocksync.enable=false`.
|
||||
|
||||
In this mode, the Tendermint daemon will sync hundreds of times faster
|
||||
than if it used the real-time consensus process. Once caught up, the
|
||||
daemon will switch out of Block Sync and into the normal consensus mode.
|
||||
After running for some time, the node is considered `caught up` if it
|
||||
has at least one peer and it's height is at least as high as the max
|
||||
reported peer height. See [the IsCaughtUp
|
||||
method](https://github.com/tendermint/tendermint/blob/b467515719e686e4678e6da4e102f32a491b85a0/blockchain/pool.go#L128).
|
||||
|
||||
Note: There are multiple versions of Block Sync. Please use v0 as the other versions are no longer supported.
|
||||
If you would like to use a different version you can do so by changing the version in the `config.toml`:
|
||||
|
||||
```toml
|
||||
#######################################################
|
||||
### Block Sync Configuration Connections ###
|
||||
#######################################################
|
||||
[blocksync]
|
||||
|
||||
# If this node is many blocks behind the tip of the chain, BlockSync
|
||||
# allows them to catchup quickly by downloading blocks in parallel
|
||||
# and verifying their commits
|
||||
enable = true
|
||||
|
||||
# Block Sync version to use:
|
||||
# 1) "v0" (default) - the standard Block Sync implementation
|
||||
# 2) "v2" - DEPRECATED, please use v0
|
||||
version = "v0"
|
||||
```
|
||||
|
||||
If we're lagging sufficiently, we should go back to block syncing, but
|
||||
this is an [open issue](https://github.com/tendermint/tendermint/issues/129).
|
||||
|
||||
## The Block Sync event
|
||||
When the tendermint blockchain core launches, it might switch to the `block-sync`
|
||||
mode to catch up the states to the current network best height. the core will emits
|
||||
a fast-sync event to expose the current status and the sync height. Once it catched
|
||||
the network best height, it will switches to the state sync mechanism and then emit
|
||||
another event for exposing the fast-sync `complete` status and the state `height`.
|
||||
|
||||
The user can query the events by subscribing `EventQueryBlockSyncStatus`
|
||||
Please check [types](https://pkg.go.dev/github.com/tendermint/tendermint/types?utm_source=godoc#pkg-constants) for the details.
|
||||
|
||||
## Implementation
|
||||
|
||||
To read more on the implamentation please see the [reactor doc](./reactor.md) and the [implementation doc](./implementation.md)
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 265 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 43 KiB |
@@ -0,0 +1,47 @@
|
||||
---
|
||||
order: 3
|
||||
---
|
||||
|
||||
# Implementation
|
||||
|
||||
## Blocksync Reactor
|
||||
|
||||
- coordinates the pool for syncing
|
||||
- coordinates the store for persistence
|
||||
- coordinates the playing of blocks towards the app using a sm.BlockExecutor
|
||||
- handles switching between fastsync and consensus
|
||||
- it is a p2p.BaseReactor
|
||||
- starts the pool.Start() and its poolRoutine()
|
||||
- registers all the concrete types and interfaces for serialisation
|
||||
|
||||
### poolRoutine
|
||||
|
||||
- listens to these channels:
|
||||
- pool requests blocks from a specific peer by posting to requestsCh, block reactor then sends
|
||||
a &bcBlockRequestMessage for a specific height
|
||||
- pool signals timeout of a specific peer by posting to timeoutsCh
|
||||
- switchToConsensusTicker to periodically try and switch to consensus
|
||||
- trySyncTicker to periodically check if we have fallen behind and then catch-up sync
|
||||
- if there aren't any new blocks available on the pool it skips syncing
|
||||
- tries to sync the app by taking downloaded blocks from the pool, gives them to the app and stores
|
||||
them on disk
|
||||
- implements Receive which is called by the switch/peer
|
||||
- calls AddBlock on the pool when it receives a new block from a peer
|
||||
|
||||
## Block Pool
|
||||
|
||||
- responsible for downloading blocks from peers
|
||||
- makeRequestersRoutine()
|
||||
- removes timeout peers
|
||||
- starts new requesters by calling makeNextRequester()
|
||||
- requestRoutine():
|
||||
- picks a peer and sends the request, then blocks until:
|
||||
- pool is stopped by listening to pool.Quit
|
||||
- requester is stopped by listening to Quit
|
||||
- request is redone
|
||||
- we receive a block
|
||||
- gotBlockCh is strange
|
||||
|
||||
## Go Routines in Blocksync Reactor
|
||||
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
---
|
||||
order: 2
|
||||
---
|
||||
# Reactor
|
||||
|
||||
The Blocksync Reactor's high level responsibility is to enable peers who are
|
||||
far behind the current state of the consensus to quickly catch up by downloading
|
||||
many blocks in parallel, verifying their commits, and executing them against the
|
||||
ABCI application.
|
||||
|
||||
Tendermint full nodes run the Blocksync Reactor as a service to provide blocks
|
||||
to new nodes. New nodes run the Blocksync Reactor in "fast_sync" mode,
|
||||
where they actively make requests for more blocks until they sync up.
|
||||
Once caught up, "fast_sync" mode is disabled and the node switches to
|
||||
using (and turns on) the Consensus Reactor.
|
||||
|
||||
## Architecture and algorithm
|
||||
|
||||
The Blocksync reactor is organised as a set of concurrent tasks:
|
||||
|
||||
- Receive routine of Blocksync Reactor
|
||||
- Task for creating Requesters
|
||||
- Set of Requesters tasks and - Controller task.
|
||||
|
||||
|
||||
|
||||
### Data structures
|
||||
|
||||
These are the core data structures necessarily to provide the Blocksync Reactor logic.
|
||||
|
||||
Requester data structure is used to track assignment of request for `block` at position `height` to a peer with id equals to `peerID`.
|
||||
|
||||
```go
|
||||
type Requester {
|
||||
mtx Mutex
|
||||
block Block
|
||||
height int64
|
||||
peerID p2p.ID
|
||||
redoChannel chan p2p.ID //redo may send multi-time; peerId is used to identify repeat
|
||||
}
|
||||
```
|
||||
|
||||
Pool is a core data structure that stores last executed block (`height`), assignment of requests to peers (`requesters`), current height for each peer and number of pending requests for each peer (`peers`), maximum peer height, etc.
|
||||
|
||||
```go
|
||||
type Pool {
|
||||
mtx Mutex
|
||||
requesters map[int64]*Requester
|
||||
height int64
|
||||
peers map[p2p.ID]*Peer
|
||||
maxPeerHeight int64
|
||||
numPending int32
|
||||
store BlockStore
|
||||
requestsChannel chan<- BlockRequest
|
||||
errorsChannel chan<- peerError
|
||||
}
|
||||
```
|
||||
|
||||
Peer data structure stores for each peer current `height` and number of pending requests sent to the peer (`numPending`), etc.
|
||||
|
||||
```go
|
||||
type Peer struct {
|
||||
id p2p.ID
|
||||
height int64
|
||||
numPending int32
|
||||
timeout *time.Timer
|
||||
didTimeout bool
|
||||
}
|
||||
```
|
||||
|
||||
BlockRequest is internal data structure used to denote current mapping of request for a block at some `height` to a peer (`PeerID`).
|
||||
|
||||
```go
|
||||
type BlockRequest {
|
||||
Height int64
|
||||
PeerID p2p.ID
|
||||
}
|
||||
```
|
||||
|
||||
### Receive routine of Blocksync Reactor
|
||||
|
||||
It is executed upon message reception on the BlocksyncChannel inside p2p receive routine. There is a separate p2p receive routine (and therefore receive routine of the Blocksync Reactor) executed for each peer. Note that try to send will not block (returns immediately) if outgoing buffer is full.
|
||||
|
||||
```go
|
||||
handleMsg(pool, m):
|
||||
upon receiving bcBlockRequestMessage m from peer p:
|
||||
block = load block for height m.Height from pool.store
|
||||
if block != nil then
|
||||
try to send BlockResponseMessage(block) to p
|
||||
else
|
||||
try to send bcNoBlockResponseMessage(m.Height) to p
|
||||
|
||||
upon receiving bcBlockResponseMessage m from peer p:
|
||||
pool.mtx.Lock()
|
||||
requester = pool.requesters[m.Height]
|
||||
if requester == nil then
|
||||
error("peer sent us a block we didn't expect")
|
||||
continue
|
||||
|
||||
if requester.block == nil and requester.peerID == p then
|
||||
requester.block = m
|
||||
pool.numPending -= 1 // atomic decrement
|
||||
peer = pool.peers[p]
|
||||
if peer != nil then
|
||||
peer.numPending--
|
||||
if peer.numPending == 0 then
|
||||
peer.timeout.Stop()
|
||||
// NOTE: we don't send Quit signal to the corresponding requester task!
|
||||
else
|
||||
trigger peer timeout to expire after peerTimeout
|
||||
pool.mtx.Unlock()
|
||||
|
||||
|
||||
upon receiving bcStatusRequestMessage m from peer p:
|
||||
try to send bcStatusResponseMessage(pool.store.Height)
|
||||
|
||||
upon receiving bcStatusResponseMessage m from peer p:
|
||||
pool.mtx.Lock()
|
||||
peer = pool.peers[p]
|
||||
if peer != nil then
|
||||
peer.height = m.height
|
||||
else
|
||||
peer = create new Peer data structure with id = p and height = m.Height
|
||||
pool.peers[p] = peer
|
||||
|
||||
if m.Height > pool.maxPeerHeight then
|
||||
pool.maxPeerHeight = m.Height
|
||||
pool.mtx.Unlock()
|
||||
|
||||
onTimeout(p):
|
||||
send error message to pool error channel
|
||||
peer = pool.peers[p]
|
||||
peer.didTimeout = true
|
||||
```
|
||||
|
||||
### Requester tasks
|
||||
|
||||
Requester task is responsible for fetching a single block at position `height`.
|
||||
|
||||
```go
|
||||
fetchBlock(height, pool):
|
||||
while true do {
|
||||
peerID = nil
|
||||
block = nil
|
||||
peer = pickAvailablePeer(height)
|
||||
peerID = peer.id
|
||||
|
||||
enqueue BlockRequest(height, peerID) to pool.requestsChannel
|
||||
redo = false
|
||||
while !redo do
|
||||
select {
|
||||
upon receiving Quit message do
|
||||
return
|
||||
upon receiving redo message with id on redoChannel do
|
||||
if peerID == id {
|
||||
mtx.Lock()
|
||||
pool.numPending++
|
||||
redo = true
|
||||
mtx.UnLock()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pickAvailablePeer(height):
|
||||
selectedPeer = nil
|
||||
while selectedPeer = nil do
|
||||
pool.mtx.Lock()
|
||||
for each peer in pool.peers do
|
||||
if !peer.didTimeout and peer.numPending < maxPendingRequestsPerPeer and peer.height >= height then
|
||||
peer.numPending++
|
||||
selectedPeer = peer
|
||||
break
|
||||
pool.mtx.Unlock()
|
||||
|
||||
if selectedPeer = nil then
|
||||
sleep requestIntervalMS
|
||||
|
||||
return selectedPeer
|
||||
```
|
||||
|
||||
sleep for requestIntervalMS
|
||||
|
||||
### Task for creating Requesters
|
||||
|
||||
This task is responsible for continuously creating and starting Requester tasks.
|
||||
|
||||
```go
|
||||
createRequesters(pool):
|
||||
while true do
|
||||
if !pool.isRunning then break
|
||||
if pool.numPending < maxPendingRequests or size(pool.requesters) < maxTotalRequesters then
|
||||
pool.mtx.Lock()
|
||||
nextHeight = pool.height + size(pool.requesters)
|
||||
requester = create new requester for height nextHeight
|
||||
pool.requesters[nextHeight] = requester
|
||||
pool.numPending += 1 // atomic increment
|
||||
start requester task
|
||||
pool.mtx.Unlock()
|
||||
else
|
||||
sleep requestIntervalMS
|
||||
pool.mtx.Lock()
|
||||
for each peer in pool.peers do
|
||||
if !peer.didTimeout && peer.numPending > 0 && peer.curRate < minRecvRate then
|
||||
send error on pool error channel
|
||||
peer.didTimeout = true
|
||||
if peer.didTimeout then
|
||||
for each requester in pool.requesters do
|
||||
if requester.getPeerID() == peer then
|
||||
enqueue msg on requestor's redoChannel
|
||||
delete(pool.peers, peerID)
|
||||
pool.mtx.Unlock()
|
||||
```
|
||||
|
||||
### Main blocksync reactor controller task
|
||||
|
||||
```go
|
||||
main(pool):
|
||||
create trySyncTicker with interval trySyncIntervalMS
|
||||
create statusUpdateTicker with interval statusUpdateIntervalSeconds
|
||||
create switchToConsensusTicker with interval switchToConsensusIntervalSeconds
|
||||
|
||||
while true do
|
||||
select {
|
||||
upon receiving BlockRequest(Height, Peer) on pool.requestsChannel:
|
||||
try to send bcBlockRequestMessage(Height) to Peer
|
||||
|
||||
upon receiving error(peer) on errorsChannel:
|
||||
stop peer for error
|
||||
|
||||
upon receiving message on statusUpdateTickerChannel:
|
||||
broadcast bcStatusRequestMessage(bcR.store.Height) // message sent in a separate routine
|
||||
|
||||
upon receiving message on switchToConsensusTickerChannel:
|
||||
pool.mtx.Lock()
|
||||
receivedBlockOrTimedOut = pool.height > 0 || (time.Now() - pool.startTime) > 5 Seconds
|
||||
ourChainIsLongestAmongPeers = pool.maxPeerHeight == 0 || pool.height >= pool.maxPeerHeight
|
||||
haveSomePeers = size of pool.peers > 0
|
||||
pool.mtx.Unlock()
|
||||
if haveSomePeers && receivedBlockOrTimedOut && ourChainIsLongestAmongPeers then
|
||||
switch to consensus mode
|
||||
|
||||
upon receiving message on trySyncTickerChannel:
|
||||
for i = 0; i < 10; i++ do
|
||||
pool.mtx.Lock()
|
||||
firstBlock = pool.requesters[pool.height].block
|
||||
secondBlock = pool.requesters[pool.height].block
|
||||
if firstBlock == nil or secondBlock == nil then continue
|
||||
pool.mtx.Unlock()
|
||||
verify firstBlock using LastCommit from secondBlock
|
||||
if verification failed
|
||||
pool.mtx.Lock()
|
||||
peerID = pool.requesters[pool.height].peerID
|
||||
redoRequestsForPeer(peerId)
|
||||
delete(pool.peers, peerID)
|
||||
stop peer peerID for error
|
||||
pool.mtx.Unlock()
|
||||
else
|
||||
delete(pool.requesters, pool.height)
|
||||
save firstBlock to store
|
||||
pool.height++
|
||||
execute firstBlock
|
||||
}
|
||||
|
||||
redoRequestsForPeer(pool, peerId):
|
||||
for each requester in pool.requesters do
|
||||
if requester.getPeerID() == peerID
|
||||
enqueue msg on redoChannel for requester
|
||||
```
|
||||
|
||||
## Channels
|
||||
|
||||
Defines `maxMsgSize` for the maximum size of incoming messages,
|
||||
`SendQueueCapacity` and `RecvBufferCapacity` for maximum sending and
|
||||
receiving buffers respectively. These are supposed to prevent amplification
|
||||
attacks by setting up the upper limit on how much data we can receive & send to
|
||||
a peer.
|
||||
|
||||
Sending incorrectly encoded data will result in stopping the peer.
|
||||
Reference in New Issue
Block a user