The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase.
## Commits:
* Blockchainv 2 reactor:
+ I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase.
+ The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor.
* fixes after merge
* reorder iIO interface methodset
* change iO -> IO
* panic before send nil block
* rename switchToConsensus -> trySwitchToConsensus
* rename tdState -> tmState
* Update blockchain/v2/reactor.go
Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>
* remove peer when it sends a block unsolicited
* check for not ready in markReceived
* fix error
* fix the pcFinished event
* typo fix
* add documentation for processor fields
* simplify time.Since
* try and make the linter happy
* some doc updates
* fix channel diagram
* Update adr-043-blockchain-riri-org.md
* panic on nil switch
* liting fixes
* account for nil block in bBlockResponseMessage
* panic on duplicate block enqueued by processor
* linting
* goimport reactor_test.go
Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>
Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
* Add processor prototype
* Change processor API
+ expose a simple `handle` function which mutates internal state
* schedule event handling
* rename schedule -> scheduler
* fill in handle function
* processor tests
* fix gofmt and ohter golangci issues
* scopelint var on range scope
* add check for short block received
* small test reorg
* ci fix changes
* go.mod revert
* some cleanup and review comments
* scheduler fixes and unit tests, also small processor changes.
changed scPeerPruned to include a list of pruned peers
touchPeer to check peer state and remove the blocks from blockStates if the peer removal causes the max peer height to be lower.
remove the block at sc.initHeight
changed peersInactiveSince, peersSlowerThan, getPeersAtHeight check peer state
prunablePeers to return a sorted list of peers
lastRate in markReceived() attempted to divide by 0, temp fix.
fixed allBlocksProcessed conditions
maxHeight() and minHeight() to return sc.initHeight if no ready peers present
make selectPeer() deterministic.
added handleBlockProcessError()
added termination cond. (sc.allBlocksProcessed()) to handleTryPrunePeer() and others.
changed pcBlockVerificationFailure to include peer of H+2 block along with the one for H+1
changed the processor to call purgePeer on block verification failure.
fixed processor tests
added scheduler tests.
* typo and ci fixes
* remove height from scBlockRequest, golangci fixes
* limit on blockState map, updated tests
* remove unused
* separate test for maxHeight(), used for sched. validation
* use Math.Min
* fix golangci
* Document the semantics of blockStates in the scheduler
* better docs
* distinguish between unknown and invalid blockstate
* Standardize peer filtering methods
* feedback
* s/getPeersAtHeight/getPeersAtHeightOrAbove
* small notes
* Update blockchain/v2/scheduler.go
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* Update comments based on feedback
* Add enum offset
* panic on nil block in processor
* remove unused max height calculation
* format shorter line
* Add processor prototype
* Change processor API
+ expose a simple `handle` function which mutates internal state
* processor tests
* fix gofmt and ohter golangci issues
* scopelint var on range scope
* add check for short block received
* fix formatting
* small test reorg
* ignore unused for now
* ci fix changes
* go.mod revert
+ use `trySend` the replicate peer sending
+ expose `next()` as a chan of events as output
+ expose `final()` as a chan of error, for the final error
+ add `ready()` as chan struct when routine is ready
+ ensure that we stop accepting messages once `stop` has been called
to avoid the case in which we attempt to write to a channel which
has already been closed
+ `routine.send` returns false when routine is not running
+ this will prevent panics sending to channels which have been
closed
+ Make output channels routine specific removing the risk of someone
writting to a channel which was closed by another touine.
+ consistency changes between the routines and the demuxer
* Add Schedule:
+ The schedule is a data structure used to determine the optimal
schedule of requests to the optimal set of peers in order to perform
fast-sync as fast and efficiently as possible.
* Add some doc strings
* fix golangci
* Add Schedule:
+ The schedule is a data structure used to determine the optimal
schedule of requests to the optimal set of peers in order to perform
fast-sync as fast and efficiently as possible.
* Add some doc strings
* remove globals from tests
* [adr] First draft on adr-042
* fix diagram urls
* Update docs/architecture/adr-042-blockchain-riri-org.md
Co-Authored-By: Marko <marbar3778@yahoo.com>
* Update docs/architecture/adr-042-blockchain-riri-org.md
Co-Authored-By: Marko <marbar3778@yahoo.com>
* Update docs/architecture/adr-042-blockchain-riri-org.md
Co-Authored-By: Marko <marbar3778@yahoo.com>
* add go syntax highlight
* more highlighting
* consistency fixes
* Add references
* new adr number
* Fixes based on feedback
* aditional state info
* Add details on getSchedule
* replace spaces with tabs
* fixes based on feedback
* add clarity around r.msgs
* clarify block processing
* fix off by one error
* additional details on ioRoutine
* Update docs/architecture/adr-043-blockchain-riri-org.md
Co-Authored-By: Anca Zamfir <ancazamfir@users.noreply.github.com>
* Move peer behaviour into it's own package
* refactor wip
* Adjust API and fix tests
* remove unused test struct
* Better error message
* Restructure:
+ Now behaviour is it's own package, we don't need to include
PeerBehaviour in every type.
+ Split up behaviours and reporters into seperate files
* doc string fixes
* Fix minor typos
* Update behaviour/reporter.go
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* Update behaviour/reporter.go
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* Peer behaviour test tweaks:
Address the remaining test issues mentioned in ##3552 notably:
+ switch to p2p_test package
+ Use `Error` instead of `Errorf` when not using formatting
+ Add expected/got errors to
`TestMockPeerBehaviourReporterConcurrency` test
* Peer behaviour equal behaviours test
+ slices of PeerBehaviours should be compared as histograms to
ensure they have the same set of PeerBehaviours at the same
freequncy.
* TestEqualPeerBehaviours:
+ Add tests for the equivalence between sets of PeerBehaviours
* p2p: initial implementation of peer behaviour
* [p2p] re-use newMockPeer
* p2p: add inline docs for peer_behaviour interface
* [p2p] Align PeerBehaviour interface (#3558)
* [p2p] make switchedPeerHebaviour private
* [p2p] make storePeerHebaviour private
* [p2p] Add CHANGELOG_PENDING entry for PeerBehaviour
* [p2p] Adjustment naming for PeerBehaviour
* [p2p] Add coarse lock around storedPeerBehaviour
* [p2p]: Fix non-pointer methods in storedPeerBehaviour
+ Structs with embeded locks must specify all methods with pointer
receivers to avoid creating a copy of the embeded lock.
* [p2p] Thorough refactoring based on comments in #3552
+ Decouple PeerBehaviour interface from Peer by parametrizing
methods with `p2p.ID` instead of `p2p.Peer`
+ Setter methods wrapped in a write lock
+ Getter methods wrapped in a read lock
+ Getter methods on storedPeerBehaviour now take a `p2p.ID`
+ Getter methods return a copy of underlying stored behaviours
+ Add doc strings to public types and methods
* [p2p] make structs public
* [p2p] Test empty StoredPeerBehaviour
* [p2p] typo fix
* [p2p] add TestStoredPeerBehaviourConcurrency
+ Add a test which uses StoredPeerBehaviour in multiple goroutines
to ensure thread-safety.
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour_test.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* [p2p] field ordering convention
* p2p: peer behaviour refactor
+ Change naming of reporting behaviour to `Report`
+ Remove the responsibility of distinguishing between the categories
of good and bad behaviour and instead focus on reporting behaviour.
* p2p: rename PeerReporter -> PeerBehaviourReporter
(#2611) had suggested that an iterative version of
SimpleHashFromByteSlice would be faster, presumably because
we can envision some overhead accumulating from stack
frames and function calls. Additionally, a recursive algorithm risks
hitting the stack limit and causing a stack overflow should the tree
be too large.
Provided here is an iterative alternative, a simple test to assert
correctness and a benchmark. On the performance side, there appears to
be no overall difference:
```
BenchmarkSimpleHashAlternatives/recursive-4 20000 77677 ns/op
BenchmarkSimpleHashAlternatives/iterative-4 20000 76802 ns/op
```
On the surface it might seem that the additional overhead is due to
the different allocation patterns of the implementations. The recursive
version uses a single `[][]byte` slices which it then re-slices at each level of the tree.
The iterative version reproduces `[][]byte` once within the function and
then rewrites sub-slices of that array at each level of the tree.
Eexperimenting by modifying the code to simply calculate the
hash and not store the result show little to no difference in performance.
These preliminary results suggest:
1. The performance of the current implementation is pretty good
2. Go has low overhead for recursive functions
3. The performance of the SimpleHashFromByteSlice routine is dominated
by the actual hashing of data
Although this work is in no way exhaustive, point #3 suggests that
optimizations of this routine would need to take an alternative
approach to make significant improvements on the current performance.
Finally, considering that the recursive implementation is easier to
read, it might not be worthwhile to switch to a less intuitive
implementation for so little benefit.
* re-add slice re-writing
* [crypto] Document SimpleHashFromByteSlicesIterative
* docs: fix broken links (#3482)
A bunch of links were broken in the documentation s they included the
`docs` prefix.
* Update CHANGELOG_PENDING
* docs: switch to relative links for github compatitibility (#3482)
* docs: fix broken links (#3482)
A bunch of links were broken in the documentation s they included the
`docs` prefix.
* Update CHANGELOG_PENDING
* docs: switch to relative links for github compatitibility (#3482)
Sean BraithwaiteandGitHub
