Commit Graph
100 Commits
Author SHA1 Message Date
Alexander SimmerlandGitHub b5b3b85697 Bring back NodeInfo NetAddress form the dead (#3545)
A prior change to address accidental DNS lookups introduced the
SocketAddr on peer, which was then used to add it to the addressbook.
Which in turn swallowed the self reported port of the peer, which is
important on a reconnect. This change revives the NetAddress on NodeInfo
which the Peer carries, but now returns an error to avoid nil
dereferencing another issue observed in the past. Additionally we could
potentially address #3532, yet the original problem statemenf of that
issue stands.

As a drive-by optimisation `MarkAsGood` now takes only a `p2p.ID` which
makes it interface a bit stricter and leaner.
2019-04-12 12:31:02 +02:00
Alexander SimmerlandEthan Buchman e6a55b7be0 consensus: Add ADR for first stage consensus refactor (#2462) 2018-10-04 20:35:35 -04:00
Alexander Simmerl bdd01310a0 p2p: Integrate new Transport
We are swapping the exisiting listener implementation with the newly
introduced Transport and its default implementation MultiplexTransport,
removing a large chunk of old connection setup and handling scattered
over the Peer and Switch code. The Switch requires a Transport now and
handles externally passed Peer filters.
2018-09-18 22:26:43 +02:00
Alexander Simmerl be5d68ea4f p2p: Implement PeerTransport
This is the implementation for the design described in ADR 12[0]. It's
the first step of a larger refactor of the p2p package as tracked in
interface bundling all concerns of low-level connection handling and
isolating the rest of peer lifecycle management from the specifics of
the low-level internet protocols. Even if the swappable implementation
will never be utilised, already the isolation of conn related code in
one place will help with the reasoning about execution path and
addressation of security sensitive issues surfaced through bounty
programs and audits.

We deliberately decided to not have Peer filtering and other management
in the Transport, its sole responsibility is the translation of
connections to Peers, handing those to the caller fully setup. It's the
responsibility of the caller to reject those and or keep track. Peer
filtering will take place in the Switch and can be inspected in a the
following commit.

This changeset additionally is an exercise in clean separation of logic
and other infrastructural concerns like logging and instrumentation. By
leveraging a clean and minimal interface. How this looks can be seen in
a follow-up change.

Design #2069[2]
Refs #2067[3]
Fixes #2047[4]
Fixes #2046[5]

changes:
* describe Transport interface
* implement new default Transport: MultiplexTransport
* test MultiplexTransport with new constraints
* implement ConnSet for concurrent management of net.Conn, synchronous
to PeerSet
* implement and expose duplicate IP filter
* implemnt TransportOption for optional parametirisation

[0] https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-012-peer-transport.md
[1] https://github.com/tendermint/tendermint/issues/2067
[2] https://github.com/tendermint/tendermint/pull/2069
[3] https://github.com/tendermint/tendermint/issues/2067
[4] https://github.com/tendermint/tendermint/issues/2047
[5] https://github.com/tendermint/tendermint/issues/2046
2018-09-18 22:26:43 +02:00
Alexander SimmerlandGitHub 4be6395ee0 Merge pull request #2085 from tendermint/master
Merge 0.23.8 back into develop
2018-07-27 04:21:34 +02:00
Alexander SimmerlandGitHub fd29fd6465 adr: PeerTransport (#2069)
* p2p: Propose PeerTransport ADR
* adr: Set status to in review
* adr: Add high-level decision
* adr: Extend on the idea of guards
* adr: Rework guards into transport specific filters
* adr: Rename to nodeAddr
* adr: Incorporate review
2018-07-27 02:27:19 +02:00
Alexander SimmerlandGitHub bdab37a626 Merge pull request #2054 from tendermint/dev/secret_connection
secret connection update
2018-07-26 00:29:36 +02:00
Alexander SimmerlandGitHub d212292f86 Merge pull request #2066 from tendermint/bucky/merge-master
Bucky/merge master
2018-07-25 17:36:39 +02:00
Alexander SimmerlandGitHub abe5b63059 Merge pull request #2040 from tendermint/2027-socket-pv
fix acceptDeadline
2018-07-24 15:52:45 +02:00
Alexander SimmerlandGitHub 7e7473ad41 Merge pull request #2044 from tendermint/2019-prometheus
do not overwrite metrics provider in node#NewNode
2018-07-24 15:41:44 +02:00
Alexander SimmerlandGitHub 0f2d97dffe Merge pull request #1742 from Liamsi/proto_files
Add Proto files for types.Header (incl. BlockId, Time, PartsSetHeader)
2018-07-20 17:43:25 +02:00
Alexander SimmerlandGitHub ed8714e40c Merge pull request #1965 from tendermint/693-part-2
make Block Header and Data non-pointers
2018-07-20 17:42:42 +02:00
Alexander SimmerlandGitHub 6017d817e5 Merge pull request #2008 from tendermint/1772-rwmutex-cs
use RWMutex for consensus state
2018-07-20 17:24:55 +02:00
Alexander SimmerlandGitHub 63835c0360 Merge pull request #2009 from tendermint/1772-call-validators-with-timeout
make `/status` RPC endpoint resistant to consensus halt
2018-07-20 17:22:16 +02:00
Alexander Simmerl c82c60df11 rpc: Test Validator retrevial timeout 2018-07-20 17:08:55 +02:00
Alexander SimmerlandGitHub 4f7fac8076 Merge pull request #1950 from tendermint/1891-follow-up
remove print statements
2018-07-11 13:04:12 +02:00
Alexander SimmerlandGitHub ce33914f70 Merge pull request #1928 from tendermint/646-decode-msg
Unexport DecodeMessage
2018-07-10 12:50:33 +02:00
Alexander SimmerlandGitHub 1cd7d862fa Merge pull request #1929 from tendermint/1901-changelog-2
Add missing changelog entry
2018-07-09 19:48:08 +02:00
Alexander SimmerlandGitHub 523d29256e Merge pull request #1911 from tendermint/1901-changelog
Add missing changelog entry
2018-07-07 15:14:43 +02:00
Alexander SimmerlandGitHub 397d89e40b Merge pull request #1906 from tendermint/zach/1793-repo-consolidation
clean up abci/libs directories
2018-07-07 11:53:58 +02:00
Alexander SimmerlandGitHub f93798f72d Merge branch 'master' into release/v0.22.1 2018-07-07 11:45:20 +02:00
Alexander SimmerlandGitHub 8d2d94c1c0 Merge pull request #1922 from tendermint/zach/fix-md
docs: md fixes & latest tm-bench/monitor
2018-07-07 11:43:16 +02:00
Alexander SimmerlandGitHub 0e85ae6179 Merge pull request #1918 from tendermint/docs-staging
Re-organize docs into directories
2018-07-06 19:53:15 +02:00
Alexander SimmerlandGitHub 1f215eb875 Merge pull request #1898 from tendermint/1893-err-if-zero-power-val
state: err if 0 power validator is added to the validator set
2018-07-04 23:05:22 +02:00
Alexander SimmerlandGitHub 5cfd8ebaa2 Merge pull request #1899 from tendermint/1839-shutdown-wal-properly
Properly shutdown consensus WAL
2018-07-04 15:23:25 +02:00
Alexander SimmerlandGitHub 99e982669e Merge pull request #1902 from tendermint/1766-wrong-field-access
fix wrong field access in LoadConsensusParams
2018-07-04 15:17:10 +02:00
Alexander Simmerl 48a1f6f9b3 state: Format panics 2018-07-04 14:59:50 +02:00
Alexander SimmerlandGitHub 7a61e8cf9d Merge pull request #1862 from tendermint/zramsay-patch-1
Delete libs/common/array.go
2018-07-03 17:45:06 +02:00
Alexander SimmerlandGitHub 4c373a6e78 Merge pull request #1872 from tendermint/1829-fill-in-missing-fields
overwrite pubkey and address for convenience
2018-07-03 17:44:29 +02:00
Alexander SimmerlandGitHub 2f29aa9605 Merge pull request #1871 from tendermint/1774-remove-init-testnet-script
remove init_testnet script
2018-07-03 17:42:58 +02:00
Alexander SimmerlandGitHub eb77f662f2 Merge pull request #1886 from tendermint/zach/libs
cleanup repo merge artifacts
2018-07-03 17:40:08 +02:00
Alexander SimmerlandGitHub 277e323d4a Merge pull request #1888 from tendermint/zach/abci-spec-location
abci spec should be in one location
2018-07-03 17:38:56 +02:00
Alexander SimmerlandGitHub c3769b88e4 Merge pull request #100 from tendermint/bucky/tm-bench-fixes
Fix tm-bench metrics
2018-06-23 01:58:56 +02:00
Alexander Simmerl 65b479dd65 tm-bench: Improve code shape
* return error so main controls exit states
* formatting
* order imports
2018-06-23 01:57:50 +02:00
Alexander SimmerlandGitHub f62d6651e3 Merge pull request #1781 from tendermint/1740-node-crashes-when-too-many-rpc-connections
limit number of open connections
2018-06-23 01:15:03 +02:00
Alexander Simmerl 0d35d722cb rpc: Break up long lines 2018-06-23 01:01:31 +02:00
Alexander SimmerlandGitHub e39e43f86f Merge pull request #1790 from tendermint/xla/merge-abci-tmlibs
Merge abci
2018-06-23 00:55:31 +02:00
Alexander SimmerlandGitHub e9e00c4db7 Merge pull request #1797 from SamuelSarle/develop
Rename sync_info.syncing to sync_info.catching_up
2018-06-23 00:49:47 +02:00
Alexander Simmerl 01f6009518 abci: Add comment for doc update 2018-06-22 20:36:29 +02:00
Alexander Simmerl 3f34deab90 abci: Remove LICENSE 2018-06-22 20:29:23 +02:00
Alexander Simmerl 693a973997 abci: Remove nested .gitignore 2018-06-22 20:28:54 +02:00
Alexander Simmerl 02e5cbaa07 abci: Remove old repo docs 2018-06-22 20:23:37 +02:00
Alexander Simmerl 77573a1bad ci: Move over abci-cli tests 2018-06-22 20:09:44 +02:00
Alexander Simmerl 3ff3a8ec37 ci: Setup abci in dependency step 2018-06-22 19:29:44 +02:00
Alexander Simmerl a605b66c5a Move abci imports 2018-06-22 07:28:07 +02:00
Alexander SimmerlandAnton Kaliaev 3e1baf68f8 crypto: Rename last traces of go-crypto (#1786)
Follow-up to #1782
2018-06-22 08:24:36 +04:00
Alexander Simmerl 4bee228ba7 Add abci repo 2018-06-22 02:36:54 +02:00
Alexander SimmerlandGitHub 7e3de2027e Merge pull request #1787 from tendermint/dev/hkdfchachapoly_testvector_seal
crypto/hkdfchachapoly: Add testing seal to the test vector
2018-06-22 01:59:35 +02:00
Alexander SimmerlandGitHub 37385cb1d4 Merge pull request #1737 from tendermint/986-monitoring
metrics
2018-06-20 20:43:20 +02:00
Alexander SimmerlandGitHub 19af3e9733 Merge pull request #1738 from tendermint/zarko/add-blockchain-reactor-algorithm-spec
Add algorithm for Blockchain Reactor
2018-06-15 01:47:36 +02:00
Alexander SimmerlandGitHub fa32dc5181 Merge pull request #1739 from tendermint/dev/bump_version
Bump abci version
2018-06-13 01:48:13 +02:00
Alexander SimmerlandGitHub da4d1c2038 Merge pull request #270 from tendermint/xla/update-develop-v0120
Xla/update develop v0120
2018-06-13 01:32:14 +02:00
Alexander Simmerl 1eeebabb0b Update version and changelog 2018-06-13 01:20:21 +02:00
Alexander Simmerl 514065a3c4 Merge branch 'master' into xla/resolve-divergence 2018-06-12 04:50:20 +02:00
Alexander SimmerlandGitHub 260d69f372 Merge pull request #110 from tendermint/bucky/sig-name
SignatureKey -> Signature
2018-06-12 01:57:26 +02:00
Alexander SimmerlandGitHub dfc5aefd5f Merge pull request #1726 from tendermint/1717-panic-in-netaddress
return an error if we fail to parse external IP
2018-06-12 01:08:16 +02:00
Alexander SimmerlandGitHub eb4a8e0e7a Merge pull request #1719 from Slamper/develop
Escape paths in config template
2018-06-12 01:06:22 +02:00
Alexander SimmerlandGitHub 2e0466d1c8 Merge pull request #1722 from tendermint/dev/document_peer_types
Update description of seed and persistent peer nodes
2018-06-09 02:14:48 +02:00
Alexander Simmerl c661a3ec21 Fix race when mutating MConnConfig
Instead of mutating the passed in MConnConfig part of P2PConfig we just
use the default and override the values, the same as before as it was
always the default version. This is yet another good reason to not embed
information and access to config structs in our components and will go
away with the ongoing refactoring in #1325.
2018-06-07 01:09:13 +02:00
Alexander SimmerlandGitHub 1318bd18cd Merge pull request #1680 from Slamper/develop
Return 404 for unknown RPC endpoints
2018-06-05 02:14:16 +02:00
Alexander Simmerl ea896865a7 Collapse PeerConfig into P2PConfig
As both configs are concerned with the p2p packaage and PeerConfig is
only used inside of the package there is no good reason to keep the
couple of fields separate, therefore it is collapsed into the more
general P2PConifg. This is a stepping stone towards a setup where the
components inside of p2p do not have any knowledge about the config.

follow-up to #1325
2018-06-05 02:07:56 +02:00
Alexander Simmerl bf370d36c2 Extract priv_validator into first class package
This is a maintenance change to move the private validator package out
of the types and to a top-level location. There is no good reason to
keep it under the types and it will more clearly coommunicate where
additions related to the privval belong. It leaves the interface and the
mock in types for now as it would introduce circular dependency between
privval and types, this should be resolved eventually.

* mv priv_validator to privval pkg
* use consistent `privval` as import

Follow-up to #1255
2018-06-03 13:51:58 +02:00
Alexander SimmerlandGitHub 0b0290bdb2 Merge pull request #1675 from tendermint/xla/disable-slate-ci
Disable slate step in CI workflow
2018-06-02 15:56:43 +02:00
Alexander Simmerl a4779fdf51 Disable slate step in CI workflow
It's currently breaking for unknown reasons, until fixed we going to
disable it, to not block on it for unrelated PRs.
2018-06-02 15:49:25 +02:00
Alexander SimmerlandGitHub 0e3dc32b3d Merge pull request #1672 from tendermint/xla/store-ci-test-logs
Store CI test logs
2018-06-02 02:44:01 +02:00
Alexander Simmerl d292fa4541 Store CI test logs
For post-mortem introspection it is helpful to have the full logs of
test runs available for download.
2018-06-02 02:33:31 +02:00
Alexander Simmerl 3255c076e5 Remove auth_enc config option
As we didn't hear any voices requesting this feature, we removed the
option to disable it and always have peer connection auth encrypted.

closes #1518
follow-up #1325
2018-06-01 21:07:20 +02:00
Alexander Simmerl 5796e879b9 Introduce option to skip duplicate ip check
In some scenarios like tests we want to disable the guard which prevents
peers connecting from the same ip.

Fixes #1632
Closes #1634
2018-05-30 10:40:22 +02:00
Alexander Simmerl 186d38dd8a Use different loopback addresses for test switch 2018-05-23 02:36:48 +02:00
Alexander Simmerl 01fd102dba Incoporate review feedback 2018-05-23 01:56:03 +02:00
Alexander Simmerl e11f3167ff Fix pex reactor test 2018-05-23 01:35:03 +02:00
Alexander Simmerl 7d98cfd3d6 Test duplicate IP guard in peer set 2018-05-23 01:24:27 +02:00
Alexander Simmerl 4848e88737 Fix persistent peer switch test 2018-05-23 00:24:40 +02:00
Alexander Simmerl 91b6d3f18c Do not set address for self error 2018-05-21 18:47:14 +02:00
Alexander Simmerl 20e9dd0737 Return fake IP even when there is no conn 2018-05-21 17:55:40 +02:00
Alexander Simmerl 7b02b5b66b Add RemoteIP to test implementation 2018-05-21 17:41:34 +02:00
Alexander Simmerl 0cd92a4948 Fix race in test suffix 2018-05-21 17:35:49 +02:00
Alexander Simmerl d596ed1bc2 Let peerConn handle IPs in for tests 2018-05-18 16:27:57 +02:00
Alexander Simmerl b698a9febc Remove double locking in HasIP 2018-05-16 19:21:12 +02:00
Alexander Simmerl c5f45275ec Use remotePeer for test switch 2018-05-16 19:21:12 +02:00
Alexander Simmerl 77f09f5b5e Move to ne.IP 2018-05-16 19:21:12 +02:00
Alexander SimmerlandEthan Buchman d46b9afb79 Simplify WriteFileAtomic
We can make the implementation more robust by adjusting our assumptions
and leverage explicit file modes for syncing. Additionally we going to
assume that we want to clean up and can't really recover if thos
operations (file close and removal) fail.

* utilise file mode for majority of concerns
* improve test coverage by covering more assumptions
* signature parity with ioutil.WriteFile
* always clean up

Replaces #160
2018-03-22 19:36:24 -04:00
Alexander SimmerlandGitHub 79315efd1f Merge pull request #1341 from EugeneChung/develop
Remove unnecessary bytes.Compare() call
2018-03-20 16:27:06 +01:00
Alexander SimmerlandGitHub 5a51a0ba06 Merge pull request #1337 from tendermint/1296-follow-up
Follow up for /health endpoint
2018-03-20 10:36:47 +01:00
Alexander Simmerl 50ae892d5e p2p: Keep reference to connections in test peer
We observed non-deterministic test failures in one of our switch tests,
which would happen if the GC would run between iterations of the accept
loop. As we don't hold any reference to the connection the setup
finalizer might get triggered and therefore the file handle closed. For
the curious check the references on finalizers and the variable scoping
in the spec:

https://groups.google.com/forum/#!topic/golang-nuts/xWkhGJ5PY6c
https://groups.google.com/forum/#!topic/golang-nuts/d8aF4rAob7U/discussion
https://golang.org/ref/spec#Declarations_and_scope

Fixes #1266
2018-03-19 20:35:12 +01:00
Alexander Simmerl bb65f097fd Simplify WriteFileAtomic
We can make the implementation more robust by adjusting our assumptions
and leverage explicit file modes for syncing. Additionally we going to
assume that we want to clean up and can't really recover if thos
operations (file close and removal) fail.

* utilise file mode for majority of concerns
* improve test coverage by covering more assumptions
* signature parity with ioutil.WriteFile
* always clean up

Replaces #160
2018-03-19 09:38:28 +01:00
Alexander SimmerlandAnton Kaliaev 9b9022f8df privVal: Improve SocketClient network code (#1315)
Follow-up to feedback from #1286, this change simplifies the connection
handling in the SocketClient and makes the communication via TCP more
robust. It introduces the tcpTimeoutListener to encapsulate accept and
i/o timeout handling as well as connection keep-alive, this type could
likely be upgraded to handle more fine-grained tuning of the tcp stack
(linger, nodelay, etc.) according to the properties we desire. The same
methods should be applied to the RemoteSigner which will be overhauled
when the priv_val_server is fleshed out.

* require private key
* simplify connect logic
* break out conn upgrades to tcpTimeoutListener
* extend test coverage and simplify component setup
2018-03-16 16:32:17 +04:00
Alexander Simmerl 589781721a Invert privVal socket communication
Follow-up to #1255 aligning with the expectation that the external
signing process connects to the node. The SocketClient will block on
start until one connection has been established, support for multiple
signers connected simultaneously is a planned future extension.

* SocketClient accepts connection
* PrivValSocketServer renamed to RemoteSigner
* extend tests
2018-03-07 12:37:05 +01:00
Alexander Simmerl 2aa77025c3 Fix typo 2018-03-06 19:56:31 +01:00
Alexander Simmerl 8e1856a90a Use builtin panic 2018-03-06 19:56:31 +01:00
Alexander Simmerl ca619c80b6 Stop privVal socket client on node shutdown 2018-03-06 19:56:30 +01:00
Alexander Simmerl 25ff699425 Improve method docs 2018-03-06 19:55:26 +01:00
Alexander Simmerl 879b4c0a2c Use common method to determine file existence 2018-03-06 19:55:26 +01:00
Alexander Simmerl b7ce89e568 Speed up CircleCI builds
To achieve faster feedback cycles for our feature PRs this change
reduces the average buildtime from 35 to ~6min by utilising their new
2.0 offering based on docker and nomad. We make use of parallel build
steps wherever possible so that the duration is determined by the
slowest test suite (p2p).

This is an intermediate step until we move our CI/CD completely
on-premise for more control and added security.
2018-03-06 17:36:44 +01:00
Alexander Simmerl 533ed2a876 adr: Amend decisions for PrivValidator 2018-03-05 17:38:05 +01:00
Alexander SimmerlandAnton Kaliaev ee51ad8e29 Make RPC handler protocol agnostic (#1276) 2018-03-05 19:59:04 +04:00
Alexander Simmerl 8f2703e8b2 Dial seeds directly without potential peers
In order to improve the operator experience we want the node to dial
seeds immediately if there are no peers to connect to. Until now the
routine responsible for ensuring peers are connected to would wait
a random amount of time up to 30s (if not configured otherwise).
2018-03-02 12:55:01 +01:00
Alexander Simmerl 458a40f74e Integrate CodeCov as change acceptance stage
As a rough measure to keep quality up we want to integrate our code
coverage tooling to the point where it is required for changes to be
merged.

Example taken from https://github.com/cosmos/voyager/blob/develop/codecov.yaml
2018-02-27 15:39:28 +01:00
Alexander Simmerl 74d3f7e1fd Integrate private validator socket client
Following ADDR 008 the node will connect to an external
process to handle signing requests. Operation of the external process is
left to the user.

* introduce alias for PrivValidator interface on socket client
* integrate socket client in node
* structure tests
* remove unnecessary flag
2018-02-23 13:58:22 +01:00
Alexander Simmerl a14aab67de Integrate PrivValidator socket server 2018-02-19 19:20:01 +01:00