Commit Graph

93 Commits

Author SHA1 Message Date
Marko
f05788e632 privval: Query validator key (#5876)
## Description

- Query validator key when a remote signer is used. This is supported gRPC remote signing and filePV only. 


Closes: #3009
2021-01-12 10:06:33 +00:00
Marko
09cf0bcb01 privval: add grpc (#5725)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2021-01-06 10:49:30 -08:00
Erik Grinaker
1ccd23ca1d p2p: fix MConnection inbound traffic statistics and rate limiting (#5868)
Fixes #5866. Inbound traffic monitoring (and by extension inbound rate limiting) was inadvertently removed in 660e72a.
2021-01-06 15:38:23 +01:00
Marko
b2d72dce7e e2e: use ed25519 for secretConn (remote signer) (#5678)
## Description

Hardcode ed25519 to dialTCPFn in e2e tests. 

I will backport `DefaultRequestHandler` fixes

This will be replaced when grpc is implemented.
2020-11-17 13:55:23 +00:00
Erik Grinaker
2d0fcf498d privval: duplicate SecretConnection from p2p package (#5672)
This is so that the `privval` package will not be affected when we refactor `p2p` (#5670). We will be migrating to gRPC shortly (#4698).
2020-11-16 18:16:10 +00:00
Anton Kaliaev
af645ac778 privval: reset pingTimer to avoid sending unnecessary pings (#5642)
Refs #5550
2020-11-10 11:43:01 +00:00
Anton Kaliaev
fa522ca323 privval: increase read/write timeout to 5s and calculate ping interval based on it (#5638)
Partially closes #5550
2020-11-10 11:10:12 +00:00
Marko
bf35cc6443 cmd: add support for --key (#5612)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-11-09 15:22:36 +01:00
Marko
cafad28293 privval: make response values non nullable (#5583)
## Description

make response values non nullable in privval

Does this need a changelog for master?

Closes: #5581 

cc @tarcieri
2020-10-28 15:16:38 +00:00
Marko
346aa14db5 fix lint failures with 1.31 (#5489) 2020-10-13 10:22:53 +02:00
Erik Grinaker
250c3aa92e test: add end-to-end testing framework (#5435)
Partial fix for #5291. For details, see [README.md](https://github.com/tendermint/tendermint/blob/erik/e2e-tests/test/e2e/README.md) and [RFC-001](https://github.com/tendermint/tendermint/blob/master/docs/rfc/rfc-001-end-to-end-testing.md).

This only includes a single test case under `test/e2e/tests/`, as a proof of concept - additional test cases will be submitted separately. A randomized testnet generator will also be submitted separately, there a currently just a handful of static testnets under `test/e2e/networks/`. This will eventually replace the current P2P tests and run in CI.
2020-10-05 09:35:01 +00:00
Erik Grinaker
08708046a7 privval: fix ping message encoding (#5441)
Fixes #5371.
2020-10-01 14:46:21 +00:00
Erik Grinaker
b0130b4661 privval: allow passing options to NewSignerDialerEndpoint (#5434)
Required for #5291 to set timeouts for remote signers.
2020-09-30 16:45:46 +00:00
Marko
b8d08b9ef4 lint: add errchecks (#5316)
## Description

Work towards enabling errcheck

ref #5059
2020-09-04 11:58:03 +00:00
Marko
8cdaa7f515 privval: add chainID to requests (#5239)
## Description

Add chainid to requests to privval. This is a non-breaking change and hardware devices can opt to ignore the field.
 
Closes: #4503 

Took the approach of passing chainID to the client instead of modifying `GetPubKey` because it would lead to a larger change throughout the codebase and in some places it could get tricky to get chainID.
2020-08-17 09:07:15 +00:00
Marko
40bd416d59 test: protobuf vectors for reactors (#5221)
## Description

Add test vectors for all reactors

- [x] state-sync
- [x] privval
- [x] mempool
- [x] p2p
- [x] evidence
- [ ] light?

this PR is primarily oriented at testvectors for things going over the wire. should we expand the testvectors into types as well?

Closes: #XXX
2020-08-11 14:00:11 +00:00
Anton Kaliaev
940e68292c privval: if remote signer errors, don't retry (#5140)
Closes #5112
2020-07-22 10:12:37 +04:00
Marko
2ac5a559b4 libs: wrap mutexes for build flag with godeadlock (#5126)
## Description

This PR wraps the stdlib sync.(RW)Mutex & godeadlock.(RW)Mutex. This enables using go-deadlock via a build flag instead of using sed to replace sync with godeadlock in all files

Closes: #3242
2020-07-20 07:55:09 +00:00
Marko
6ccccb0933 lint: errcheck (#5091)
## Description

add more error checks to tests


gonna do a third PR that tackles the non test cases
2020-07-14 11:04:41 +00:00
Marko
dedf0d2350 proto: folder structure adhere to buf (#5025) 2020-06-22 10:00:51 +02:00
Marko
51da4fe356 types: rename partsheader to partsetheader (#5029)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-06-22 09:38:03 +02:00
Ismail Khoffi
4774a8ec61 non breaking signbytes (#5008)
* test-vectors for backwards compatibility:

 - copy & paste test-vectors from v0.33.5 to ensure
 backwards compatibility for vote's SignBytes

* WIP: everything besides time seems to match :-/

* almost

* Found the culprit: field nums weren't consecutive ints ...

* fix order of partset header too

* this last votes-related test can easily be fixed

* some minor changes and fix last failing test

* move proto types back to stdtime, fix various linting

* use libs/protoio

* remvoe commented code

* add comments

* fix tests

* uncomment testscases

* dont ignore error panic

* fix signable test

* fix happy path testing

* fix comment

Co-authored-by: Marko Baricevic <marbar3778@yahoo.com>
2020-06-15 21:54:58 +02:00
Marko
f6243d8b9e privval: migrate to protobuf (#4985) 2020-06-11 11:54:02 +02:00
Erik Grinaker
db8f1b3df3 migrate all JSON to new JSON encoder (#4975)
Uses new JSON encoder in #4955 for all JSON. Branched off of #4968.
2020-06-08 12:22:59 +00:00
Marko
a88537bb88 ints: stricter numbers (#4939) 2020-06-04 16:34:56 +02:00
Marko
7c576f02ab keys: change to []bytes (#4950) 2020-06-04 15:32:42 +02:00
Marko
99d88cbe2f crypto: remove key suffixes (#4941)
## Description

- remove keyname suffix from keys


Closes: #2228
2020-06-03 05:46:29 +00:00
Anton Kaliaev
b7b721c484 change use of errors.Wrap to fmt.Errorf with %w verb
Closes #4603

Commands used (VIM):

```
:args `rg -l errors.Wrap`
:argdo normal @q | update
```

where q is a macros rewriting the `errors.Wrap` to `fmt.Errorf`.
2020-05-12 03:35:47 +00:00
Anton Kaliaev
2afae13a48 privval: retry GetPubKey/SignVote/SignProposal N times before
returning an error

Closes #4707
2020-05-11 13:42:19 +00:00
Anton Kaliaev
f31f4327b5 privval: remove deprecated OldFilePV
The old format was deprecated in v0.28. It's time we remove it.
2020-04-28 08:51:51 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
Marko
48f073d796 privval: return error on getpubkey (#4534)
closes #3602

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-03-12 11:10:36 +04:00
Erik Grinaker
fe11219795 Fix unexported returns (#4450) 2020-02-21 19:21:39 +01:00
Marko
9bd0f9e634 lint: golint issue fixes (#4258)
* lint: golint issue fixes

- on my local machine golint is a lot stricter than the bot so slowly going through and fixing things.

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* more fixes from golint

* remove isPeerPersistentFn

* add changelog entry
2019-12-17 13:02:45 +01:00
Marko
7b52f51700 libs/common: Refactor libs/common 5 (#4240)
* libs/common: Refactor libs/common 5

- move mathematical functions and types out of `libs/common` to math pkg
- move net functions out of `libs/common` to net pkg
- move string functions out of `libs/common` to strings pkg
- move async functions out of `libs/common` to async pkg
- move bit functions out of `libs/common` to bits pkg
- move cmap functions out of `libs/common` to cmap pkg
- move os functions out of `libs/common` to os pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing issues

* fix tests

closes #41417

woooooooooooooooooo kill the cmn pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix goimport issues

* run gofmt
2019-12-12 09:33:27 +01:00
Marko
89f0bbbd76 libs/common: Refactor libs/common 4 (#4237)
* libs/common: Refactor libs/common 4

- move byte function out of cmn to its own pkg
- move tempfile out of cmn to its own pkg
- move throttletimer to its own pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix linting issues
2019-12-11 23:16:35 +01:00
Marko
27b00cf8d1 libs/common: refactor libs common 3 (#4232)
* libs/common: refactor libs common 3

- move nil.go into types folder and make private
- move service & baseservice out of common into service pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry
2019-12-11 09:31:25 +01:00
Marko
afc4d7a61f libs/common: refactor libs/common 2 (#4231)
* libs/common: refactor libs/common 2

- move random function to there own pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* change imports and usage throughout repo

* fix goimports

* add changelog entry
2019-12-10 18:38:34 +01:00
Anton Kaliaev
3e1516b624 linters: enable stylecheck (#4153)
Refs #3262
2019-11-16 19:35:39 +04:00
cosmostuba
4b15b96588 privval: add SignerDialerEndpointRetryWaitInterval option (#4115)
Added a small function to be able to change the default retry interval for the privval. The default is 100ms, this function allows to change to any time.Duration.

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
2019-11-07 17:40:00 +04:00
cosmostuba
0335d07ff6 privval: remove misplaced debug statement (#4103)
Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

There's no debug statement in Read func, so should be OK remove this one too.
2019-11-03 19:48:45 +04:00
Phil Salant
bc572217c0 Fix linter errors thrown by lll (#3970)
* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter
2019-10-17 10:42:28 +02:00
Phil Salant
05075ea5b7 linters: modify code to pass maligned and interfacer (#3959)
* Fix maligned structs

* Fix interfacer errors

* Revert accidental go.mod and go.sum changes

* Revert P2PConfig struct maligned reorder

* Revert PeerRoundState struct maligned reordering

* Revert RoundState struct maligned reordering

* Reorder WSClient struct

* Revert accidental type change

* Clean up type change

* Clean up type changes

* Revert to types.ABCIApplicationServer in GRPCServer struct

* Revert maligned changes to BaseConfig struct

* Fix tests in io_test.go

* Fix client_test package tests

* Fix reactor tests in consensus package

* Fix new interfacer errors
2019-09-30 17:12:51 -07:00
Marko
8a282a5fee replace errors.go with github.com/pkg/errors (2/2) (#3890)
* init of (2/2) common errors

* Remove instances of cmn.Error (2/2)

- Replace usage of cmnError and errorWrap
- ref #3862

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* comment wording

* simplify IsErrXXX functions

* log panic along with stopping the MConnection
2019-08-11 21:03:40 +04:00
Marko
8dc39b69b7 replace errors.go with github.com/pkg/errors (1/2) (#3888)
* (1/2) of replace errors.go with github.com/pkg/errors

ref #3862

- step one in removing instances of errors.go in favor of github.com/pkg/errors

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* gofmt

* add in /store
2019-08-08 14:31:13 +04:00
Juan Leni
f7f034a8be privval: refactor Remote signers (#3370)
This PR is related to #3107 and a continuation of #3351

It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction.

Given two hosts A and B:

    Host A is listener/client
    Host B is dialer/server (contains the secret key)
    When A requires a signature, it needs to wait for B to dial in before it can issue a request.
    A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect.

The original rationale behind this design was based on security.

    Host B only allows outbound connections to a list of whitelisted hosts.
    It is not possible to reach B unless B dials in. There are no listening/open ports in B.

This PR results in the following changes:

    Refactors ping/heartbeat to avoid previously existing race conditions.
    Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow.
    Unifies and abstracts away the differences between unix and tcp sockets.
    A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj)
    The signer request handler (server side) is customizable to increase testability.
    Updates and extends unit tests

A high level overview of the classes is as follows:

Transport (endpoints): The following classes take care of establishing a connection

    SignerDialerEndpoint
    SignerListeningEndpoint
    SignerEndpoint groups common functionality (read/write/timeouts/etc.)

Signing (client/server): The following classes take care of exchanging request/responses

    SignerClient
    SignerServer

This PR also closes #3601

Commits:

* refactoring - work in progress

* reworking unit tests

* Encapsulating and fixing unit tests

* Improve tests

* Clean up

* Fix/improve unit tests

* clean up tests

* Improving service endpoint

* fixing unit test

* fix linter issues

* avoid invalid cache values (improve later?)

* complete implementation

* wip

* improved connection loop

* Improve reconnections + fixing unit tests

* addressing comments

* small formatting changes

* clean up

* Update node/node.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_client.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_client_test.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* check during initialization

* dropping connecting when writing fails

* removing break

* use t.log instead

* unifying and using cmn.GetFreePort()

* review fixes

* reordering and unifying drop connection

* closing instead of signalling

* refactored service loop

* removed superfluous brackets

* GetPubKey can return errors

* Revert "GetPubKey can return errors"

This reverts commit 68c06f19b4.

* adding entry to changelog

* Update CHANGELOG_PENDING.md

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_client.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_dialer_endpoint.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_dialer_endpoint.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_dialer_endpoint.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_dialer_endpoint.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* Update privval/signer_listener_endpoint_test.go

Co-Authored-By: jleni <juan.leni@zondax.ch>

* updating node.go

* review fixes

* fixes linter

* fixing unit test

* small fixes in comments

* addressing review comments

* addressing review comments 2

* reverting suggestion

* Update privval/signer_client_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update privval/signer_client_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update privval/signer_listener_endpoint_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* do not expose brokenSignerDialerEndpoint

* clean up logging

* unifying methods

shorten test time
signer also drops

* reenabling pings

* improving testability + unit test

* fixing go fmt + unit test

* remove unused code

* Addressing review comments

* simplifying connection workflow

* fix linter/go import issue

* using base service quit

* updating comment

* Simplifying design + adjusting names

* fixing linter issues

* refactoring test harness + fixes

* Addressing review comments

* cleaning up

* adding additional error check
2019-08-05 19:09:10 +04:00
Marko
98cb8c9783 add staticcheck linting (#3828)
cleanup to add linter

    grpc change:
        https://godoc.org/google.golang.org/grpc#WithContextDialer
        https://godoc.org/google.golang.org/grpc#WithDialer
        grpc/grpc-go#2627
    prometheous change:
        due to UninstrumentedHandler, being deprecated in the future
    empty branch = empty if or else statement
        didn't delete them entirely but commented
        couldn't find a reason to have them
    could not replicate the issue #3406
        but if want to keep it commented then we should comment out the if statement as well
2019-07-25 09:35:30 +04:00
Marko
362729c2bb Renamed wire.go to codec.go (#3827)
* Renamed wire.go to codec.go

- Wire was the previous name of amino
- Codec describes the file better than `wire` & `amino`

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* ide error

* rename amino.go to codec.go
2019-07-23 15:35:36 +02:00
Marko
866b343c0c Changes to files that had linting issue (#3731)
- Govet issues fixed
- 1 gosec issue solved using nolint

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
2019-06-21 09:58:32 +04:00
Anton Kaliaev
43348022d6 pex: various follow-ups (#3605)
* p2p: merge switch cases

also improve the error msg in privval

* pex: refactor code plus update specification

follow-up to https://github.com/tendermint/tendermint/pull/3603

* Update docs/spec/reactors/pex/pex.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>
2019-05-01 09:38:26 +04:00