Commit Graph

105 Commits

Author SHA1 Message Date
M. J. Fromberger
cf7537ea5f cleanup: Reduce and normalize import path aliasing. (#6975)
The code in the Tendermint repository makes heavy use of import aliasing.
This is made necessary by our extensive reuse of common base package names, and
by repetition of similar names across different subdirectories.

Unfortunately we have not been very consistent about which packages we alias in
various circumstances, and the aliases we use vary. In the spirit of the advice
in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports,
his change makes an effort to clean up and normalize import aliasing.

This change makes no API or behavioral changes. It is a pure cleanup intended
o help make the code more readable to developers (including myself) trying to
understand what is being imported where.

Only unexported names have been modified, and the changes were generated and
applied mechanically with gofmt -r and comby, respecting the lexical and
syntactic rules of Go.  Even so, I did not fix every inconsistency. Where the
changes would be too disruptive, I left it alone.

The principles I followed in this cleanup are:

- Remove aliases that restate the package name.
- Remove aliases where the base package name is unambiguous.
- Move overly-terse abbreviations from the import to the usage site.
- Fix lexical issues (remove underscores, remove capitalization).
- Fix import groupings to more closely match the style guide.
- Group blank (side-effecting) imports and ensure they are commented.
- Add aliases to multiple imports with the same base package name.
2021-09-23 07:52:07 -07:00
M. J. Fromberger
8f06e0c9e7 cleanup: remove redundant error plumbing (#6778)
This is a mostly-automated fixup using Comby (https://comby.dev) to remove 
lexically-obvious redundant error checks. No functional changes are intended.

To reproduce the core change:

    # Collapse redundant error check conditionals
    % comby -in-place 'if err != nil {
       return err
    }
    return nil' 'return err' .go

    # Fold out unnecessary error temporaries
    % comby -in-place ':[spc~^\s*]err :[~:?]= :[any]
       return err' ':[spc]return :[any]' .go

Fixes #6479 and related cases.
2021-07-28 15:38:46 -04:00
JayT106
11a71c228c state/privval: no GetPubKey retry beyond the proposal/voting window (#6578)
Closes #5142
2021-06-28 13:50:25 +00:00
Yawning Angel
c5cc3c8d3f crypto: Use a different library for ed25519/sr25519 (#6526)
At Oasis we have spend some time writing a new Ed25519/X25519/sr25519 implementation called curve25519-voi.  This PR switches the import from ed25519consensus/go-schnorrkel, which should lead to performance gains on most systems.

Summary of changes:
 * curve25519-voi is now used for Ed25519 operations, following the existing ZIP-215 semantics.
 * curve25519-voi's public key cache is enabled (hardcoded size of 4096 entries, should be tuned, see the code comment) to accelerate repeated Ed25519 verification with the same public key(s).
 * (BREAKING) curve25519-voi is now used for sr25519 operations.  This is a breaking change as the current sr25519 support does something decidedly non-standard when going from a MiniSecretKey to a SecretKey and or PublicKey (The expansion routine is called twice).  While I believe the new behavior (that expands once and only once) to be more "correct", this changes the semantics as implemented.
 * curve25519-voi is now used for merlin since the included STROBE implementation produces much less garbage on the heap.

Side issues fixed:
 * The version of go-schnorrkel that is currently imported by tendermint has a badly broken batch verification implementation.  Upstream has fixed the issue after I reported it, so the version should be bumped in the interim.

Open design questions/issues:
 * As noted, the public key cache size should be tuned.  It is currently backed by a trivial thread-safe LRU cache, which is not scan-resistant, but replacing it with something better is a matter of implementing an interface.
 * As far as I can tell, the only reason why serial verification on batch failure is necessary is to provide more detailed error messages (that are only used in some unit tests).  If you trust the batch verification to be consistent with serial verification then the fallback can be eliminated entirely (the BatchVerifier provided by the new library supports an option that omits the fallback if this is chosen as the way forward).
 * curve25519-voi's sr25519 support could use more optimization and more eyes on the code.  The algorithm unfortunately is woefully under-specified, and the implementation was done primarily because I got really sad when I actually looked at go-schnorrkel, and we do not use the algorithm at this time.
2021-06-26 16:53:30 +00:00
Sam Kleinman
a6b30faf35 libs/time: move types/time into libs (#6595) 2021-06-17 16:02:31 -04:00
Callum Waters
c0f7fb08c0 config: add root dir to priv validator (#6585) 2021-06-16 17:13:14 +02:00
Sam Kleinman
7bf84d9d7f config: seperate priv validator config into seperate section (#6462)
Addresses a beginning component of #6255
2021-05-31 13:16:46 +00:00
Marko
719e028e00 libs: internalize some packages (#6366)
## Description

Internalize some libs. This reduces the amount ot public API tendermint is supporting. The moved libraries are mainly ones that are used within Tendermint-core.
2021-05-25 16:25:31 +00:00
Marko
efd2fde474 privval: add ctx to privval interface (#6240)
## Description

- Add `context.Context` to Privval interface

This pr does not introduce context into our custom privval connection protocol because this will be removed in the next release. When this pr is released.
2021-03-16 14:41:03 +00:00
Marko
e2384a00ce privval: return errors on loadFilePV (#6185)
## Description

- return errors on `loadFilePv`

closes #6182
2021-03-01 09:10:09 +00:00
Callum Waters
162f67cf26 correct spelling to US english (#6077) 2021-02-11 18:59:18 +01:00
Marko
1f01e5d726 params: remove blockTimeIota (#5987)
## Description

- removes blocktimeiota 
- merges block params in abci and core state
- spec change: https://github.com/tendermint/spec/pull/248


Closes: #5939
2021-01-28 13:47:24 +00:00
Marko
f05788e632 privval: Query validator key (#5876)
## Description

- Query validator key when a remote signer is used. This is supported gRPC remote signing and filePV only. 


Closes: #3009
2021-01-12 10:06:33 +00:00
Marko
09cf0bcb01 privval: add grpc (#5725)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2021-01-06 10:49:30 -08:00
Erik Grinaker
1ccd23ca1d p2p: fix MConnection inbound traffic statistics and rate limiting (#5868)
Fixes #5866. Inbound traffic monitoring (and by extension inbound rate limiting) was inadvertently removed in 660e72a.
2021-01-06 15:38:23 +01:00
Marko
b2d72dce7e e2e: use ed25519 for secretConn (remote signer) (#5678)
## Description

Hardcode ed25519 to dialTCPFn in e2e tests. 

I will backport `DefaultRequestHandler` fixes

This will be replaced when grpc is implemented.
2020-11-17 13:55:23 +00:00
Erik Grinaker
2d0fcf498d privval: duplicate SecretConnection from p2p package (#5672)
This is so that the `privval` package will not be affected when we refactor `p2p` (#5670). We will be migrating to gRPC shortly (#4698).
2020-11-16 18:16:10 +00:00
Anton Kaliaev
af645ac778 privval: reset pingTimer to avoid sending unnecessary pings (#5642)
Refs #5550
2020-11-10 11:43:01 +00:00
Anton Kaliaev
fa522ca323 privval: increase read/write timeout to 5s and calculate ping interval based on it (#5638)
Partially closes #5550
2020-11-10 11:10:12 +00:00
Marko
bf35cc6443 cmd: add support for --key (#5612)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-11-09 15:22:36 +01:00
Marko
cafad28293 privval: make response values non nullable (#5583)
## Description

make response values non nullable in privval

Does this need a changelog for master?

Closes: #5581 

cc @tarcieri
2020-10-28 15:16:38 +00:00
Marko
346aa14db5 fix lint failures with 1.31 (#5489) 2020-10-13 10:22:53 +02:00
Erik Grinaker
250c3aa92e test: add end-to-end testing framework (#5435)
Partial fix for #5291. For details, see [README.md](https://github.com/tendermint/tendermint/blob/erik/e2e-tests/test/e2e/README.md) and [RFC-001](https://github.com/tendermint/tendermint/blob/master/docs/rfc/rfc-001-end-to-end-testing.md).

This only includes a single test case under `test/e2e/tests/`, as a proof of concept - additional test cases will be submitted separately. A randomized testnet generator will also be submitted separately, there a currently just a handful of static testnets under `test/e2e/networks/`. This will eventually replace the current P2P tests and run in CI.
2020-10-05 09:35:01 +00:00
Erik Grinaker
08708046a7 privval: fix ping message encoding (#5441)
Fixes #5371.
2020-10-01 14:46:21 +00:00
Erik Grinaker
b0130b4661 privval: allow passing options to NewSignerDialerEndpoint (#5434)
Required for #5291 to set timeouts for remote signers.
2020-09-30 16:45:46 +00:00
Marko
b8d08b9ef4 lint: add errchecks (#5316)
## Description

Work towards enabling errcheck

ref #5059
2020-09-04 11:58:03 +00:00
Marko
8cdaa7f515 privval: add chainID to requests (#5239)
## Description

Add chainid to requests to privval. This is a non-breaking change and hardware devices can opt to ignore the field.
 
Closes: #4503 

Took the approach of passing chainID to the client instead of modifying `GetPubKey` because it would lead to a larger change throughout the codebase and in some places it could get tricky to get chainID.
2020-08-17 09:07:15 +00:00
Marko
40bd416d59 test: protobuf vectors for reactors (#5221)
## Description

Add test vectors for all reactors

- [x] state-sync
- [x] privval
- [x] mempool
- [x] p2p
- [x] evidence
- [ ] light?

this PR is primarily oriented at testvectors for things going over the wire. should we expand the testvectors into types as well?

Closes: #XXX
2020-08-11 14:00:11 +00:00
Anton Kaliaev
940e68292c privval: if remote signer errors, don't retry (#5140)
Closes #5112
2020-07-22 10:12:37 +04:00
Marko
2ac5a559b4 libs: wrap mutexes for build flag with godeadlock (#5126)
## Description

This PR wraps the stdlib sync.(RW)Mutex & godeadlock.(RW)Mutex. This enables using go-deadlock via a build flag instead of using sed to replace sync with godeadlock in all files

Closes: #3242
2020-07-20 07:55:09 +00:00
Marko
6ccccb0933 lint: errcheck (#5091)
## Description

add more error checks to tests


gonna do a third PR that tackles the non test cases
2020-07-14 11:04:41 +00:00
Marko
dedf0d2350 proto: folder structure adhere to buf (#5025) 2020-06-22 10:00:51 +02:00
Marko
51da4fe356 types: rename partsheader to partsetheader (#5029)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-06-22 09:38:03 +02:00
Ismail Khoffi
4774a8ec61 non breaking signbytes (#5008)
* test-vectors for backwards compatibility:

 - copy & paste test-vectors from v0.33.5 to ensure
 backwards compatibility for vote's SignBytes

* WIP: everything besides time seems to match :-/

* almost

* Found the culprit: field nums weren't consecutive ints ...

* fix order of partset header too

* this last votes-related test can easily be fixed

* some minor changes and fix last failing test

* move proto types back to stdtime, fix various linting

* use libs/protoio

* remvoe commented code

* add comments

* fix tests

* uncomment testscases

* dont ignore error panic

* fix signable test

* fix happy path testing

* fix comment

Co-authored-by: Marko Baricevic <marbar3778@yahoo.com>
2020-06-15 21:54:58 +02:00
Marko
f6243d8b9e privval: migrate to protobuf (#4985) 2020-06-11 11:54:02 +02:00
Erik Grinaker
db8f1b3df3 migrate all JSON to new JSON encoder (#4975)
Uses new JSON encoder in #4955 for all JSON. Branched off of #4968.
2020-06-08 12:22:59 +00:00
Marko
a88537bb88 ints: stricter numbers (#4939) 2020-06-04 16:34:56 +02:00
Marko
7c576f02ab keys: change to []bytes (#4950) 2020-06-04 15:32:42 +02:00
Marko
99d88cbe2f crypto: remove key suffixes (#4941)
## Description

- remove keyname suffix from keys


Closes: #2228
2020-06-03 05:46:29 +00:00
Anton Kaliaev
b7b721c484 change use of errors.Wrap to fmt.Errorf with %w verb
Closes #4603

Commands used (VIM):

```
:args `rg -l errors.Wrap`
:argdo normal @q | update
```

where q is a macros rewriting the `errors.Wrap` to `fmt.Errorf`.
2020-05-12 03:35:47 +00:00
Anton Kaliaev
2afae13a48 privval: retry GetPubKey/SignVote/SignProposal N times before
returning an error

Closes #4707
2020-05-11 13:42:19 +00:00
Anton Kaliaev
f31f4327b5 privval: remove deprecated OldFilePV
The old format was deprecated in v0.28. It's time we remove it.
2020-04-28 08:51:51 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
Marko
48f073d796 privval: return error on getpubkey (#4534)
closes #3602

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-03-12 11:10:36 +04:00
Erik Grinaker
fe11219795 Fix unexported returns (#4450) 2020-02-21 19:21:39 +01:00
Marko
9bd0f9e634 lint: golint issue fixes (#4258)
* lint: golint issue fixes

- on my local machine golint is a lot stricter than the bot so slowly going through and fixing things.

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* more fixes from golint

* remove isPeerPersistentFn

* add changelog entry
2019-12-17 13:02:45 +01:00
Marko
7b52f51700 libs/common: Refactor libs/common 5 (#4240)
* libs/common: Refactor libs/common 5

- move mathematical functions and types out of `libs/common` to math pkg
- move net functions out of `libs/common` to net pkg
- move string functions out of `libs/common` to strings pkg
- move async functions out of `libs/common` to async pkg
- move bit functions out of `libs/common` to bits pkg
- move cmap functions out of `libs/common` to cmap pkg
- move os functions out of `libs/common` to os pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing issues

* fix tests

closes #41417

woooooooooooooooooo kill the cmn pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix goimport issues

* run gofmt
2019-12-12 09:33:27 +01:00
Marko
89f0bbbd76 libs/common: Refactor libs/common 4 (#4237)
* libs/common: Refactor libs/common 4

- move byte function out of cmn to its own pkg
- move tempfile out of cmn to its own pkg
- move throttletimer to its own pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix linting issues
2019-12-11 23:16:35 +01:00
Marko
27b00cf8d1 libs/common: refactor libs common 3 (#4232)
* libs/common: refactor libs common 3

- move nil.go into types folder and make private
- move service & baseservice out of common into service pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry
2019-12-11 09:31:25 +01:00
Marko
afc4d7a61f libs/common: refactor libs/common 2 (#4231)
* libs/common: refactor libs/common 2

- move random function to there own pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* change imports and usage throughout repo

* fix goimports

* add changelog entry
2019-12-10 18:38:34 +01:00