Commit Graph

63 Commits

Author SHA1 Message Date
Erik Grinaker
fdf9c7ae64 rpc/client: split out client packages (#4628)
* rpc/client: initial split into directories

* lite2: split out test package

* rpc/client: simplify client constructurs

* updated docs

* updated changelog
2020-04-02 13:25:30 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
Marko
89f0bbbd76 libs/common: Refactor libs/common 4 (#4237)
* libs/common: Refactor libs/common 4

- move byte function out of cmn to its own pkg
- move tempfile out of cmn to its own pkg
- move throttletimer to its own pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix linting issues
2019-12-11 23:16:35 +01:00
Marko
3e2751d274 lint: Enable Golint (#4212)
* Fix many golint errors

* Fix golint errors in the 'lite' package

* Don't export Pool.store

* Fix typo

* Revert unwanted changes

* Fix errors in counter package

* Fix linter errors in kvstore package

* Fix linter error in example package

* Fix error in tests package

* Fix linter errors in v2 package

* Fix linter errors in consensus package

* Fix linter errors in evidence package

* Fix linter error in fail package

* Fix linter errors in query package

* Fix linter errors in core package

* Fix linter errors in node package

* Fix linter errors in mempool package

* Fix linter error in conn package

* Fix linter errors in pex package

* Rename PEXReactor export to Reactor

* Fix linter errors in trust package

* Fix linter errors in upnp package

* Fix linter errors in p2p package

* Fix linter errors in proxy package

* Fix linter errors in mock_test package

* Fix linter error in client_test package

* Fix linter errors in coretypes package

* Fix linter errors in coregrpc package

* Fix linter errors in rpcserver package

* Fix linter errors in rpctypes package

* Fix linter errors in rpctest package

* Fix linter error in json2wal script

* Fix linter error in wal2json script

* Fix linter errors in kv package

* Fix linter error in state package

* Fix linter error in grpc_client

* Fix linter errors in types package

* Fix linter error in version package

* Fix remaining errors

* Address review comments

* Fix broken tests

* Reconcile package coregrpc

* Fix golangci bot error

* Fix new golint errors

* Fix broken reference

* Enable golint linter

* minor changes to bring golint into line

* fix failing test

* fix pex reactor naming

* address PR comments
2019-12-05 10:12:08 +01:00
Anton Kaliaev
ad715fe966 types: change Commit to consist of just signatures (#4146)
* types: change `Commit` to consist of just signatures

These are final changes towards removing votes from commit and leaving
only signatures (see ADR-25)

Fixes #1648

* bring back TestCommitToVoteSetWithVotesForAnotherBlockOrNilBlock

+ add absent flag to Vote to indicate that it's for another block

* encode nil votes as CommitSig with BlockIDFlagAbsent

+ make Commit#Precommits array of non-pointers
because precommit will never be nil

* add NewCommitSigAbsent and Absent() funcs

* uncomment validation in CommitSig#ValidateBasic

* add comments to ValidatorSet funcs

* add a changelog entry

* break instead of continue

continue does not make sense in these cases

* types: rename Commit#Precommits to Signatures

* swagger: fix /commit response

* swagger: change block_id_flag type

* fix merge conflicts
2019-11-26 14:10:38 +04:00
Marko
aca94fd9a0 Remvoe errors package from libs (#4157)
*libs/common/errors: remove package

- remove errors file from cmn pkg

- use errorf instead of wrap in async function

- add changelog entry

- closes #3862
2019-11-19 12:47:22 +01:00
Anton Kaliaev
3e1516b624 linters: enable stylecheck (#4153)
Refs #3262
2019-11-16 19:35:39 +04:00
Anton Kaliaev
44a3fbf109 rpc/lib/client & server: try to conform to JSON-RPC 2.0 spec (#4141)
https://www.jsonrpc.org/specification

What is done in this PR:

    JSONRPCClient: validate that Response.ID matches Request.ID I wanted
    to do the same for the WSClient, but since we're sending events as
    responses, not notifications, checking IDs would require storing
    them in memory indefinitely (and we won't be able to remove them
    upon client unsubscribing because ID is different then).

    Request.ID is now optional. Notification is a Request without an ID.
    Previously "" or 0 were considered as notifications

    Remove #event suffix from ID from an event response (partially fixes
    #2949) ID must be either string, int or null AND must be equal to
    request's ID. Now, because we've implemented events as responses, WS
    clients are tripping when they see Response.ID("0#event") !=
    Request.ID("0"). Implementing events as requests would require a lot
    of time (~ 2 days to completely rewrite WS client and server)

    generate unique ID for each request

    switch to integer IDs instead of "json-client-XYZ"
    id=0 method=/subscribe
    id=0 result=...
    id=1 method=/abci_query
    id=1 result=...

> send events (resulting from /subscribe) as requests+notifications (not
responses)

this will require a lot of work. probably not worth it

* rpc: generate an unique ID for each request
in conformance with JSON-RPC spec

* WSClient: check for unsolicited responses

* fix golangci warnings

* save commit

* fix errors

* remove ID from responses from subscribe
Refs #2949

* clients are safe for concurrent access

* tm-bench: switch to int ID

* fixes after my own review

* comment out sentIDs in WSClient
see commit body for the reason

* remove body.Close
it will be closed automatically

* stop ws connection outside of write/read routines
also, use t.Rate in tm-bench indexer when calculating ID

fix gocritic issues

* update swagger.yaml

* Apply suggestions from code review

* fix stylecheck and golint linter warnings

* update changelog

* update changelog2
2019-11-15 14:16:04 +04:00
Marko
1ddc43d029 rpc: add pagination to /validators (#3993)
* Add pagination to /validators

- closes #3472

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add swagger params, default returns all

* address pr comments

* golint fix

* swagger default change, change to default in comment

* swagger.yaml: replace x-example with example

https://swagger.io/docs/specification/adding-examples/

* Revert "swagger.yaml: replace x-example with example"

This reverts commit 9df1b006de.

* update changelog and remove extra body close
2019-11-15 09:38:43 +04:00
Marko
41403d5261 rpc: remove duplication of data in ResultBlock (#3856)
## Issue:

    Removed BlockMeta in ResultBlock in favor of BlockId for /block
    Added block_size to BlockMeta this is reflected in /blockchain

fixes #3188

added breaking as some clients may be using header from blockmeta instead of block in /block

## Commits:

* cleanup block path

Remove duplication of data in `/block`

fixes #3188

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* Remove duplication of data in `/block`

- Created a secondary type to be used for /block

fixes #3188

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* remove commented out code

* fix ci

* add changelog_pending entry

* remove extra variable

* update swagger

* change int to uint for blocksize

* fix swagger

* remove extensive comments

* update changelog

* fix conflicts after merge

* use int for BlockSize and NumTxs in BlockMeta

- with 99.9% guarantee, the size of either will never reach int32
- most of the Go "Size" stdlib functions return int
2019-11-14 14:46:44 +04:00
Phil Salant
bc572217c0 Fix linter errors thrown by lll (#3970)
* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter
2019-10-17 10:42:28 +02:00
Phil Salant
05075ea5b7 linters: modify code to pass maligned and interfacer (#3959)
* Fix maligned structs

* Fix interfacer errors

* Revert accidental go.mod and go.sum changes

* Revert P2PConfig struct maligned reorder

* Revert PeerRoundState struct maligned reordering

* Revert RoundState struct maligned reordering

* Reorder WSClient struct

* Revert accidental type change

* Clean up type change

* Clean up type changes

* Revert to types.ABCIApplicationServer in GRPCServer struct

* Revert maligned changes to BaseConfig struct

* Fix tests in io_test.go

* Fix client_test package tests

* Fix reactor tests in consensus package

* Fix new interfacer errors
2019-09-30 17:12:51 -07:00
Phil Salant
04d13d9945 fix linter errors thrown by unconvert, goconst, and nakedret (#3960)
* Remove unnecessary type conversions

* Consolidate repeated strings into consts

* Clothe return statements

* Update blockchain/v1/reactor_fsm_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

This PR repairs linter errors seen when running the following commands:
golangci-lint run --no-config --disable-all=true --enable=unconvert
golangci-lint run --no-config --disable-all=true --enable=goconst
golangci-lint run --no-config --disable-all=true --enable=nakedret

Contributes to #3262
2019-09-10 11:31:44 +04:00
Marko
8a282a5fee replace errors.go with github.com/pkg/errors (2/2) (#3890)
* init of (2/2) common errors

* Remove instances of cmn.Error (2/2)

- Replace usage of cmnError and errorWrap
- ref #3862

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* comment wording

* simplify IsErrXXX functions

* log panic along with stopping the MConnection
2019-08-11 21:03:40 +04:00
Marko
a1eb2f6c6b tm-cmn to tm-db (#3850)
* tm-cmn to tm-db

* go.mod changes

* go.mod changes

* more go.mod

* fix tm-db

* ci fix, pending change
2019-07-31 11:34:17 +02:00
Marko
98cb8c9783 add staticcheck linting (#3828)
cleanup to add linter

    grpc change:
        https://godoc.org/google.golang.org/grpc#WithContextDialer
        https://godoc.org/google.golang.org/grpc#WithDialer
        grpc/grpc-go#2627
    prometheous change:
        due to UninstrumentedHandler, being deprecated in the future
    empty branch = empty if or else statement
        didn't delete them entirely but commented
        couldn't find a reason to have them
    could not replicate the issue #3406
        but if want to keep it commented then we should comment out the if statement as well
2019-07-25 09:35:30 +04:00
Marko
816dfce8fe libs: Remove db from tendermint in favor of tendermint/tm-cmn (#3811)
* Remove db from tendemrint in favor of tendermint/tm-cmn

- remove db from `libs`
- update dependancy, there have been no breaking changes in the updated deps
	- https://github.com/grpc/grpc-go/releases
	- https://github.com/golang/protobuf/releases

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* changelog add

* gofmt

* more gofmt
2019-07-19 09:54:45 +04:00
yutianwu
fcce9ed4db fix lint 2019-05-21 14:23:50 +08:00
yutianwu
c21b4fcc93 fix bug of proxy 2019-05-21 14:12:40 +08:00
Anton Kaliaev
a7358bc69f libs/db: conditional compilation (#3628)
* libs/db: conditional compilation

For cleveldb: go build -tags cleveldb
For boltdb: go build -tags boltdb

Fixes #3611

* document db_backend param better

* remove deprecated LevelDBBackend

* update changelog

* add missing lines

* add new line

* fix TestRemoteDB

* add a line about boltdb tag

* Revert "remove deprecated LevelDBBackend"

This reverts commit 1aa85453f7.

* make PR non breaking

* change DEPRECATED label format

https://stackoverflow.com/a/36360323/820520
2019-05-07 12:33:47 +04:00
Ismail Khoffi
1e3469789d Ensure WriteTimeout > TimeoutBroadcastTxCommit (#3443)
* Make sure config.TimeoutBroadcastTxCommit < rpcserver.WriteTimeout()

* remove redundant comment

* libs/rpc/http_server: move Read/WriteTimeout into Config

* increase defaults for read/write timeouts

Based on this article
https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration

* WriteTimeout should be larger than TimeoutBroadcastTxCommit

* set a deadline for subscribing to txs

* extract duration into const

* add two changelog entries

* Update CHANGELOG_PENDING.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* Update CHANGELOG_PENDING.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* 12 -> 10

* changelog

* changelog
2019-03-19 19:45:51 -04:00
Anton Kaliaev
d741c7b478 limit number of /subscribe clients and queries per client (#3269)
* limit number of /subscribe clients and queries per client

Add the following config variables (under [rpc] section):
  * max_subscription_clients
  * max_subscriptions_per_client
  * timeout_broadcast_tx_commit

Fixes #2826

new HTTPClient interface for subscriptions

finalize HTTPClient events interface

remove EventSubscriber

fix data race

```
WARNING: DATA RACE
Read at 0x00c000a36060 by goroutine 129:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0

Previous write at 0x00c000a36060 by goroutine 132:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 129 (running) created at:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 132 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
```

lite client works (tested manually)

godoc comments

httpclient: do not close the out channel

use TimeoutBroadcastTxCommit

no timeout for unsubscribe

but 1s Local (5s HTTP) timeout for resubscribe

format code

change Subscribe#out cap to 1

and replace config vars with RPCConfig

TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout

rpc: Context as first parameter to all functions

reformat code

fixes after my own review

fixes after Ethan's review

add test stubs

fix config.toml

* fixes after manual testing

- rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes
Tendermint resources (pubsub)
- rpc: better error in Subscribe and BroadcastTxCommit
- HTTPClient: do not resubscribe if err = ErrAlreadySubscribed

* fixes after Ismail's review

* Update rpc/grpc/grpc_test.go

Co-Authored-By: melekes <anton.kalyaev@gmail.com>
2019-03-11 22:45:58 +04:00
Anton Kaliaev
b6a510a3e7 make ineffassign linter pass (#3386)
Refs #3262

This fixes two small bugs:

1) lite/dbprovider: return `ok` instead of true in parse* functions. It's weird that we're ignoring `ok` value before.
2) consensus/state: previously because of the shadowing we almost never output "Error with msg". Now we declare both `added` and `err` in the beginning of the function, so there's no shadowing.
2019-03-08 09:46:09 +04:00
zjubfd
3421e4dcd7 make lightd availbe (#3364)
1."abci_query": rpcserver.NewRPCFunc(c.ABCIQuery, "path,data,prove")
"validators": rpcserver.NewRPCFunc(c.Validators, "height"),
the parameters and function do not match, cause index out of range error.
2. the prove of query is forced to be true, while default option is false.
3. fix the wrong key of merkle
2019-03-02 16:36:52 -05:00
Anton Kaliaev
8283ca7ddb 3291 follow-up (#3323)
* changelog: use issue number instead of PR number

* follow up to #3291

- rpc/test/helpers.go add StopTendermint(node) func
- remove ensureDir(filepath.Dir(walFile), 0700)
- mempool/mempool_test.go add type cleanupFunc func()

* cmd/show_validator: wrap err to make it more clear
2019-02-18 13:23:40 +04:00
Alessio Treglia
59cc6d36c9 improve ResetTestRootWithChainID() concurrency safety (#3291)
* improve ResetTestRootWithChainID() concurrency safety

Rely on ioutil.TempDir() to create test root directories and ensure
multiple same-chain id test cases can run in parallel.

* Update config/toml.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* clean up test directories after completion

Closes: #1034

* Remove redundant EnsureDir call

* s/PanicSafety()/panic()/s

* Put create dir functionality back in ResetTestRootWithChainID

* Place test directories in OS's tempdir

In modern UNIX and UNIX-like systems /tmp is very often
mounted as tmpfs. This might speed test execution a bit.

* Set 0700 to a const

* rootsDirs -> configRootDirs

* Don't double remove directories

* Avoid global variables

* Fix consensus tests

* Reduce defer stack

* Address review comments

* Try to fix tests

* Update CHANGELOG_PENDING.md

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* Update consensus/common_test.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* Update consensus/common_test.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>
2019-02-18 11:45:27 +04:00
Ethan Buchman
4f2ef36701 types.NewCommit (#3275)
* types.NewCommit

* use types.NewCommit everywhere

* fix log in unsafe_reset

* memoize height and round in constructor

* notes about deprecating toVote

* bring back memoizeHeightRound
2019-02-08 18:40:41 -05:00
Anton Kaliaev
6941d1bb35 use nolint label instead of commenting 2019-02-06 18:21:32 +04:00
Anton Kaliaev
ffd3bf8448 remove or comment out unused code 2019-02-06 15:16:38 +04:00
Dev Ojha
ec53ce359b Simple merkle rfc compatibility (#2713)
* Begin simple merkle compatibility PR

* Fix query_test

* Use trillian test vectors

* Change the split point per RFC 6962

* update spec

* refactor innerhash to match spec

* Update changelog

* Address @liamsi's comments

* Write the comment requested by @liamsi
2019-01-13 18:02:38 -05:00
Alessio Treglia
b646437ec7 Decouple StartHTTP{,AndTLS}Server from Listen() (#2791)
* Decouple StartHTTP{,AndTLS}Server from Listen()

This should help solve cosmos/cosmos-sdk#2715

* Fix small mistake

* Update StartGRPCServer

* s/rpc/rpcserver/

* Start grpccore.StartGRPCServer in a goroutine

* Reinstate l.Close()

* Fix rpc/lib/test/main.go

* Update code comment

* update changelog and comments

* fix tm-monitor. more comments
2018-11-15 15:33:04 -05:00
bradyjoestar
5b120d788a lite support maxOpenConnections (#2413) 2018-10-04 20:39:24 -04:00
Joon
71a34adfe5 General Merkle Proof (#2298)
* first commit

finalize rebase

add protoc_merkle to Makefile

* in progress

* fix kvstore

* fix tests

* remove iavl dependency

* fix tx_test

* fix test_abci_cli

fix test_apps

* fix test_apps

* fix test_cover

* rm rebase residue

* address comment in progress

* finalize rebase
2018-09-28 20:03:19 -04:00
HaoyangLiu
8dda3c3b28 lite: Add synchronization in lite verify (#2396)
* Implement issues 2386: add synchronization in lite verify and change all Certify to Verify

* Replace make(chan struct{}, 0) with make(chan struct{})

* Parameterize memroy cache size and add concurrent test

* Refactor import order
2018-09-28 19:23:21 -04:00
Dev Ojha
2756be5a59 libs: Remove usage of custom Fmt, in favor of fmt.Sprintf (#2199)
* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!
2018-08-10 09:25:57 +04:00
Jae Kwon
e719a93d1d Addressed review for #1815 except those marked as 'TODO make issue' 2018-08-02 03:10:50 -07:00
Ethan Buchman
f6705f02c7 fixes post merge 2018-07-23 23:39:22 -04:00
Ethan Buchman
ea31c4836a Merge branch 'develop' into jae/literefactor4 2018-07-23 23:28:14 -04:00
Anton Kaliaev
270659f03f make Block Header and Data non-pointers
make BlockMeta Header a non-pointer

Refs #693
2018-07-13 12:05:54 +04:00
Ethan Buchman
9018acde5f tmlibs -> tendermint/libs 2018-07-02 14:58:07 -04:00
Ethan Buchman
5453aa6169 Merge branch 'develop' into jae/literefactor4 2018-07-02 14:57:30 -04:00
Ethan Buchman
d55243f0e6 fix import paths 2018-07-01 22:36:49 -04:00
Jae Kwon
37ef5485b4 Add logs to lite/*; Fix rpc status to return consensus height, not blockstore height 2018-06-26 16:53:06 -07:00
Jae Kwon
c3296f2e01 Garbage collect DBProvider (unoptimized); Certifier creation takes a client 2018-06-25 17:13:00 -07:00
Jae Kwon
bf0ff212b9 Refactor "lite" to handle delayed validator set changes.
Also, fix consensus liveness issue.
2018-06-25 17:12:25 -07:00
Alexander Simmerl
f62d6651e3 Merge pull request #1781 from tendermint/1740-node-crashes-when-too-many-rpc-connections
limit number of open connections
2018-06-23 01:15:03 +02:00
Anton Kaliaev
936a655990 limit number of open connections
Refs #1740

also, expose limit option for number concurrent streams for gRPC
(unlimited by default)
2018-06-22 19:25:11 +04:00
Alexander Simmerl
a605b66c5a Move abci imports 2018-06-22 07:28:07 +02:00
Ethan Buchman
c170800fbd Merge branch 'develop' into jae/aminoify 2018-04-07 16:16:53 +03:00
Jae Kwon
32e1d195a0 Fix cmd and lite 2018-04-05 22:05:30 -07:00