Commit Graph

235 Commits

Author SHA1 Message Date
Sam Kleinman
886519e3ca rpc: add subscription id to events (#6386)
Addresses  #3931
2021-06-15 17:33:47 +00:00
Aleksandr Bezobchuk
3635c7a382 logger: refactor Tendermint logger by using zerolog (#6534) 2021-06-07 08:30:48 -04:00
Sam Kleinman
663c0bba9c rpc: decouple test fixtures from node implementation (#6533) 2021-06-04 09:10:38 -04:00
Emmanuel T Odeke
1e0efb6758 libs/log: use fmt.Fprintf directly with *bytes.Buffer to avoid unnecessary allocations (#6503)
Noticed from profiling that the loggers are memory hungry and alas,
we were passing fmt.Sprintf strings to (*bytes.Buffer).WriteString
which defeats the purpose of using fmt.* This change fixes that and
instead directly invokes fmt.Fprintf.
The benchmarks show the improvement:

```shell
$ benchstat before.txt after.txt
name                     old time/op    new time/op    delta
TMLoggerSimple-8           1.67µs ± 4%    1.69µs ±13%     ~     (p=0.118 n=20+19)
TMLoggerContextual-8       2.01µs ± 8%    1.94µs ± 1%   -3.79%  (p=0.000 n=18+20)
TMFmtLoggerSimple-8        1.20µs ± 3%    1.16µs ± 2%   -3.39%  (p=0.000 n=20+16)
TMFmtLoggerContextual-8    1.53µs ±19%    1.43µs ±13%   -6.10%  (p=0.001 n=20+20)

name                     old alloc/op   new alloc/op   delta
TMLoggerSimple-8             696B ± 0%      616B ± 0%  -11.49%  (p=0.000 n=20+20)
TMLoggerContextual-8       1.02kB ± 0%    0.94kB ± 0%   -7.87%  (p=0.000 n=20+20)
TMFmtLoggerSimple-8          240B ± 0%      160B ± 0%  -33.33%  (p=0.000 n=20+20)
TMFmtLoggerContextual-8      416B ± 0%      336B ± 0%  -19.23%  (p=0.000 n=20+20)

name                     old allocs/op  new allocs/op  delta
TMLoggerSimple-8             13.0 ± 0%      12.0 ± 0%   -7.69%  (p=0.000 n=20+20)
TMLoggerContextual-8         17.0 ± 0%      16.0 ± 0%   -5.88%  (p=0.000 n=20+20)
TMFmtLoggerSimple-8          6.00 ± 0%      5.00 ± 0%  -16.67%  (p=0.000 n=20+20)
TMFmtLoggerContextual-8      8.00 ± 0%      7.00 ± 0%  -12.50%  (p=0.000 n=20+20)
```

Fixes #6502
2021-05-28 17:27:47 +00:00
Marko
719e028e00 libs: internalize some packages (#6366)
## Description

Internalize some libs. This reduces the amount ot public API tendermint is supporting. The moved libraries are mainly ones that are used within Tendermint-core.
2021-05-25 16:25:31 +00:00
Cuong Manh Le
2d0b3a300f libs/clist: fix flaky tests (#6453)
To make sure finalizers run, we use channel for synchronization, and a
separate goroutine for trigger runtime.GC every 1 second. In practice,
just two consecutive runtime.GC calls can make all finalizers will run,
but using a separate goroutine make the code more robust and not depend
on garbage collector internal implementation.

Fixes #6452
2021-05-25 09:36:11 +00:00
Emmanuel T Odeke
6fdf665385 libs/os: avoid CopyFile truncating destination before checking if regular file (#6428)
This change fixes a potential exploitable vulnerability
that can cause the WAL to be consistently truncated by falsely
supplying the WAL path which would be any arbitrary dirrectory.

Fixes #6427
2021-05-07 14:46:16 +02:00
Aleksandr Bezobchuk
bc643b19c4 p2p: support private peer IDs in new p2p stack (#6409)
Pass a set of private peer ids to the `PeerManager` and any node that exists in this set is not returned in the `Advertise` method.

closes: #6405
2021-04-30 20:37:17 +00:00
Ismail Khoffi
ee70430255 libs: remove most of libs/rand (#6364) 2021-04-23 16:00:35 +02:00
Sam Kleinman
0811c7be99 fix: theoretical leak in clisit.Init (#6302) 2021-04-01 10:03:51 -04:00
Sam Kleinman
0124593a61 fix: avoid race with a deeper copy (#6285) 2021-03-26 15:15:45 -04:00
Anton Kaliaev
fc5a108d53 libs/log: [JSON format] include timestamp (#6174)
Closes #6146
2021-02-25 07:06:24 +00:00
Marko
45572117ae logging: print string instead of callback (#6177)
## Description

Fixes marshaling error in sdk

closes https://github.com/cosmos/cosmos-sdk/issues/8578

the output stays the same, we are avoiding the passing of the callback because sdk uses typed logging.
2021-02-24 18:56:28 +00:00
Callum Waters
162f67cf26 correct spelling to US english (#6077) 2021-02-11 18:59:18 +01:00
Erik Grinaker
2aad26e2f1 p2p: tighten up and test PeerManager (#6034)
This tightens up the `PeerManager` and related code, adds a ton of tests, and fixes a bunch of inconsistencies and bugs.
2021-02-03 06:15:23 +00:00
Aleksandr Bezobchuk
17905cbaa2 sync: move closer to separate file (#6015)
Closes: #6013
2021-01-29 16:59:15 +00:00
Anton Kaliaev
d76add65a6 libs/log: format []byte as hexidecimal string (uppercased) (#5960)
Closes: #5806 

Co-authored-by: Lanie Hei <heixx011@umn.edu>
2021-01-25 16:25:29 +04:00
Aleksandr Bezobchuk
68bd2116f0 mempool: p2p refactor (#5919) 2021-01-22 09:34:12 -05:00
Marko
09cf0bcb01 privval: add grpc (#5725)
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2021-01-06 10:49:30 -08:00
Erik Grinaker
a0d4d85375 os: simplify EnsureDir() (#5871)
#5852 fixed an issue with error propagation in `os.EnsureDir()`. However, this function is basically identical to `os.MkdirAll()`, and can be replaced entirely with a call to it. We keep the function for backwards compatibility.
2021-01-06 15:27:35 +00:00
Erik Grinaker
1ccd23ca1d p2p: fix MConnection inbound traffic statistics and rate limiting (#5868)
Fixes #5866. Inbound traffic monitoring (and by extension inbound rate limiting) was inadvertently removed in 660e72a.
2021-01-06 15:38:23 +01:00
Erik Grinaker
9c47b572f7 libs/os: EnsureDir now returns IO errors and checks file type (#5852)
Fixes #5839.
2021-01-04 14:30:38 +00:00
Erik Grinaker
bcfc889f25 p2p: implement new Transport interface (#5791)
This implements a new `Transport` interface and related types for the P2P refactor in #5670. Previously, `conn.MConnection` was very tightly coupled to the `Peer` implementation -- in order to allow alternative non-multiplexed transports (e.g. QUIC), MConnection has now been moved below the `Transport` interface, as `MConnTransport`, and decoupled from the peer. Since the `p2p` package is not covered by our Go API stability, this is not considered a breaking change, and not listed in the changelog.

The initial approach was to implement the new interface in its final form (which also involved possible protocol changes, see https://github.com/tendermint/spec/pull/227). However, it turned out that this would require a large amount of changes to existing P2P code because of the previous tight coupling between `Peer` and `MConnection` and the reliance on subtleties in the MConnection behavior. Instead, I have broadened the `Transport` interface to expose much of the existing MConnection interface, preserved much of the existing MConnection logic and behavior in the transport implementation, and tried to make as few changes to the rest of the P2P stack as possible. We will instead reduce this interface gradually as we refactor other parts of the P2P stack.

The low-level transport code and protocol (e.g. MConnection, SecretConnection and so on) has not been significantly changed, and refactoring this is not a priority until we come up with a plan for QUIC adoption, as we may end up discarding the MConnection code entirely.

There are no tests of the new `MConnTransport`, as this code is likely to evolve as we proceed with the P2P refactor, but tests should be added before a final release. The E2E tests are sufficient for basic validation in the meanwhile.
2020-12-15 15:08:16 +00:00
Anton Kaliaev
b1bbd37519 libs/bits: validate BitArray in FromProto (#5720)
Closes #5705
2020-12-01 12:44:56 +00:00
Marko
781f4badc3 ci: build for 32 bit, libs: fix overflow (#5700) 2020-11-26 16:12:25 +01:00
Callum Waters
909da42789 light: make fraction parts uint64, ensuring that it is always positive (#5655) 2020-11-17 14:23:16 +01:00
Alessio Treglia
8bd3d5105f libs/os: remove unused aliases, add test cases (#5654)
Remove unused ReadFile (unused) and
WriteFile (almost unused, alias of ioutil.WriteFile).

Add testcases for Must{Read,Write}File.
2020-11-13 10:59:45 +00:00
Alessio Treglia
eb0d353767 libs/os: add test case for TrapSignal (#5646) 2020-11-12 10:19:05 +04:00
Anton Kaliaev
8e6194626e light: model-based tests (#5461)
This is the first iteration of model-based testing in Go Tendermint. The test runner is using the static JSON fixtures located under the ./json directory. In the future, the Rust tensgen binary will be used to generate those (given the static intermediate scenarios and the test seed, which will be published along with each testgen release).

Closes: #5322
2020-11-02 12:07:18 +04:00
Marko
346aa14db5 fix lint failures with 1.31 (#5489) 2020-10-13 10:22:53 +02:00
Marko
0ed8dba991 lint: enable errcheck (#5336)
## Description

Enable errcheck linter throughout the codebase

Closes: #5059
2020-09-07 15:03:18 +00:00
Anton Kaliaev
4827ed121f libs/bits: inline defer and change order of mutexes (#5187)
* libs/bits: inline defer and change order of mutexes

Closes #3217

* abci/client: unexpose StopForError func

a) it's not called outside
b) the reason for exposing it in the first place is unclear
c) Stop already exist if someone from outside wants to stop the client
2020-08-26 13:18:30 +04:00
Marko
a6032f4183 fix go 1.15 errors (#5277)
## Description

fix golang 1.15 errors. 

Closes: #XXX
2020-08-24 12:26:05 +00:00
Marko
42e4e8b58e lint: add markdown linter (#5254) 2020-08-17 16:40:50 +02:00
Sad Pencil
62d09ccc10 libs/rand: fix "out-of-memory" error on unexpected argument (#5215) 2020-08-10 10:42:14 +02:00
Marko
2d167aefcf ci: freeze golangci action version (#5196)
## Description

This PR updates golang-ci to latest and stops looking at master for the action. 

Closes: #XXX
2020-08-03 07:57:06 +00:00
Callum Waters
cf84dcd44c light cli: add feature flags and save providers (#5148) 2020-07-28 12:11:55 +02:00
n-hutton
375f0c819f add fixes for flaky tests (#5146)
While working on tendermint my colleague @jinmannwong fixed a few of the unit tests that we found to be flaky in our CI. We thought that you might find this useful, see below for comments.
2020-07-27 10:36:56 +04:00
Marko
2ac5a559b4 libs: wrap mutexes for build flag with godeadlock (#5126)
## Description

This PR wraps the stdlib sync.(RW)Mutex & godeadlock.(RW)Mutex. This enables using go-deadlock via a build flag instead of using sed to replace sync with godeadlock in all files

Closes: #3242
2020-07-20 07:55:09 +00:00
Marko
6ccccb0933 lint: errcheck (#5091)
## Description

add more error checks to tests


gonna do a third PR that tackles the non test cases
2020-07-14 11:04:41 +00:00
Erik Grinaker
bf3c87c864 test: deflake TestAddAndRemoveListenerConcurrency and TestSyncer_SyncAny (#5101)
Fixes #5094.
2020-07-08 13:33:50 +00:00
Marko
7e2cc1db5e linter: (1/2) enable errcheck (#5064)
## Description

partially cleanup in preparation for errcheck

i ignored a bunch of defer errors in tests but with the update to go 1.14 we can use `t.Cleanup(func() { if err := <>; err != nil {..}}` to cover those errors, I will do this in pr number two of enabling errcheck.

ref #5059
2020-07-01 15:13:11 +00:00
Marko
dedf0d2350 proto: folder structure adhere to buf (#5025) 2020-06-22 10:00:51 +02:00
Marko
f6243d8b9e privval: migrate to protobuf (#4985) 2020-06-11 11:54:02 +02:00
Marko
d54de61bf6 consensus: proto migration (#4984)
## Description

migrate consensus to protobuf

Closes: #XXX
2020-06-10 12:08:47 +00:00
Erik Grinaker
660e72a12f p2p/conn: migrate to Protobuf (#4990)
Migrates the p2p connections to Protobuf. Supersedes #4800.

gogoproto's `NewDelimitedReader()` uses an internal buffer, which makes it unsuitable for reading individual messages from a shared reader (since any remaining data in the buffer will be discarded). We therefore add a new `protoio` package with an unbuffered `NewDelimitedReader()`. Additionally, the `NewDelimitedWriter()` returns the number of bytes written, and we've added `MarshalDelimited()` and `UnmarshalDelimited()`, to ease migration of existing code.
2020-06-09 16:09:51 +00:00
Erik Grinaker
ccc990498d json: add Amino-compatible encoder/decoder (#4955)
Amino-compatible JSON encoder/decoder, including bug compatibility. Interface types must be registered via `json.RegisterType()`. Unlike Amino, this allows floats to be encoded/decoded.

Partial fix for #4828, needs code migration.
2020-06-08 11:42:35 +00:00
Marko
a88537bb88 ints: stricter numbers (#4939) 2020-06-04 16:34:56 +02:00
Anton Kaliaev
994912211c p2p/conn: add a test for MakeSecretConnection (#4829)
Refs #4154
2020-06-03 16:28:23 +04:00
Alessio Treglia
c8483531d8 consensus: attempt to repair the WAL file on data corruption (#4682)
Closes: #4578

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-06-03 15:14:12 +04:00