Commit Graph

169 Commits

Author SHA1 Message Date
mergify[bot]
f6bbd8302c migration: scope key migration to stores (#9005) (#9027) 2022-07-20 14:24:53 +02:00
Callum Waters
3e96a376b0 spec: merge v0.35 spec into tendermint (#9018) 2022-07-20 12:37:46 +02:00
mergify[bot]
6b18dfcea1 Extract a library from the confix command-line tool. (backport #9012) (#9025)
(cherry picked from commit 18b5a500da)

Pull out the library functionality from scripts/confix and move it to
internal/libs/confix. Replace scripts/confix with a simple stub that has the
same command-line API, but uses the library instead.

Related:

- Move and update unit tests.
- Move scripts/confix/condiff to scripts/condiff.
- Update test data for v34, v35, and v36.
- Update reference diffs.
- Update testdata README.

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
2022-07-15 08:46:28 -07:00
William Banfield
978f754ad3 p2p: set empty timeouts to configed values. (manual backport of #8847) (#8869)
* regenerate mocks using newer style

* p2p: set empty timeouts to small values. (#8847)

These timeouts default to 'do not time out' if they are not set. This times up resources, potentially indefinitely. If node on the other side of the the handshake is up but unresponsive, the[ handshake call](edec79448a/internal/p2p/router.go (L720)) will _never_ return.

* fix light client select statement
2022-06-28 16:07:15 -04:00
mergify[bot]
383408479d keymigrate: improve filtering for legacy transaction hashes (#8466) (#8467)
This is a follow-up to #8352. The check for legacy evidence keys is only based
on the prefix of the key. Hashes, which are unprefixed, could easily have this
form and be misdiagnosed.

Because the conversion for evidence checks the key structure, this should not
cause corruption. The probability that a hash is a syntactically valid evidence
key is negligible.  The tool will report an error rather than storing bad data.
But this does mean that such transaction hashes could cause the migration to
stop and report an error before it is complete.

To ensure we convert all the data, refine the legacy key check to filter these
keys more precisely. Update the test cases to exercise this condition.

(cherry picked from commit dd4fee88ef)
2022-05-04 13:32:40 -07:00
mergify[bot]
9195a005bd Add config samples from TM v26, v27, v28, v29. (#8384) (#8387) 2022-04-21 08:58:18 -07:00
mergify[bot]
2a91d21b61 Add confix testdata for Tendermint v0.30. (#8380) (#8381)
Some additional testdata I grabbed while writing up the draft of RFC 019.

(cherry picked from commit d56392cee9)
2022-04-20 10:04:21 -07:00
mergify[bot]
641d290a6d keymigrate: fix conversion of transaction hash keys (backport #8352) (#8353)
In the legacy database format, keys were generally stored with a string prefix
to partition the key space. Transaction hashes, however, were not prefixed: The
hash of a transaction was the entire key for its record.

When the key migration script scans its input, it checks the format of each
key to determine whether it has already been converted, so that it is safe to run
the script over an already-converted database.

After checking for known prefixes, the migration script used two heuristics to
distinguish ABCI events and transaction hashes: For ABCI events, whose keys
used the form "name/value/height/index", it checked for the right number of
separators. For hashes, it checked that the length is exactly 32 bytes (the
length of a SHA-256 digest) AND that the value does not contain a "/".

This last check is problematic: Any hash containing the byte 0x2f (the code
point for "/") would be incorrectly filtered out from conversion. This leads to
some transaction hashes not being converted.

To fix this problem, this changes how the script recognizes keys:

1. Use a more rigorous syntactic check to filter out ABCI metadata.
2. Use only the length to identify hashes among what remains.

This change is still not a complete fix: It is possible, though unlikely, that
a valid hash could happen to look exactly like an ABCI metadata key. However,
the chance of that happening is vastly smaller than the chance of generating a
hash that contains at least one "/" byte.

Similarly, it is possible that an already-converted key of some other type
could be mistaken for a hash (not a converted hash, ironically, but another
type of the right length). Again, we can't do anything about that.

(cherry picked from commit 34e727676c)
2022-04-14 17:04:28 -07:00
mergify[bot]
52bcd56d60 confix: convert tx-index.indexer from string to array (backport #8342) (#8348)
The format of this config value was changed in v0.35.

- Move plan to its own file (for ease of reading).
- Convert indexer string to an array if not already done.

(cherry picked from commit 69874c2050)
2022-04-14 06:59:16 -07:00
M. J. Fromberger
12e0ea6ea7 confix: add default mempool.version = "v1" in v0.35. (#8335) 2022-04-13 14:28:54 -07:00
M. J. Fromberger
a639323cf0 Add a tool to update old config files to the latest version. (#8300)
A manual backport of #8281, adjusted to stop at v0.35.

* Update pending changelog.
* Backport applicable fixes for v0.35 from master.
2022-04-12 21:19:12 -07:00
mergify[bot]
e4d83ba2ad keymigrate: fix decoding of block-hash row keys (backport #8294) (#8295)
(cherry picked from commit 322bb460dd)
2022-04-09 09:17:28 -07:00
mergify[bot]
fb7ce48c15 scmigrate: ensure target key is correctly renamed (backport #8276) (#8280)
Prior to v0.35, the keys for seen-commit records included the applicable
height.  In v0.35 and beyond, we only keep the record for the latest height,
and its key does not include the height.

Update the seen-commit migration to ensure that the record we retain after
migration is correctly renamed to omit the height from its key.

Update the test cases to check for this condition after migrating.

(cherry picked from commit f3858e52de)
2022-04-07 15:51:26 -07:00
mergify[bot]
813a3f2c7e migration: remove stale seen commits (backport #8205) (#8211) 2022-03-29 15:38:49 -04:00
Sam Kleinman
e40a8468a4 config: backport file writing changes (#7182) 2021-10-29 06:38:52 -04:00
M. J. Fromberger
cf7537ea5f cleanup: Reduce and normalize import path aliasing. (#6975)
The code in the Tendermint repository makes heavy use of import aliasing.
This is made necessary by our extensive reuse of common base package names, and
by repetition of similar names across different subdirectories.

Unfortunately we have not been very consistent about which packages we alias in
various circumstances, and the aliases we use vary. In the spirit of the advice
in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports,
his change makes an effort to clean up and normalize import aliasing.

This change makes no API or behavioral changes. It is a pure cleanup intended
o help make the code more readable to developers (including myself) trying to
understand what is being imported where.

Only unexported names have been modified, and the changes were generated and
applied mechanically with gofmt -r and comby, respecting the lexical and
syntactic rules of Go.  Even so, I did not fix every inconsistency. Where the
changes would be too disruptive, I left it alone.

The principles I followed in this cleanup are:

- Remove aliases that restate the package name.
- Remove aliases where the base package name is unambiguous.
- Move overly-terse abbreviations from the import to the usage site.
- Fix lexical issues (remove underscores, remove capitalization).
- Fix import groupings to more closely match the style guide.
- Group blank (side-effecting) imports and ensure they are commented.
- Add aliases to multiple imports with the same base package name.
2021-09-23 07:52:07 -07:00
William Banfield
4e96c6b234 tools: add mockery to tools.go and remove mockery version strings (#6787)
This change aims to keep versions of mockery consistent across developer laptops.

This change adds mockery to the `tools.go` file so that its version can be managed consistently in the `go.mod` file.

Additionally, this change temporarily disables adding mockery's version number to generated files. There is an outstanding issue against the mockery project related to the version string behavior when running from `go get`. I have created a pull request to fix this issue in the mockery project.
see: https://github.com/vektra/mockery/issues/397
2021-07-30 20:47:15 +00:00
Marko
96863decca deps: remove pkg errors (#6666)
## Description

remove pkg/errors since we use the provided fmt.Errorf
2021-07-07 11:39:19 +00:00
Sam Kleinman
a855f96946 p2p: renames for reactors and routing layer internal moves (#6547) 2021-06-08 08:17:09 -04:00
Sam Kleinman
0c2c0afaf8 db: migration script for key format change (#6355) 2021-04-16 11:53:43 -04:00
Tess Rinearson
f97a498cee scripts: make linkifier default to 'pull' rather than 'issue' (#5689) 2020-11-19 14:23:08 +01:00
Marko
39e81807a3 buf: modify buf.yml, add buf generate (#5653) 2020-11-17 10:30:35 +01:00
Marko
95cff1efb4 proto: buf for everything (#5650)
## Description

- remove installation of protoc
- use buf protoc to generate proto stubs

prior to approving could someone test locally. I restarted my docker instance and its been stuck for 20+ minutes

Closes: #XXX
2020-11-12 11:15:18 +00:00
Tess Rinearson
8ae5c60637 scripts: move build.sh into scripts 2020-11-06 01:02:52 +01:00
Marko
d0db59e16c ci: add goreleaser (#5527)
Co-authored-by: Erik Grinaker <erik@interchain.berlin>
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2020-10-27 13:54:53 +01:00
Marko
346aa14db5 fix lint failures with 1.31 (#5489) 2020-10-13 10:22:53 +02:00
Erik Grinaker
b894f07380 test: remove P2P tests (#5453) 2020-10-05 17:52:01 +02:00
Erik Grinaker
a6b22cfa97 circleci: remove Gitian reproducible_builds job (#5462) 2020-10-05 17:16:05 +02:00
Marko
42e4e8b58e lint: add markdown linter (#5254) 2020-08-17 16:40:50 +02:00
Erik Grinaker
66ed8ec39d proto: reorganize Protobuf schemas (#5102)
Reorganizes the Protobuf schemas. It is mostly bikeshedding, so if something is contentious or causes a lot of extra work then I'm fine with reverting. Some Protobuf and Go import paths will change.

* Move `abci/types/types.proto` to `abci/types.proto`.

* Move `crypto/keys/types.proto` and `crypto/merkle/types.proto` to `crypto/keys.proto` and `crypto/proof.proto`.

* Drop the use of `msgs` in filenames, as "message" is a very overloaded term (all Protobuf types are messages, and we also have `message Message`). Use `types.proto` as a catch-all, and otherwise name files by conceptual grouping instead of message kind.
2020-07-08 15:47:01 +00:00
Marko
7e2cc1db5e linter: (1/2) enable errcheck (#5064)
## Description

partially cleanup in preparation for errcheck

i ignored a bunch of defer errors in tests but with the update to go 1.14 we can use `t.Cleanup(func() { if err := <>; err != nil {..}}` to cover those errors, I will do this in pr number two of enabling errcheck.

ref #5059
2020-07-01 15:13:11 +00:00
Cyrus Goh
5d4eee2eea docs: update .vuepress/config.js (#5043)
## Description

![image](https://user-images.githubusercontent.com/1021101/86193320-7aa13200-bb00-11ea-95ac-aa6468ab7f46.png)


### Next steps:
- [x] bump vuepress-theme-cosmos version
- [x] tweak config.js
- [x] update color variables
- [x] sh, shell → bash for syntax highlighting
2020-07-01 06:43:35 +00:00
Marko
73e22a3c5a abci: proto files follow same path (#5039)
## Description

have the defined abci proto types follow the same directory structure as in abci folder.

Closes: #XXX
2020-06-23 05:38:50 +00:00
Marko
dedf0d2350 proto: folder structure adhere to buf (#5025) 2020-06-22 10:00:51 +02:00
Marko
a12bfca93d proto: move all proto dirs to /proto (#5012)
## Description

This PR moves all proto files under one dir (`/proto`). The script to generate adding functionality to copy the files that still need to be in the same place. (abci & rpc)

renames every proto package from `tendermint.proto.<pkg_name>` to `tendermint.<pkg_name>`

Removes unneeded types in privval proto directory

Closes: #XXX
2020-06-16 14:41:36 +00:00
Ismail Khoffi
4774a8ec61 non breaking signbytes (#5008)
* test-vectors for backwards compatibility:

 - copy & paste test-vectors from v0.33.5 to ensure
 backwards compatibility for vote's SignBytes

* WIP: everything besides time seems to match :-/

* almost

* Found the culprit: field nums weren't consecutive ints ...

* fix order of partset header too

* this last votes-related test can easily be fixed

* some minor changes and fix last failing test

* move proto types back to stdtime, fix various linting

* use libs/protoio

* remvoe commented code

* add comments

* fix tests

* uncomment testscases

* dont ignore error panic

* fix signable test

* fix happy path testing

* fix comment

Co-authored-by: Marko Baricevic <marbar3778@yahoo.com>
2020-06-15 21:54:58 +02:00
Anton Kaliaev
a8d8600308 [block#LastResultsHash] add Events + GasWanted/Used (#4845)
Closes #1007
2020-06-12 12:49:14 +04:00
Erik Grinaker
db8f1b3df3 migrate all JSON to new JSON encoder (#4975)
Uses new JSON encoder in #4955 for all JSON. Branched off of #4968.
2020-06-08 12:22:59 +00:00
Marko
b4aca980e9 proto: change to use gogofaster (#4957) 2020-06-05 14:18:44 +02:00
Marko
a88537bb88 ints: stricter numbers (#4939) 2020-06-04 16:34:56 +02:00
Marko
cf4e056a33 abci: remove python examples
## Description

remove unused and unmaintained python examples of abci

Closes: #XXX
2020-05-12 09:52:04 +00:00
Anton Kaliaev
f31f4327b5 privval: remove deprecated OldFilePV
The old format was deprecated in v0.28. It's time we remove it.
2020-04-28 08:51:51 +00:00
Marko
044f1bf288 format: add format cmd & goimport repo (#4586)
* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes
2020-03-23 09:19:26 +01:00
Marko
48f073d796 privval: return error on getpubkey (#4534)
closes #3602

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-03-12 11:10:36 +04:00
mergify[bot]
4c8e3c8145 fix: proto-breakage (#4506)
* fix: fix proto-breakage

- this is amed to fix proto breakage for consumers

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix for importing third_party everywhere

* undo change

* test breakage change

* test ssh

* test https

* change ssh to https

* fix phony
2020-03-05 08:41:36 +00:00
Marko
31fd99a91a proto: add buf and protogen script (#4369)
* proto: add buf and protogen script

- add buf with minimal changes
- add protogen script to easier generate proto files

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add protoc needs

* add some needed shell cmds

* remove buf from tools as it is not needed everytime

* add proto lint and breakage to ci

* add section in changelog and upgrading files

* address pr comments

* remove space in circle config

* remove spaces in makefile comment

* add section on contributing on how to work with proto

* bump buf to 0.7

* test bufbuild image

* test install make in bufbuild image

* revert to tendermintdev image

* Update Makefile

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
2020-02-11 10:31:15 +01:00
Anton Kaliaev
7e02211246 scripts: remove install scripts (#4242)
Closes #4202

- nobody's supporting them
- users who want to try TM should use pre-build binaries
- devs should be able to install using git clone (or use Vagrant)
- validators don't use them (my guess)
2019-12-12 12:11:28 +04:00
dongsamb
a61f19b12e set Go 1.13 as min required version (#4228) 2019-12-10 11:42:55 +04:00
Marko
3e2751d274 lint: Enable Golint (#4212)
* Fix many golint errors

* Fix golint errors in the 'lite' package

* Don't export Pool.store

* Fix typo

* Revert unwanted changes

* Fix errors in counter package

* Fix linter errors in kvstore package

* Fix linter error in example package

* Fix error in tests package

* Fix linter errors in v2 package

* Fix linter errors in consensus package

* Fix linter errors in evidence package

* Fix linter error in fail package

* Fix linter errors in query package

* Fix linter errors in core package

* Fix linter errors in node package

* Fix linter errors in mempool package

* Fix linter error in conn package

* Fix linter errors in pex package

* Rename PEXReactor export to Reactor

* Fix linter errors in trust package

* Fix linter errors in upnp package

* Fix linter errors in p2p package

* Fix linter errors in proxy package

* Fix linter errors in mock_test package

* Fix linter error in client_test package

* Fix linter errors in coretypes package

* Fix linter errors in coregrpc package

* Fix linter errors in rpcserver package

* Fix linter errors in rpctypes package

* Fix linter errors in rpctest package

* Fix linter error in json2wal script

* Fix linter error in wal2json script

* Fix linter errors in kv package

* Fix linter error in state package

* Fix linter error in grpc_client

* Fix linter errors in types package

* Fix linter error in version package

* Fix remaining errors

* Address review comments

* Fix broken tests

* Reconcile package coregrpc

* Fix golangci bot error

* Fix new golint errors

* Fix broken reference

* Enable golint linter

* minor changes to bring golint into line

* fix failing test

* fix pex reactor naming

* address PR comments
2019-12-05 10:12:08 +01:00
Anton Kaliaev
3e1516b624 linters: enable stylecheck (#4153)
Refs #3262
2019-11-16 19:35:39 +04:00