types: CommitVotes struct as last step towards #1648 (#3298)

* VoteSignBytes builds CanonicalVote

* CommitVotes implements VoteSetReader

- new CommitVotes struct holds both the Commit and the ValidatorSet and
  implements VoteSetReader
- ToVote takes a ValidatorSet

* fix TestCommit

* use CommitSig.BlockID

Commits may include votes for a different BlockID, could be nil,
or different altogether. This means we can't use `commit.BlockID`
for reconstructing the sign bytes, since up to -1/3 of the commits
might be for independent BlockIDs. This means CommitSig will need to
include an indicator for what BlockID it signed - if it's not the
committed one or nil, it will need to include it fully in order to be
verified. This is unfortunate but unavoidable so long as we include
votes for non-committed BlockIDs (which we do to track validator
liveness)

* fixes from review

* remove CommitVotes. CommitSig contains address

* remove commit.canonicalVote method

* toVote -> getVote, takes valIdx

* update adr-025

* commit.ToVoteSet -> CommitToVoteSet

* add test

* fix from review

docs/architecture/adr-025-commit.md

@@ -1,14 +1,18 @@
 # ADR 025 Commit
 
 ## Context
+
 Currently the `Commit` structure contains a lot of potentially redundant or unnecessary data.
-In particular it contains an array of every precommit from the validators, which includes many copies of the same data. Such as `Height`, `Round`, `Type`, and `BlockID`. Also the `ValidatorIndex` could be derived from the vote's position in the array, and the `ValidatorAddress` could potentially be derived from runtime context. The only truely necessary data is the `Signature` and `Timestamp` associated with each `Vote`.
+It contains a list of precommits from every validator, where the precommit
+includes the whole `Vote` structure. Thus each of the commit height, round,
+type, and blockID are repeated for every validator, and could be deduplicated.
 
 ```
 type Commit struct {
     BlockID    BlockID `json:"block_id"`
     Precommits []*Vote `json:"precommits"`
 }
+
 type Vote struct {
     ValidatorAddress Address   `json:"validator_address"`
     ValidatorIndex   int       `json:"validator_index"`
@@ -26,7 +30,9 @@ References:
 [#2226](https://github.com/tendermint/tendermint/issues/2226)
 
 ## Proposed Solution
+
 We can improve efficiency by replacing the usage of the `Vote` struct with a subset of each vote, and by storing the constant values (`Height`, `Round`, `BlockID`) in the Commit itself.
+
 ```
 type Commit struct {
     Height  int64
@@ -34,42 +40,56 @@ type Commit struct {
     BlockID    BlockID      `json:"block_id"`
     Precommits []*CommitSig `json:"precommits"`
 }
+
 type CommitSig struct {
+    BlockID  BlockIDFlag
     ValidatorAddress Address
-    Signature []byte
     Timestamp time.Time
+    Signature []byte
 }
+
+
+// indicate which BlockID the signature is for
+type BlockIDFlag int
+
+const (
+	BlockIDFlagAbsent BlockIDFlag = iota // vote is not included in the Commit.Precommits
+	BlockIDFlagCommit                    // voted for the Commit.BlockID
+	BlockIDFlagNil                       // voted for nil
+)
+
 ```
-Continuing to store the `ValidatorAddress` in the `CommitSig` takes up extra space, but simplifies the process and allows for easier debugging.
+
+Note the need for an extra byte to indicate whether the signature is for the BlockID or for nil.
+This byte can also be used to indicate an absent vote, rather than using a nil object like we currently do,
+which has been [problematic for compatibility between Amino and proto3](https://github.com/tendermint/go-amino/issues/260).
+
+Note we also continue to store the `ValidatorAddress` in the `CommitSig`.
+While this still takes 20-bytes per signature, it ensures that the Commit has all
+information necessary to reconstruct Vote, which simplifies mapping between Commit and Vote objects
+and with debugging. It also may be necessary for the light-client to know which address a signature corresponds to if
+it is trying to verify a current commit with an older validtor set.
 
 ## Status
+
 Proposed
 
 ## Consequences
 
 ### Positive
-The size of a `Commit` transmitted over the network goes from:
 
-|BlockID| + n * (|Address| + |ValidatorIndex| + |Height| + |Round| + |Timestamp| + |Type| + |BlockID| + |Signature|)
+Removing the Type/Height/Round/Index and the BlockID saves roughly 80 bytes per precommit.
+It varies because some integers are varint. The BlockID contains two 32-byte hashes an integer,
+and the Height is 8-bytes.
 
-to:
+For a chain with 100 validators, that's up to 8kB in savings per block!
 
 
-|BlockID|+|Height|+|Round| + n*(|Address| + |Signature| + |Timestamp|)
-
-This saves:
-
-n * (|BlockID| + |ValidatorIndex| + |Type|) + (n-1) * (Height + Round)
-
-In the current context, this would concretely be:
-(assuming all ints are int64, and hashes are 32 bytes)
-
-n *(72 + 8 + 1 + 8 + 8) - 16 = n * 97 - 16
-
-With 100 validators this is a savings of almost 10KB on every block.
-
 ### Negative
-This would add some complexity to the processing and verification of blocks and commits, as votes would have to be reconstructed to be verified and gossiped. The reconstruction could be relatively straightforward, only requiring the copying of data from the `Commit` itself into the newly created `Vote`.
+
+- Large breaking change to the block and commit structure
+- Requires differentiating in code between the Vote and CommitSig objects, which may add some complexity (votes need to be reconstructed to be verified and gossiped)
 
 ### Neutral
-This design leaves the `ValidatorAddress` in the `CommitSig` and in the `Vote`. These could be removed at some point for additional savings, but that would introduce more complexity, and make printing of `Commit` and `VoteSet` objects less informative, which could harm debugging efficiency and UI/UX.
+
+- Commit.Precommits no longer contains nil values