Port the bug fix terra-money#76 to upstream. This is critical for ethermint json-rpc to work.
fix: prevent duplicate tx index if it succeeded before
fix: use CodeTypeOk instead of 0
fix: handle duplicate txs within the same block
Co-authored-by: jess jesse@soob.co
ref: #5281
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Closes#8069
* Type `ABCIResponses` was just wrapping type `ResponseFinalizeBlock`. This patch removes the former.
* Did some renaming to avoid confusion on the data structure we are working with.
* We also remove any stale ABCIResponses we may have in the state store at the time of pruning
**IMPORTANT**: There is an undesirable side-effect of the unwrapping. An empty `ResponseFinalizeBlock` yields a 0-length proto-buf serialized buffer. This was not the case with `ABCIResponses`. I have added an interim solution, but open for suggestions on more elegant ones.
This pull requests adds the protocol buffer field for the `ABCI.VoteExtensionsEnableHeight` parameter. This proto field is threaded throughout all of the relevant places where consensus params are used and referenced.
This PR also adds validation of the consensus param updates. Previous consensus param changes didn't depend on _previous_ versions of the params, so this change adds a method for validating against the old params as well.
closes: #8453
This PR makes vote extensions optional within Tendermint. A new ConsensusParams field, called ABCIParams.VoteExtensionsEnableHeight, has been added to toggle whether or not extensions should be enabled or disabled depending on the current height of the consensus engine. Related to: #8453
## What does this change do?
This pull request completes the change to the `metricsgen` metrics. It adds `go generate` directives to all of the files containing the `Metrics` structs.
Using the outputs of `metricsdiff` between these generated metrics and `master`, we can see that there is not a diff between the two sets of metrics when run locally.
```
[william@sidewinder] tendermint[wb/metrics-gen-transition]:. ◆ ./scripts/metricsgen/metricsdiff/metricsdiff metrics_master metrics_generated
[william@sidewinder] tendermint[wb/metrics-gen-transition]:. ◆
```
This change also adds parsing for a `metrics:` key in a field comment. If a comment line begins with `//metrics:` the rest of the line is interpreted to be the metric help text. Additionally, a bug where lists of labels were not properly quoted in the `metricsgen` rendered output was fixed.
Although we index block.height for blocks in the KV indexer, this reserved
attribute was not previously exposed to the event subscription API. Despite
being advertised in the OpenAPI spec, neither the old (websocket) nor new
(events) query interface could see it. This change exposes block.height to the
/events API.
In addition: Remove a non-public constant from types (finalize_block). This
value is used only as an internal tag by the indexer, and should not be exposed
to users of the public interface. (We could probably drop it entirely, as it
was previously a disambiguator for BeginBlock vs. EndBlock events, but keeping
a tag here simplifies the cleanup).
This pull request adds an additional set of metrics targeted at providing more visibility into `abci++`.
The following set of metrics are added and exposed through the `metrics` endpoint:
```
tendermint_consensus_proposal_receive_count{chain_id="test-chain-IrF74Y",status="accepted"} 34
tendermint_consensus_proposal_create_count{chain_id="test-chain-IrF74Y"} 34
tendermint_consensus_vote_extension_receive_count{chain_id="test-chain-IrF74Y",status="accepted"} 34
tendermint_consensus_round_voting_power_percent{chain_id="test-chain-IrF74Y",vote_type="precommit"} 1
tendermint_consensus_round_voting_power_percent{chain_id="test-chain-IrF74Y",vote_type="prevote"} 1
tendermint_state_consensus_param_updates{chain_id="test-chain-IrF74Y"} 0
tendermint_state_validator_set_updates{chain_id="test-chain-IrF74Y"} 0
tendermint_consensus_late_votes{chain_id="test-chain-IrF74Y",vote_type="precommit"} 16
```
This pull request also updates the `metrics.md` file to include some metrics that were previously missed. My hope is to generate the `metrics.md` file with a future version of the tool being architected in #8479
This pull requests adds a new tool, metricsgen, for generating Tendermint metrics constructors from `Metrics` struct definitions. This tool aims to reduce the amount of boilerplate required to add additional metrics to Tendermint.
Its working is fairly simple, it parses the go ast, extracts field information, and uses this field information to execute a go template.
This pull request also adds a proof-of-concept of the tool's output and working by using it to generate the [indexer metrics](https://github.com/tendermint/tendermint/pull/8479/files#diff-4b0c597b6fa05332a2f9a8e0ce079e360602942fae99dc5485f1edfe71c0a29e) using `//go:generate` directives and a simple `make` target.
The next steps for this tool are documented in https://github.com/tendermint/tendermint/issues/8485 and https://github.com/tendermint/tendermint/issues/8486, which detail using the tool to generate the `metrics.md` documentation file and using the tool to migrate away from `go-kit`.
* Refactor so building and linting works
This is the first step towards implementing vote extensions: generating
the relevant proto stubs and getting the build and linter to pass.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix typo
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Better describe method given vote extensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix types tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Move CanonicalVoteExtension to canonical types proto defs
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Regenerate protos including latest PBTS synchrony params update
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Inject vote extensions into proposal
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Thread vote extensions through code and fix tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove extraneous empty value initialization
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix lint
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix missing VerifyVoteExtension request data
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Explicitly ensure length > 0 to sign vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Explicitly ensure length > 0 to sign vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove extraneous comment
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update privval/file.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update types/vote_test.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Format
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Fix ABCI proto generation scripts for Linux
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sync intermediate and goal protos
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update internal/consensus/common_test.go
Co-authored-by: Sergio Mena <sergio@informal.systems>
* Use dummy value with clearer meaning
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Rewrite loop for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Panic on ABCI++ method call failure
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add strong correctness guarantees when constructing extended commit info for ABCI++
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add strong guarantee in extendedCommitInfo that the number of votes corresponds
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Make extendedCommitInfo function more robust
At first extendedCommitInfo expected votes to be in the same order as
their corresponding validators in the supplied CommitInfo struct, but
this proved to be rather difficult since when a validator set's loaded
from state it's first sorted by voting power and then by address.
Instead of sorting the votes in the same way, this approach simply maps
votes to their corresponding validator's address prior to constructing
the extended commit info. This way it's easy to look up the
corresponding vote and we don't need to care about vote order.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove extraneous validator address assignment
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Sign over canonical vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Validate vote extension signature against canonical vote extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Update privval tests for more meaningful dummy value
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add vote extension capability to E2E test app
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Disable lint for weak RNG usage for test app
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Use parseVoteExtension instead of custom parsing in PrepareProposal
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Only include extension if we have received txs
It's unclear at this point why this is necessary to ensure that the
application's local app_hash matches that committed in the previous
block.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Require app_hash from app to match that from last block
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add contrived (possibly flaky) test to check that vote extensions code works
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove workaround for problem now solved by #8229
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* add tests for vote extension cases
* Fix spelling mistake to appease linter
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Collapse redundant if statement
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Formatting
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Always expect an extension signature, regardless of whether an extension is present
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Votes constructed from commits cannot include extensions or signatures
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Pass through vote extension in test helpers
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Temporarily disable vote extension signature requirement
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Expand on vote equality test errors for clarity
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Expand on vote matching error messages in testing
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Allow for selective subscription by vote type
This is an attempt to fix the intermittently failing
`TestPrepareProposalReceivesVoteExtensions` test in the internal
consensus package.
Occasionally we get prevote messages via the subscription channel, and
we're not interested in those. This change allows us to specify what
types of votes we're interested in (i.e. precommits) and discard the
rest.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Read lock consensus state mutex in test helper to avoid data race
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Revert BlockIDFlag parameter in node test
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Perform additional check in ProcessProposal for special txs generated by vote extensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: check that our added tx does not cause all txs to exceed req.MaxTxBytes
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Only set vote extension signatures when signing is successful
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove channel capacity constraint in test helper to avoid missing messages
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add TODO to always require extension signatures in vote validation
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: reject vote extensions if the request height does not match what we expect
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* types: remove extraneous call to voteWithoutExtension in test
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Remove unnecessary address parameter from CanonicalVoteExtension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval: change test vote type to precommit since we use an extension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval: update signing logic to cater for vote extensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* proto: update field descriptions for vote message
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* proto: update field description for vote extension sig in vote message
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* proto/types: use fixed-length 64-bit integers for rounds in CanonicalVoteExtension
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* consensus: fix flaky TestPrepareProposalReceivesVoteExtensions
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* consensus: remove previously added test helper functionality
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* e2e: add error logs when we get an unexpected height in ExtendVote or VerifyVoteExtension requests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* node_test: get validator addresses from privvals
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval/file_test: optimize filepv creation in tests
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* privval: add test to check that vote extensions are always signed
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add a script to check documentation for ToC entries. (#8356)
This script verifies that each document in the docs and architecture directory
has a corresponding table-of-contents entry in its README file. It can be run
manually from the command line.
- Hook up this script to run in CI (optional workflow).
- Update ADR ToC to include missing entries this script found.
* build(deps): Bump async from 2.6.3 to 2.6.4 in /docs (#8357)
Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4.
- [Release notes](https://github.com/caolan/async/releases)
- [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md)
- [Commits](https://github.com/caolan/async/compare/v2.6.3...v2.6.4)
---
updated-dependencies:
- dependency-name: async
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* privval/file_test: reset vote ext sig before signing
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
Co-authored-by: Sergio Mena <sergio@informal.systems>
Co-authored-by: William Banfield <wbanfield@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/vektra/mockery/v2](https://github.com/vektra/mockery) from 2.10.2 to 2.10.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/vektra/mockery/releases">github.com/vektra/mockery/v2's releases</a>.</em></p>
<blockquote>
<h2>v2.10.4</h2>
<h2>Changelog</h2>
<ul>
<li>c943e69 Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/441">#441</a> from cfstras/fix/support-more-env-keys</li>
<li>ed87cf6 fix: allow configuring flags with "-" as Env var</li>
<li>17abd96 fix: unused config field <code>Tags</code></li>
<li>53114cf test: add test for env var configurations</li>
</ul>
<h2>v2.10.3</h2>
<h2>Changelog</h2>
<ul>
<li>ee25bcf Add/update mocks</li>
<li>4703d1a Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/444">#444</a> from vektra/remove_need_deps</li>
<li>ba1f213 Remove packages.NeedDeps</li>
<li>ed38b20 Update go.sum</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="c943e696a0"><code>c943e69</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/441">#441</a> from cfstras/fix/support-more-env-keys</li>
<li><a href="4703d1a42f"><code>4703d1a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/vektra/mockery/issues/444">#444</a> from vektra/remove_need_deps</li>
<li><a href="ed38b20a49"><code>ed38b20</code></a> Update go.sum</li>
<li><a href="ee25bcfcb5"><code>ee25bcf</code></a> Add/update mocks</li>
<li><a href="ba1f213e17"><code>ba1f213</code></a> Remove packages.NeedDeps</li>
<li><a href="17abd96c57"><code>17abd96</code></a> fix: unused config field <code>Tags</code></li>
<li><a href="53114cfce4"><code>53114cf</code></a> test: add test for env var configurations</li>
<li><a href="ed87cf671a"><code>ed87cf6</code></a> fix: allow configuring flags with "-" as Env var</li>
<li>See full diff in <a href="https://github.com/vektra/mockery/compare/v2.10.2...v2.10.4">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
* Outstanding abci-gen changes to 'pb.go' files
* Removed modified_tx_status from spec and protobufs
* Fix sed for OSX
* Regenerated abci protobufs with 'abci-proto-gen'
* Code changes. UTs e2e tests passing
* Recovered UT: TestPrepareProposalModifiedTxStatusFalse
* Adapted UT
* Fixed UT
* Revert "Fix sed for OSX"
This reverts commit e576708c61.
* Update internal/state/execution_test.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Update abci/example/kvstore/kvstore.go
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
* Update internal/state/execution_test.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Addressed some comments
* Added one test that tests error at the ABCI client + Fixed some mock calls
* Addressed remaining comments
* Update abci/example/kvstore/kvstore.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Update abci/example/kvstore/kvstore.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Update abci/example/kvstore/kvstore.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Update spec/abci++/abci++_tmint_expected_behavior_002_draft.md
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Addressed William's latest comments
* Adressed Michael's comment
* Fixed UT
* Some md fixes
* More md fixes
* gofmt
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: M. J. Fromberger <fromberger@interchain.io>
closes: #8039
This pull request updates the new ABCI++ protos to use `enum`s in place of `bool`s. `enums` may be preferred over `bool` because an `enum` can be udpated to include new statuses in the future, whereas a `bool` cannot and is fixed as just `true` or `false` over the whole lifecycle of the API.
This contains two major changes:
- Remove the legacy test logging method, and just explicitly call the
noop logger. This is just to make the test logging behavior more
coherent and clear.
- Move the logging in the light package from the testing.T logger to
the noop logger. It's really the case that we very rarely need/want
to consider test logs unless we're doing reproductions and running a
narrow set of tests.
In most cases, I (for one) prefer to run in verbose mode so I can
watch progress of tests, but I basically never need to consider
logs. If I do want to see logs, then I can edit in the testing.T
logger locally (which is what you have to do today, anyway.)
This change implements the logic for the PrepareProposal ABCI++ method call. The main logic for creating and issuing the PrepareProposal request lives in execution.go and is tested in a set of new tests in execution_test.go. This change also updates the mempool mock to use a mockery generated version and removes much of the plumbing for the no longer used ABCIResponses.
This change set implements the most recent version of `FinalizeBlock`.
# What does this change actually contain?
* This change set is rather large but fear not! The majority of the files touched and changes are renaming `ResponseDeliverTx` to `ExecTxResult`. This should be a pretty inoffensive change since they're effectively the same type but with a different name.
* The `execBlockOnProxyApp` was totally removed since it served as just a wrapper around the logic that is now mostly encapsulated within `FinalizeBlock`
* The `updateState` helper function has been made a public method on `State`. It was being exposed as a shim through the testing infrastructure, so this seemed innocuous.
* Tests already existed to ensure that the application received the `ByzantineValidators` and the `ValidatorUpdates`, but one was fixed up to ensure that `LastCommitInfo` was being sent across.
* Tests were removed from the `psql` indexer that seemed to search for an event in the indexer that was not being created.
# Questions for reviewers
* We store this [ABCIResponses](5721a13ab1/proto/tendermint/state/types.pb.go (L37)) type in the data base as the block results. This type has changed since v0.35 to contain the `FinalizeBlock` response. I'm wondering if we need to do any shimming to keep the old data retrieveable?
* Similarly, this change is exposed via the RPC through [ResultBlockResults](5721a13ab1/rpc/coretypes/responses.go (L69)) changing. Should we somehow shim or notify for this change?
closes: #7658
This change implements the spec for `ProcessProposal`. It first calls the Tendermint block validation logic to check that all of the proposed block fields are well formed and do not violate any of the rules for Tendermint to consider the block valid and then passes the validated block the `ProcessProposal`.
This change also adds additional fixtures to test the change. It adds the `baseMock` types that holds a mock as well as a reference to `BaseApplication`. If the function was not setup by the test on the contained mock Application, the type delegates to the `BaseApplication` and returns what `BaseApplication` returns.
The change also switches the `makeState` helper to take an arg struct so that an ABCI application can be plumbed through when needed.
closes: #7656
* event: Added Events after evidence validation; evidence: refactored AddEvidence
Added context and Metrics as parameter for the pool constructor
* evidence: pushed event firing into evidence pool and added metrics to represent the size of the evpool
* state: fixed parameters of evpool mock functions
* evidence: added test to confirm events are generated
* Removed obsolete EvidenceEventPublisher interface
* evidence: pool removed error on missing eventbus
When testing rollback feature in the Cosmos SDK, we found that the app hash
in Tendermint after rollback was the value after the latest block, rather than
before it.
Co-authored-by: Callum Waters <cmwaters19@gmail.com>
* Rebased and git-squashed the commits in PR #6546
migrate abci to finalizeBlock
work on abci, proxy and mempool
abciresponse, blok events, indexer, some tests
fix some tests
fix errors
fix errors in abci
fix tests amd errors
* Fixes after rebasing PR#6546
* Restored height to RequestFinalizeBlock & other
* Fixed more UTs
* Fixed kvstore
* More UT fixes
* last TC fixed
* make format
* Update internal/consensus/mempool_test.go
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
* Addressed @williambanfield's comments
* Fixed UTs
* Addressed last comments from @williambanfield
* make format
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Our test cases spew a lot of files and directories around $TMPDIR. Make more
thorough use of the testing package's TempDir methods to ensure these are
cleaned up.
In a few cases, this required plumbing test contexts through existing helper
code. In a couple places an explicit path was required, to work around cases
where we do global setup during a TestMain function. Those cases probably
deserve more thorough cleansing (preferably with fire), but for now I have just
worked around it to keep focused on the cleanup.
To simplify local testing, do not report failures for tests that require Docker
when Docker is not avaliable. Instead, log a warning and skip the tests.
This has no effect in CI, where Docker is installed.
