diff --git a/docs/rfc/rfc-004-e2e-framework.rst b/docs/rfc/rfc-004-e2e-framework.rst index 209602cc4..8508ca173 100644 --- a/docs/rfc/rfc-004-e2e-framework.rst +++ b/docs/rfc/rfc-004-e2e-framework.rst @@ -17,7 +17,7 @@ Tendermint releases, and improve developer efficiency. Background ---------- -During the 0.35 release cycle, the E2E tests were both a source of great +During the 0.35 release cycle, the E2E tests were a source of great value, helping to identify a number of bugs before release. At the same time, the tests were not consistently passing during this time, thereby reducing their value, and forcing the core development team to allocate time and energy @@ -37,9 +37,9 @@ exists simply to ensure that the test networks have some work during their runs. However, the shape and volume of the work is very consistent and is very gentle to help ensure test reliability. -We should avoid implementing a complex workload generation framework, but -being able to have a few different workload shapes available for test -networks, both generated and hand-crafted, to be able to use would be useful. +We don't need a complex workload generation framework, but being able to have +a few different workload shapes available for test networks, both generated and +hand-crafted, would be useful. Workload patterns/configurations might include: @@ -48,7 +48,7 @@ Workload patterns/configurations might include: - variable transaction size over time. -- transaction broadcaste option (synchronously, checked, fire-and-forget, +- transaction broadcast option (synchronously, checked, fire-and-forget, mixed). - number of transactions to submit. @@ -58,12 +58,12 @@ Workload patterns/configurations might include: Configurable Generator ~~~~~~~~~~~~~~~~~~~~~~ -The scope nightly e2e suite is defined by the `testnet generator +The nightly e2e suite is defined by the `testnet generator `_, and it's difficult to add dimensions or change the focus of the test suite in any way without modifying the implementation of the generator. If the -generator were more configurable, potentially via a file rather than the in -the go implementation, we could modify the focus of the test suite on the +generator were more configurable, potentially via a file rather than in +the Go implementation, we could modify the focus of the test suite on the fly. Features that we might want to configure: @@ -99,14 +99,15 @@ testing early in the development cycle, and if we intend to modify the e2e tests to exercise a feature, we should identify this early and begin the integration process as early as possible. -To facilitate this, we should addopt a practice whereby we exercise specific +To facilitate this, we should adopt a practice whereby we exercise specific features that are currently under development more rigorously in the e2e suite, and then as development stabilizes we can reduce the number or weight of these features in the suite. -There are essentially two end to end tests: the suite of (as of 0.35) 64 -different test networks, and the hand crafted `ci.toml` test case. The -generated test cases help provide +As of 0.35 there are essentially two end to end tests: the suite of 64 +generated test networks, and the hand crafted `ci.toml` test case. The +generated test cases help provide systemtic coverage, while the `ci` run +provides coverage for a large number of features. Reduce Cycle Time ~~~~~~~~~~~~~~~~~