e2e: allow load generator to succed for short tests (#6952)

This should address last night's failure. We've taken the perspective
of "the load generator shouldn't cause tests to fail" in recent
days/weeks, and I think this is just a next step along that line. The
e2e tests shouldn't test performance. 

I included some comments indicating the ways that this isn't ideal (it
is perhaps not), and I think that if test networks could make
assertions about the required rate, that might be a cool future
improvement (and good, perhaps, for system benchmarking.)
This commit is contained in:
Sam Kleinman
2021-09-16 11:45:51 -04:00
committed by GitHub
parent b0684bd300
commit b0423e2445
+17 -1
View File
@@ -54,9 +54,25 @@ func Load(ctx context.Context, testnet *e2e.Testnet) error {
case numSeen := <-chSuccess:
success += numSeen
case <-ctx.Done():
if success == 0 {
// if we couldn't submit any transactions,
// that's probably a problem and the test
// should error; however, for very short tests
// we shouldn't abort.
//
// The 2s cut off, is a rough guess based on
// the expected value of
// loadGenerateWaitTime. If the implementation
// of that function changes, then this might
// also need to change without more
// refactoring.
if success == 0 && time.Since(started) > 2*time.Second {
return errors.New("failed to submit any transactions")
}
// TODO perhaps allow test networks to
// declare required transaction rates, which
// might allow us to avoid the special case
// around 0 txs above.
rate := float64(success) / time.Since(started).Seconds()
logger.Info("ending transaction load",