e2e: add limit and sort to generator (#6998)

I observed a couple of problems with the generator in some recent tests: 

- there were a couple of hybrid test cases which did not have any
  legacy nodes (randomness and all.) I change the probability to
  produce more reliable results.

- added options to the generation to be able to add a max (to
  compliment the earlier min) number of nodes for local testing. 

- added an option to support reversing the sort order so "more
  complex" networks were first, as well as tweaked some of the point
  values. 

- this refactored the generators cli parsing to be a bit more clear.
This commit is contained in:
Sam Kleinman
2021-09-25 15:53:04 +00:00
committed by GitHub
parent 118bfe2087
commit c101fa17ab
3 changed files with 40 additions and 28 deletions
+21 -5
View File
@@ -172,8 +172,11 @@ func LoadManifest(file string) (Manifest, error) {
// SortManifests orders (in-place) a list of manifests such that the
// manifests will be ordered in terms of complexity (or expected
// runtime). Complexity is determined first by the number of nodes,
// and then by the total number of perturbations in the network
func SortManifests(manifests []Manifest) {
// and then by the total number of perturbations in the network.
//
// If reverse is true, then the manifests are ordered with the most
// complex networks before the less complex networks.
func SortManifests(manifests []Manifest, reverse bool) {
sort.SliceStable(manifests, func(i, j int) bool {
// sort based on a point-based comparison between two
// manifests.
@@ -194,24 +197,37 @@ func SortManifests(manifests []Manifest) {
leftScore += (len(n.Perturb) * 2)
if n.StartAt > 0 {
leftScore++
leftScore += 3
}
}
for _, n := range right.Nodes {
rightScore += (len(n.Perturb) * 2)
if n.StartAt > 0 {
rightScore++
rightScore += 3
}
}
// add one point if the network has evidence.
if left.Evidence > 0 {
leftScore += 2
}
if right.Evidence > 0 {
rightScore += 2
}
if left.TxSize > right.TxSize {
leftScore++
}
if right.Evidence > 0 {
if right.TxSize > left.TxSize {
rightScore++
}
if reverse {
return leftScore >= rightScore
}
return leftScore < rightScore
})
}