-
Notifications
You must be signed in to change notification settings - Fork 0
Description
The crux of Ciclops, internally, is the test_summary data structure.
It is computed with a single traversal through the directory where the test artifacts
are held.
Each test file contains a single test execution in a single platform-k8s-postgres.
Each is ingested by a function that then updates the test_summary as a side effect.
This is how the test_summary looks overall.
{
"total_run": 0,
"total_failed": 0,
"total_special_fails": 0,
"by_test": { … },
"by_code": { … },
"by_special_failures": { … },
"by_matrix": { … },
"by_k8s": { … },
"by_platform": { … },
"by_postgres": { … },
"test_durations": { … },
"suite_durations": { … },
}internally, the by_XYZ dictionaries contain, within them, something like
by_test = {
"total": {},
"failed": {},
"k8s_versions_failed": {},
"pg_versions_failed": {},
"platforms_failed": {},
}And inside of each of these dictionaries, the element in question will be the key.
E.g. the test name, in the dictionary above, after a few iterations, we could have something like
by_test = {
"total": {
"test A": 12,
"test B", 10,
},
"failed": {
"test A": 2,
"test B", 1,
},
"k8s_versions_failed": {
"test A": {
"1.26": True,
"1.27": True,
},
"test B": {
"1.26": True,
}
},
...
...
}The above structure gets difficult to reason about.
It would be a lot more understandable to use the following representation.
by_test = {
"test A": {
"total": 12,
"failed": 2,
"k8s_versions_failed": ["1.26", "1.27"],
...
},
"test B": {
"total": 10,
"failed": 1,
"k8s_versions_failed": ["1.26"],
...
},
...
...
}