Skip to content

Commit 37de343

Browse files
author
David Roberts
authored
[ML] Accept a close match for results of DFA evaluation tests (elastic#100810)
The results of these tests can vary in the last significant digit depending on hardware and maths library optimisations. Therefore we need to tolerate small failures in results to avoid spurious failures.
1 parent 12884a8 commit 37de343

File tree

1 file changed

+23
-9
lines changed

1 file changed

+23
-9
lines changed

x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/ml/evaluate_data_frame.yml

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -705,6 +705,8 @@ setup:
705705
}
706706
---
707707
"Test classification auc_roc":
708+
- skip:
709+
features: close_to
708710
- do:
709711
ml.evaluate_data_frame:
710712
body: >
@@ -722,10 +724,12 @@ setup:
722724
}
723725
}
724726
}
725-
- match: { classification.auc_roc.value: 0.8612749970197677 }
727+
- close_to: { classification.auc_roc.value: { value: 0.8612749970197677, error: 0.00000000001 }}
726728
- is_false: classification.auc_roc.curve
727729
---
728730
"Test classification auc_roc with default top_classes_field":
731+
- skip:
732+
features: close_to
729733
- do:
730734
ml.evaluate_data_frame:
731735
body: >
@@ -742,7 +746,7 @@ setup:
742746
}
743747
}
744748
}
745-
- match: { classification.auc_roc.value: 0.8612749970197677 }
749+
- close_to: { classification.auc_roc.value: { value: 0.8612749970197677, error: 0.00000000001 }}
746750
- is_false: classification.auc_roc.curve
747751
---
748752
"Test classification accuracy with missing predicted_field":
@@ -1014,6 +1018,8 @@ setup:
10141018
}
10151019
---
10161020
"Test regression mean_squared_error":
1021+
- skip:
1022+
features: close_to
10171023
- do:
10181024
ml.evaluate_data_frame:
10191025
body: >
@@ -1028,12 +1034,14 @@ setup:
10281034
}
10291035
}
10301036
1031-
- match: { regression.mse.value: 28.67749840974834 }
1037+
- close_to: { regression.mse.value: { value: 28.67749840974834, error: 0.00000000001 }}
10321038
- is_false: regression.msle.value
10331039
- is_false: regression.r_squared.value
10341040
- is_false: regression.huber.value
10351041
---
10361042
"Test regression mean_squared_logarithmic_error":
1043+
- skip:
1044+
features: close_to
10371045
- do:
10381046
ml.evaluate_data_frame:
10391047
body: >
@@ -1048,12 +1056,14 @@ setup:
10481056
}
10491057
}
10501058
1051-
- match: { regression.msle.value: 0.08680568028334916 }
1059+
- close_to: { regression.msle.value: { value: 0.08680568028334916, error: 0.00000000001 }}
10521060
- is_false: regression.mse.value
10531061
- is_false: regression.r_squared.value
10541062
- is_false: regression.huber.value
10551063
---
10561064
"Test regression huber":
1065+
- skip:
1066+
features: close_to
10571067
- do:
10581068
ml.evaluate_data_frame:
10591069
body: >
@@ -1068,12 +1078,14 @@ setup:
10681078
}
10691079
}
10701080
1071-
- match: { regression.huber.value: 3.5088110471730145 }
1081+
- close_to: { regression.huber.value: { value: 3.5088110471730145, error: 0.00000000001 }}
10721082
- is_false: regression.msle.value
10731083
- is_false: regression.mse.value
10741084
- is_false: regression.r_squared.value
10751085
---
10761086
"Test regression r_squared":
1087+
- skip:
1088+
features: close_to
10771089
- do:
10781090
ml.evaluate_data_frame:
10791091
body: >
@@ -1087,13 +1099,15 @@ setup:
10871099
}
10881100
}
10891101
}
1090-
- match: { regression.r_squared.value: 0.8551031778603486 }
1102+
- close_to: { regression.r_squared.value: { value: 0.8551031778603486, error: 0.00000000001 }}
10911103
- is_false: regression.mse
10921104
- is_false: regression.msle.value
10931105
- is_false: regression.huber.value
10941106

10951107
---
10961108
"Test regression with null metrics":
1109+
- skip:
1110+
features: close_to
10971111
- do:
10981112
ml.evaluate_data_frame:
10991113
body: >
@@ -1107,9 +1121,9 @@ setup:
11071121
}
11081122
}
11091123
1110-
- match: { regression.mse.value: 28.67749840974834 }
1111-
- match: { regression.r_squared.value: 0.8551031778603486 }
1112-
- match: { regression.huber.value: 1.9205280586939963 }
1124+
- close_to: { regression.mse.value: { value: 28.67749840974834, error: 0.00000000001 }}
1125+
- close_to: { regression.r_squared.value: { value: 0.8551031778603486, error: 0.00000000001 }}
1126+
- close_to: { regression.huber.value: { value: 1.9205280586939963, error: 0.00000000001 }}
11131127
- is_false: regression.msle.value
11141128
---
11151129
"Test regression given missing actual_field":

0 commit comments

Comments
 (0)