You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since #3900 I have observed hudson.plugins.ec2.EC2RetentionStrategyTest.testRetentionDespiteIdleWithMinimumInstanceActiveTimeRange failing multiple times in CI (but not locally) with
java.lang.AssertionError: expected:<2> but was:<3>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633)
at hudson.plugins.ec2.EC2RetentionStrategyTest.testRetentionDespiteIdleWithMinimumInstanceActiveTimeRange(EC2RetentionStrategyTest.java:612)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:706)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.lang.Thread.run(Thread.java:1583)
We run BOM tests with surefire.rerunFailingTestsCount=1 to retry flaky tests one time, which prevents mildly flaky tests from creating too much noise but is ineffective against very flaky tests. The fact that it is ineffective here indicates this is a very flaky test.
We should look into making the test less flaky or exclude it if we cannot figure out how.