Summary
TestFunctional/parallel/MySQL is flaky. The test deploys a MySQL 8.4 pod, waits for it to become Ready, and then runs mysql -ppassword -e show databases; inside the pod with exponential retry. It fails when the MySQL image cannot be pulled within the 10-minute PodWait timeout.
Failure details
From pull-minikube-none-docker-linux-x86 on PR #23137:
functional_test.go:1917: ***** TestFunctional/parallel/MySQL: pod "app=mysql" failed to start within 10m0s: context deadline exceeded ****
The pod is stuck in ImagePullBackOff because ECR rate-limits the image pull:
Warning Failed 8m38s (x3 over 10m) kubelet Failed to pull image "public.ecr.aws/docker/library/mysql:8.4": Error response from daemon: toomanyrequests: Rate exceeded
Warning Failed 7m4s (x5 over 10m) kubelet Error: ErrImagePull
Warning Failed 4m49s (x20 over 10m) kubelet Error: ImagePullBackOff
The test runs as part of TestFunctional/parallel, so multiple tests pull images concurrently, making rate limiting more likely.
History
This is a recurring issue. A previous instance was tracked in #13473 (closed Aug 2024), which reported flake rates of 39% on Docker/CRI-O and 32.5% on KVM/CRI-O. The test was also reported failing in #7401 (CrashLoopBackOff, 2020).
Typically fails in the GitHub Functional Tests workflow, rarely also in prow.
Impact
This flaky test blocks unrelated PRs from merging in CI jobs.
Summary
TestFunctional/parallel/MySQLis flaky. The test deploys a MySQL 8.4 pod, waits for it to become Ready, and then runsmysql -ppassword -e show databases;inside the pod with exponential retry. It fails when the MySQL image cannot be pulled within the 10-minute PodWait timeout.Failure details
From pull-minikube-none-docker-linux-x86 on PR #23137:
The pod is stuck in
ImagePullBackOffbecause ECR rate-limits the image pull:The test runs as part of
TestFunctional/parallel, so multiple tests pull images concurrently, making rate limiting more likely.History
This is a recurring issue. A previous instance was tracked in #13473 (closed Aug 2024), which reported flake rates of 39% on Docker/CRI-O and 32.5% on KVM/CRI-O. The test was also reported failing in #7401 (CrashLoopBackOff, 2020).
Typically fails in the GitHub Functional Tests workflow, rarely also in prow.
Impact
This flaky test blocks unrelated PRs from merging in CI jobs.