Commit 03fe256
committed
bug #1669 Fix the E2E tests workflow (javiereguiluz)
This PR was merged into the main branch.
Discussion
----------
Fix the E2E tests workflow
Fixes #1662
The daily E2E workflow has been failing every day since June 26. This PR fixes the root cause and makes two improvements to the workflow.
**Why the tests were failing**
Only the Test Composer Create Project job failed: the install worked, bin/console about worked, the server started, but the first request to the homepage returned 502 Bad Gateway. The root cause is a PHP-FPM segfault caused by the opcache JIT:
```
php-fpm8.5[5018]: segfault at 7f273e598ffc ip 00005581c66e35a6 ... in php-fpm8.5
```
The conditions that had to combine:
* This job is the only one using php-version: latest (currently PHP 8.5.7); the other jobs pin PHP 8.4 and kept passing.
* setup-php enables the opcache JIT by default (opcache.jit=1235, 256M buffer). Stock PHP builds (official Docker images, sury packages, Homebrew) ship with the JIT disabled, which is why this was not reproducible outside GitHub runners
* PHP 8.5.7's JIT crashes on a hot code path in Symfony 8.1. Failures started exactly when demo v3.1.0 (the Symfony 8.1 upgrade) was released on June 25: the last green run installed v3.0.2/Symfony 8.0 on the same PHP 8.5.7. PHP 8.4's JIT is not affected.
The crashed FPM worker made the Symfony CLI proxy return 502; an immediate retry succeeded (fresh worker), which is how the crash was narrowed down.
**Changes**
* Disable the opcache JIT in the composer job (ini-values: opcache.jit=disable)
* Add a "Show diagnostics on failure" step to each test job: PHP version and key ini values, a retry of the request (to tell first-request-only failures from persistent ones), the application logs and the kernel messages (dmesg, which is what exposed the segfault). Until now the workflow captured nothing on failure, making issues like this one invisible.
* Create at most one E2E failure issue at a time: previously the workflow created one issue per day (see #1663–#1668, all closed manually). Now, if any issue whose title starts with E2E Test Failure is still open, the workflow doesn't create a new issue.
Commits
-------
7e7f61b Fix the E2E tests workflow1 file changed
Lines changed: 64 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
45 | 62 | | |
46 | 63 | | |
47 | 64 | | |
| |||
52 | 69 | | |
53 | 70 | | |
54 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
55 | 78 | | |
56 | 79 | | |
57 | 80 | | |
| |||
66 | 89 | | |
67 | 90 | | |
68 | 91 | | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
69 | 109 | | |
70 | 110 | | |
71 | 111 | | |
| |||
93 | 133 | | |
94 | 134 | | |
95 | 135 | | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
96 | 153 | | |
97 | 154 | | |
98 | 155 | | |
| |||
151 | 208 | | |
152 | 209 | | |
153 | 210 | | |
154 | | - | |
| 211 | + | |
155 | 212 | | |
156 | | - | |
| 213 | + | |
157 | 214 | | |
158 | | - | |
159 | | - | |
| 215 | + | |
160 | 216 | | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
| 217 | + | |
| 218 | + | |
167 | 219 | | |
| 220 | + | |
| 221 | + | |
0 commit comments