Random "failures": "Agent process on a persistent worker exited unexpectedly!"

[real-or-random]

I'm seeing "Agent process on a persistent worker exited unexpectedly!"

https://cirrus-ci.com/task/4695778608283648
https://cirrus-ci.com/task/5716125398859776

Interestingly, the logs look file. It seems that everything is running smoothly but the task is marked as failed.

[fkorotkov]

Sorry about that! Unpredicted side effect of an update. Should be working now. We just moved to another virtualization and overall architecture and hitting some bumps. Should be working now. Re-ran your tasks

[real-or-random]

Hey @fkorotkov, we're seeing this again for a few tasks, e.g.:
https://cirrus-ci.com/task/4637194787749888

We also saw it yesterday on the same PR.

[fkorotkov]

Seems all the issue were with d9-6 worker. I've checked it and it got the VM corrupted.

Unfortunately, it was happening for a not very popular image and therefor alert didn't fire. I've lowered the alert threshold so hopefully next time we'll fix it pro-actively. Sorry for the inconveniences!