CI: Yet again add more locking to prevent parallel tests from colliding

[tbradsha]

Closes MONOREP-222

As done in #45851 and #45902, this adds locking to prevent parallel tests from colliding.

It seems Sync is the main culprit; perhaps if this continues to be an issue we should add locking to the Jetpack_Sync_TestBase class.

Proposed changes:

Other information:

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

• Did this break CI?
• Does this change make sense?

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Review, ...).
  
• ✅ Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

Follow this PR Review Process:

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
3. You can use GitHub's Reviewers functionality to request a review.
4. When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

---

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

• WordPress.com Simple releases happen as soon as you deploy your changes after merging this PR (PCYsg-Jjm-p2).
• WoA releases happen weekly.
• Releases to self-hosted sites happen monthly:
  • Scheduled release: December 2, 2025

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WoA dev site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin (Jetpack), and enable the add/ci/yet_more_locking branch.
• To test on Simple, run the following command on your sandbox:

bin/jetpack-downloader test jetpack add/ci/yet_more_locking

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[jp-launch-control]

Code Coverage Summary

This PR did not change code coverage!

That could be good or bad, depending on the situation. Everything covered before, and still is? Great! Nothing was covered before? Not so great. 🤷

Full summary · PHP report · JS report

[anomiex]

I'm not seeing any theme stuff in this file. What is colliding here?

[tbradsha]

The issue is triggered here:
https://github.com/Automattic/jetpack/actions/runs/19333566055/job/55302450703#step:11:11641

My guess is that $this->server_event_storage->reset(); is wiping things out during the Sync Users test, and that interferes with Theme tests.

[anomiex]

Looking at the error message there, it seems more likely that it's a race where the Users test somewhere inside of sync calls search_theme_directories() (probably via wp_get_themes()) and gets past line 515, then the other process cleans up the directory before it reaches line 539 for that theme.

If that's what's going on here, then it could probably happen in any of these tests and the idea of locking in Jetpack_Sync_TestBase may be the best one. But we may want to see if Vulcan has any better insight here.

@Automattic/jetpack-vulcan: TL;DR: When running coverage tests, we run the normal and multisite tests in parallel to save some wall clock time. Sometimes this makes Sync tests fail because both are using the same themes directory in the filesystem. We already added locking to Jetpack_Sync_Themes_Test (the one that actually writes the temporary themes) in #45851, and Jetpack_Sync_Plugins_Test (which writes a the/the.php plugin) in #45902. But it's looking like we might need to just lock all the Sync tests, since even filesystem-read-only ones can get broken if one of the filesystem-writing tests writes the wrong thing in the other process. Does that seem right to you?

[anomiex]

If we do wind up deciding that "lock everything" is the way to go, probably we should mostly revert #45851 and #45902 in favor of a trait that provides the locking functions.

[coder-karen]

Thanks both for digging in to this!

I’d be inclined to support locking in the base class. I’m not confident per‑test locks would fully prevent this race, since even 'read‑only' tests can traverse the themes directory while another parallel test mutates or cleans it (e.g., wp_get_themes → search_theme_directories), leading to 'not-readable' failures. That could possibly happen in any Sync test that indirectly touches theme discovery.
A centralized lock in Jetpack_Sync_TestBase should help ensure isolation across all Sync tests when the suite runs in parallel, and reduce cross‑process issues. It also avoids accumulating one‑off locks in various places.
I’d also keep it scoped to parallel runs only (like the current approach in this PR) so we don’t slow down normal/local runs.

As for testing, I’m not entirely sure how to reproduce the exact CI setup locally, since it seems to require running normal and multisite tests in parallel. That said, I did run coverage tests for Sync in each environment separately (which didn’t reproduce the issue), and verified that adding a base‑class locking trait doesn’t cause regressions.

For transparency, this is what I used:

jetpack docker phpunit jp-multisite --php=8.4 -- --group jetpack-sync --coverage-php test.cov

[tbradsha]

Thanks for your feedback. Unfortunately, the actual issue is intermittent anyway, so it'd be hard to consistently reproduce anyway.

It sounds like editing the base class is the best route; we'll tackle this when we get a chance.

[tbradsha]

Here's a pass at that:
#45948