fix(tests): snapshot ExecutionLog under lock to fix parallel race#6194
Conversation
ExecutionOrderReproductionTests.ExecutionLog is a shared static List<string>. Log() locks on write, but the final foreach enumerated the list without the lock. Parallel test instances calling Log() from their lifecycle hooks mutated the list mid-enumeration, throwing "Collection was modified; enumeration operation may not execute" (flaky, observed on ubuntu CI). Snapshot the list under the lock before enumerating.
There was a problem hiding this comment.
Code Review
Summary
Clean, minimal fix for a real flaky-test race condition. The diagnosis is accurate and the solution is correct.
What the PR does
ExecutionLog is a static List<string> shared across all parallel test instances in the class. Log() correctly acquires Lock before calling Add(), but the foreach at the end of Api_Should_Start_After_ConfigureTestConfiguration enumerated the list without holding the lock. Parallel test instances calling Log() from their lifecycle hooks (ConfigureTestOptions, SetupAsync, etc.) could mutate the list mid-enumeration, throwing the observed InvalidOperationException.
The fix snapshots the list inside the lock before enumerating:
List<string> executionLogSnapshot;
lock (Lock)
{
executionLogSnapshot = ExecutionLog.ToList();
}
foreach (var entry in executionLogSnapshot) { ... }This is the idiomatic, correct pattern for this scenario.
Findings
No issues found. A few observations for completeness:
-
The fix is sufficient and well-scoped. Only the enumeration site needed the lock; the snapshot approach avoids holding the lock during I/O (
Console.WriteLine), which is the right call. -
The inline comment is justified. CLAUDE.md allows comments for non-obvious invariants/workarounds — a shared static list with parallel test mutation is exactly that kind of subtle constraint.
-
No other enumeration sites were missed.
Connection_String_Should_Be_Overriddenand other tests don't enumerateExecutionLog, so this is the only callsite that needed fixing. -
Alternative considered (not needed here): Replacing
List<string> + lockwithConcurrentBag<string>would avoid needing a separate lock object entirely, but since enumeration order matters for the diagnostic log and the existing pattern is already established, this would be a separate cleanup — not required for the correctness fix.
LGTM. This is a correct, test-only fix that eliminates a real source of CI flakiness.
Up to standards ✅🟢 Issues
|
Updated [TUnit](https://github.com/thomhurst/TUnit) from 1.51.0 to 1.53.0. <details> <summary>Release notes</summary> _Sourced from [TUnit's releases](https://github.com/thomhurst/TUnit/releases)._ ## 1.53.0 <!-- Release notes generated using configuration in .github/release.yml at v1.53.0 --> ## What's Changed ### Other Changes * feat(assertions): return typed value from IsAssignableTo<T> (#6184) by @thomhurst in thomhurst/TUnit#6187 * fix: stop doubling backslashes in source-gen emitted FilePath (breaks HTML report source links) by @thomhurst in thomhurst/TUnit#6193 * feat(assertions): add ContainsKey().And.Value drill-in for dictionaries (#6185) by @thomhurst in thomhurst/TUnit#6188 * fix(tests): snapshot ExecutionLog under lock to fix parallel race by @thomhurst in thomhurst/TUnit#6194 * fix(engine): run lifecycle hooks before test class construction (#6192) by @thomhurst in thomhurst/TUnit#6195 * feat(assertions): inference-friendly pinned overload for covariant [AssertionExtension] with own generic (#5922) by @thomhurst in thomhurst/TUnit#6196 * feat: add DeferEnumeration to defer data-source expansion to runtime (#5833) by @thomhurst in thomhurst/TUnit#6197 ### Dependencies * chore(deps): update tunit to 1.51.0 by @thomhurst in thomhurst/TUnit#6186 * chore(deps): update microsoft.testing to 18.8.0 by @thomhurst in thomhurst/TUnit#6191 * chore(deps): update aspire to 13.4.3 by @thomhurst in thomhurst/TUnit#6198 **Full Changelog**: thomhurst/TUnit@v1.51.0...v1.53.0 Commits viewable in [compare view](thomhurst/TUnit@v1.51.0...v1.53.0). </details> Updated [TUnit.AspNetCore](https://github.com/thomhurst/TUnit) from 1.51.0 to 1.53.0. <details> <summary>Release notes</summary> _Sourced from [TUnit.AspNetCore's releases](https://github.com/thomhurst/TUnit/releases)._ ## 1.53.0 <!-- Release notes generated using configuration in .github/release.yml at v1.53.0 --> ## What's Changed ### Other Changes * feat(assertions): return typed value from IsAssignableTo<T> (#6184) by @thomhurst in thomhurst/TUnit#6187 * fix: stop doubling backslashes in source-gen emitted FilePath (breaks HTML report source links) by @thomhurst in thomhurst/TUnit#6193 * feat(assertions): add ContainsKey().And.Value drill-in for dictionaries (#6185) by @thomhurst in thomhurst/TUnit#6188 * fix(tests): snapshot ExecutionLog under lock to fix parallel race by @thomhurst in thomhurst/TUnit#6194 * fix(engine): run lifecycle hooks before test class construction (#6192) by @thomhurst in thomhurst/TUnit#6195 * feat(assertions): inference-friendly pinned overload for covariant [AssertionExtension] with own generic (#5922) by @thomhurst in thomhurst/TUnit#6196 * feat: add DeferEnumeration to defer data-source expansion to runtime (#5833) by @thomhurst in thomhurst/TUnit#6197 ### Dependencies * chore(deps): update tunit to 1.51.0 by @thomhurst in thomhurst/TUnit#6186 * chore(deps): update microsoft.testing to 18.8.0 by @thomhurst in thomhurst/TUnit#6191 * chore(deps): update aspire to 13.4.3 by @thomhurst in thomhurst/TUnit#6198 **Full Changelog**: thomhurst/TUnit@v1.51.0...v1.53.0 Commits viewable in [compare view](thomhurst/TUnit@v1.51.0...v1.53.0). </details> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Problem
ExecutionOrderReproductionTests(TUnit.Example.Asp.Net.TestProject) intermittently fails on CI with:Observed on
modularpipeline (ubuntu-latest)(e.g. PR #6188, where it is unrelated to the change under test).Root cause
ExecutionLogis a sharedstatic List<string>.Log()takes a lock when writing, but the finalforeach (var entry in ExecutionLog)enumerated the list without the lock. Tests in the class run in parallel, and their lifecycle hooks (ConfigureTestOptions,SetupAsync,ConfigureTestConfiguration, …) all callLog()→Add(). One instance mutating the list while another enumerates it throws.Fix
Snapshot the list under the lock before enumerating. Test-only change.