Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wasm][debugger] Debugger tests getting randomly aborted on CI #89409

Open
radical opened this issue Jul 24, 2023 · 10 comments
Open

[wasm][debugger] Debugger tests getting randomly aborted on CI #89409

radical opened this issue Jul 24, 2023 · 10 comments
Assignees
@thaystg
Labels
arch-wasm WebAssembly architecture area-Debugger-mono Known Build Error Use this to report build issues in the .NET Helix tab os-browser Browser variant of arch-wasm
Milestone
Future

Comments

@radical
Copy link
Member

radical commented Jul 24, 2023 •
edited by build-analysis bot
Loading

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=351450
log:

...
The active test run was aborted. Reason: Test host process crashed
Results File: /root/helix/work/workitem/uploads/xharness-output/logs/testResults.trx

Passed!  - Failed:     0, Passed:    85, Skipped:     0, Total:    85, Duration: 9 m 18 s - DebuggerTestSuite.dll (net8.0)
Test Run Aborted.

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Reason: Test host process crashed",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Build Definition Test Pull Request
870136 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #104637
869615 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
868925 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #104637
868540 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #99596
868388 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution #106557
866931 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution
866215 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
866081 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution
865123 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution
863764 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #109584
862338 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution
860682 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #104637
859609 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #109495
859264 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #108469
858983 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #108469
858594 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #109381
857377 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #109079
856935 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution
856647 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #109278
846206 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #108324
846158 dotnet/runtime chrome-DebuggerTests.PointerTests.WorkItemExecution #108968

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
1 9 21

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=484656
Error message validated: Reason: Test host process crashed
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 11/30/2023 10:40:39 PM UTC
@radical radical added arch-wasm WebAssembly architecture blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' area-Debugger-mono labels Jul 24, 2023
@radical radical added this to the 8.0.0 milestone Jul 24, 2023
@radical radical self-assigned this Jul 24, 2023
@ghost
Copy link

ghost commented Jul 24, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

log:

...
Detected a container, disabling sandboxing for debugger tests.
The active test run was aborted. Reason: Test host process crashed
Results File: /root/helix/work/workitem/uploads/xharness-output/logs/testResults.trx

Passed!  - Failed:     0, Passed:    85, Skipped:     0, Total:    85, Duration: 9 m 18 s - DebuggerTestSuite.dll (net8.0)
Test Run Aborted.
Author: radical
Assignees: radical
Labels:

arch-wasm, blocking-clean-ci, area-Debugger-mono
Milestone: 8.0.0

@radical radical added the Known Build Error Use this to report build issues in the .NET Helix tab label Jul 26, 2023
@lewing lewing mentioned this issue Jul 26, 2023
Closed
@lewing
Copy link
Member

lewing commented Jul 27, 2023

I bet we are taking longer than 100ms to close https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyOperationManager.cs#L325-L327

See the discussion at microsoft/vstest#2952

@radical
Copy link
Member Author

radical commented Jul 31, 2023

I haven't seen this issue since Jul 26. It would still be a good idea to change the timeout like @lewing suggested.

@radical radical removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Jul 31, 2023
radical added a commit to radical/runtime that referenced this issue Jul 31, 2023
@radical
[wasm][debugger] Set VSTEST_TESTHOST_SHUTDOWN_TIMEOUT to 2s
9876452 
The tests have been getting randomly aborted with:
`The active test run was aborted. Reason: Test host process crashed`

This might be due to the shutdown taking too long. The default timeout
is 100ms. Increasing that to 2000ms.

Issue: dotnet#89409
@radical radical mentioned this issue Jul 31, 2023
Merged
radical added a commit that referenced this issue Jul 31, 2023
@radical
[wasm][debugger] Set VSTEST_TESTHOST_SHUTDOWN_TIMEOUT to 2s (#89703)
6b6bdf0 
The tests have been getting randomly aborted with:
`The active test run was aborted. Reason: Test host process crashed`

This might be due to the shutdown taking too long. The default timeout
is 100ms. Increasing that to 2000ms.

Issue: #89409
@radical
Copy link
Member Author

radical commented Aug 2, 2023

This hasn't been hit on main again, but it is being seen on 7.0 .

@radical radical modified the milestones: 8.0.0, 9.0.0 Aug 2, 2023
@radical radical changed the title [wasm][debugger] Debugger tests getting randomly aborted on CI [release/7.0][wasm][debugger] Debugger tests getting randomly aborted on CI Aug 2, 2023
github-actions bot pushed a commit that referenced this issue Aug 2, 2023
@radical
[wasm][debugger] Set VSTEST_TESTHOST_SHUTDOWN_TIMEOUT to 2s
2f9d7c8 
The tests have been getting randomly aborted with:
`The active test run was aborted. Reason: Test host process crashed`

This might be due to the shutdown taking too long. The default timeout
is 100ms. Increasing that to 2000ms.

Issue: #89409
thaystg pushed a commit that referenced this issue Aug 4, 2023
@github-actions @radical
[wasm][debugger] Set VSTEST_TESTHOST_SHUTDOWN_TIMEOUT to 2s (#89862)
6f5b364 
The tests have been getting randomly aborted with:
`The active test run was aborted. Reason: Test host process crashed`

This might be due to the shutdown taking too long. The default timeout
is 100ms. Increasing that to 2000ms.

Issue: #89409

Co-authored-by: Ankit Jain <[email protected]>
@SamMonoRT
Copy link
Member

hit again in #91393

@SamMonoRT
Copy link
Member

also in #91664

@radekdoulik radekdoulik mentioned this issue Oct 16, 2023
Merged
@pavelsavara
Copy link
Member

Hit again in #94299 log

@pavelsavara
Copy link
Member

This hasn't been hit on main again, but it is being seen on 7.0 .

I see it on main Log

@radical
Copy link
Member Author

radical commented Nov 6, 2023

Yeah, I have been seeing this on rolling builds pretty regularly :/

@pavelsavara pavelsavara mentioned this issue Nov 7, 2023
Open
3 tasks
@pavelsavara
Copy link
Member

See also dotnet/dnceng#451

This was referenced Nov 7, 2023
Merged
Merged
@build-analysis build-analysis bot removed this from the 9.0.0 milestone Nov 15, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Nov 15, 2023
@akoeplinger akoeplinger added this to the 9.0.0 milestone Nov 24, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Nov 24, 2023
@radical radical changed the title [release/7.0][wasm][debugger] Debugger tests getting randomly aborted on CI [wasm][debugger] Debugger tests getting randomly aborted on CI Nov 30, 2023
@radical radical mentioned this issue Nov 30, 2023
Open
@kg kg mentioned this issue Dec 6, 2023
Merged
@pavelsavara pavelsavara mentioned this issue Jan 14, 2024
Merged
@radical radical removed their assignment Feb 27, 2024
@lewing lewing added the os-browser Browser variant of arch-wasm label Apr 24, 2024
@lewing lewing modified the milestones: 9.0.0, Future Apr 26, 2024
@pavelsavara pavelsavara mentioned this issue Sep 24, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thaystg thaystg

Labels
arch-wasm WebAssembly architecture area-Debugger-mono Known Build Error Use this to report build issues in the .NET Helix tab os-browser Browser variant of arch-wasm
Projects
None yet
Milestone
Future
Development

No branches or pull requests
6 participants
@radical @lewing @pavelsavara @akoeplinger @thaystg @SamMonoRT