Skip to content

[BUG] Collector fails when one of the project's libraries is called collector #1739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alex-ntk opened this issue Mar 16, 2025 · 21 comments
Closed
Labels
as-designed Expected behaviour

Comments

@alex-ntk
Copy link

Tested on two machines, reproducible 100%.

mkdir MySolution
cd MySolution
dotnet new classlib -n MyLibrary
dotnet new console -n Collector
cd Collector
dotnet add reference ../MyLibrary/MyLibrary.csproj
cd ..
dotnet new nunit -n MyTests
cd MyTests
dotnet add reference ../Collector/Collector.csproj
cd ..
dotnet.exe test MyTests\MyTests.csproj --collect:"XPlat Code Coverage"

fails with

Data collector 'XPlat code coverage' message: [coverlet]Coverlet.Collector.Utilities.CoverletDataCollectorExceptio
      n: CoverletCoverageDataCollector: Failed to instrument modules
       ---> System.AggregateException: One or more errors occurred. (The process cannot access the file 'C:\Projects\Cor
      al.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process
      .) (The process cannot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Col
      lector.dll' because it is being used by another process.) (The process cannot access the file 'C:\Projects\Coral.T
      radeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.) (
      The process cannot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collect
      or.dll' because it is being used by another process.) (The process cannot access the file 'C:\Projects\Coral.Trade
      Collector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.) (The
      process cannot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.d
      ll' because it is being used by another process.) (The process cannot access the file 'C:\Projects\Coral.TradeColl
      ector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.) (The proc
      ess cannot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll'
      because it is being used by another process.) (The process cannot access the file 'C:\Projects\Coral.TradeCollecto
      r.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.) (The process
      cannot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' beca
      use it is being used by another process.) (The process cannot access the file 'C:\Projects\Coral.TradeCollector.1\
      MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.) (The process cann
      ot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because
      it is being used by another process.)
       ---> System.IO.IOException: The process cannot access the file 'C:\Projects\Coral.TradeCollector.1\MySolution\MyT
      ests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.
         at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
         at Coverlet.Core.Helpers.FileSystem.Copy(String sourceFileName, String destFileName, Boolean overwrite) in /_/s
      rc/coverlet.core/Helpers/FileSystem.cs:line 35
         at Coverlet.Core.Helpers.InstrumentationHelper.<>c__DisplayClass17_0.<RestoreOriginalModule>b__0() in /_/src/co
      verlet.core/Helpers/InstrumentationHelper.cs:line 269
         at Coverlet.Core.Helpers.RetryHelper.<>c__DisplayClass0_0.<Retry>b__0() in /_/src/coverlet.core/Helpers/RetryHe
      lper.cs:line 28
         at Coverlet.Core.Helpers.RetryHelper.Do[T](Func`1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in /_/
      src/coverlet.core/Helpers/RetryHelper.cs:line 55
         --- End of inner exception stack trace ---
         at Coverlet.Core.Helpers.RetryHelper.Do[T](Func`1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in /_/
      src/coverlet.core/Helpers/RetryHelper.cs:line 62
         at Coverlet.Core.Helpers.RetryHelper.Retry(Action action, Func`1 backoffStrategy, Int32 maxAttemptCount) in /_/
      src/coverlet.core/Helpers/RetryHelper.cs:line 26
         at Coverlet.Core.Helpers.InstrumentationHelper.RestoreOriginalModule(String module, String identifier) in /_/sr
      c/coverlet.core/Helpers/InstrumentationHelper.cs:line 267
         at Coverlet.Core.Coverage.PrepareModules() in /_/src/coverlet.core/Coverage.cs:line 145
         at Coverlet.Collector.DataCollection.CoverageWrapper.PrepareModules(Coverage coverage) in /_/src/coverlet.colle
      ctor/DataCollection/CoverageWrapper.cs:line 71
         at Coverlet.Collector.DataCollection.CoverageManager.InstrumentModules() in /_/src/coverlet.collector/DataColle
      ction/CoverageManager.cs:line 66
       ---> (Inner Exception #1) System.IO.IOException: The process cannot access the file 'C:\Projects\Coral.TradeColle
      ctor.1\MySolution\MyTests\bin\Debug\net9.0\Collector.dll' because it is being used by another process.
         at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
         at Coverlet.Core.Helpers.FileSystem.Copy(String sourceFileName, String destFileName, Boolean overwrite) in /_/s
      rc/coverlet.core/Helpers/FileSystem.cs:line 35
         at Coverlet.Core.Helpers.InstrumentationHelper.<>c__DisplayClass17_0.<RestoreOriginalModule>b__0() in /_/src/co
      verlet.core/Helpers/InstrumentationHelper.cs:line 269
         at Coverlet.Core.Helpers.RetryHelper.<>c__DisplayClass0_0.<Retry>b__0() in /_/src/coverlet.core/Helpers/RetryHe
      lper.cs:line 28
         at Coverlet.Core.Helpers.RetryHelper.Do[T](Func`1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in /_/
      src/coverlet.core/Helpers/RetryHelper.cs:line 55<---

... another 10 of these
@github-actions github-actions bot added the untriaged To be investigated label Mar 16, 2025
@Bertk
Copy link
Collaborator

Bertk commented Mar 24, 2025

Please provide more information e.g. diag log files. This looks like a known issue CoverletCoverageDataCollector: Failed to instrument modules

Please build test project, publish test project and try dotnet vstest … (see Coverlet integration with VSTest )

@Bertk Bertk added needs more info More details are needed waiting for customer Waiting for customer action and removed untriaged To be investigated labels Mar 24, 2025
@alex-ntk
Copy link
Author

I think it would be better if try to produce diag log files as it seems that you know how to do it and I, unfortunately, do not... If you have any troubles reproducing the issue on your end, please let me know.

@Bertk
Copy link
Collaborator

Bertk commented Mar 25, 2025

You can use optional parameters to create diag log files (see below). I used your instructions and created a repo but could not reproduce your errors.

We need still more information e.g.

  • dotnet SDK version and details for created csproj files etc.
  • details runtime enviroment

Please use latest released coverlet nuget package

dotnet build -c debug
dotnet.exe test MyTests\MyTests.csproj -c debug --collect:"XPlat Code Coverage" --no-build -bl:test.binlog --diag:"artifacts\log\debug\test.diag.log;tracelevel=verbose"

or use vstest

dotnet build -c debug
dotnet publish MyTests\MyTests.csproj -c debug
dotnet vstest artifacts\publish\MyTests\debug\MyTests.dll --collect:"XPlat Code Coverage" --diag:"artifacts\log\debug\test.diag.log;tracelevel=verbose"
Image

repo.zip

@alex-ntk
Copy link
Author

Thank you for the instruction, very helpful!

The error is reproducible on my end. The log files attached. The dotnet vstest command produces the same error.

test.diag.host.25-03-25_09-32-19_24233_5.log
test.diag.log
test.diag.datacollector.25-03-25_09-32-06_41069_5.log

@alex-ntk
Copy link
Author

C:\Projects\tmp\MySolution> dotnet --list-sdks
8.0.113 [C:\Program Files\dotnet\sdk]
8.0.406 [C:\Program Files\dotnet\sdk]
9.0.200 [C:\Program Files\dotnet\sdk]
C:\Projects\tmp\MySolution> dotnet --list-runtimes
Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
C:\Projects\tmp\MySolution>

@Bertk
Copy link
Collaborator

Bertk commented Mar 25, 2025

Please also check the whether repo.zip project also generates the same error with your environment. Thanks.

My system has:

C:\GitHub>dotnet --list-sdks
6.0.428 [C:\Program Files\dotnet\sdk]
8.0.113 [C:\Program Files\dotnet\sdk]
8.0.407 [C:\Program Files\dotnet\sdk]
9.0.200 [C:\Program Files\dotnet\sdk]
9.0.201 [C:\Program Files\dotnet\sdk]

@Bertk Bertk added the duplicate This issue or pull request already exists label Mar 25, 2025
@Bertk
Copy link
Collaborator

Bertk commented Mar 25, 2025

Duplicate of #857, #725

@Bertk
Copy link
Collaborator

Bertk commented Mar 26, 2025

The log files have no additional information but there are similar issues and only reproducible on your systems. This is hard to troubleshoot because individual system have different configuration e.g. virus scanner and so on.

The issues mentioned above have some hints which might help. Please check.

Please also check before the test execution:

dotnet --version
dotnet build-server shutdown

Try also this (from issue 725):

dotnet.exe test MyTests\MyTests.csproj -c debug --collect:"XPlat Code Coverage" --no-build /nodeReuse:false -m:1

@Bertk Bertk removed the needs more info More details are needed label Mar 26, 2025
@alex-ntk
Copy link
Author

Tried:

Image

@alex-ntk
Copy link
Author

I've tried it in Docker and it worked ok:

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /
ENV PATH="$PATH:/root/.dotnet/tools"  
RUN apt-get update
RUN apt-get upgrade -y
RUN dotnet tool install --global coverlet.console
RUN mkdir MySolution
WORKDIR /MySolution
RUN dotnet new classlib -n MyLibrary
RUN dotnet new console -n Collector
WORKDIR /MySolution/Collector
RUN dotnet add reference ../MyLibrary/MyLibrary.csproj
WORKDIR /MySolution
RUN dotnet new nunit -n MyTests
WORKDIR /MySolution/MyTests
RUN dotnet add reference ../Collector/Collector.csproj
WORKDIR /MySolution
RUN dotnet test MyTests/MyTests.csproj --collect:"XPlat Code Coverage"

@alex-ntk
Copy link
Author

alex-ntk commented Mar 26, 2025

FYI, I've just tested this:

  1. Created file Test1.ps1:

    $name = "Collector"
    
    mkdir MySolution
    cd MySolution
    dotnet new classlib -n MyLibrary
    dotnet new console -n $name
    cd $name
    dotnet add reference ../MyLibrary/MyLibrary.csproj
    cd ..
    dotnet new nunit -n MyTests
    cd MyTests
    dotnet add reference ../$name/$name.csproj
    cd ..
    dotnet test MyTests/MyTests.csproj --collect:"XPlat Code Coverage"
  2. Run it, got the error.

  3. Replaced $name = "Collector" with $name = "Collector1". Deleted the "MySolution" folder.

  4. Run again, no error.

🤷

@Bertk
Copy link
Collaborator

Bertk commented Mar 26, 2025

@alex-ntk
Copy link
Author

It looks I can put one more example into my collection. It will stay just just between the fork-bomb and date-bomb.

@Bertk
Copy link
Collaborator

Bertk commented Mar 26, 2025

I used your steps without any modification and could reproduce the error. The problem is definitely the file name "Collector.dll" on Windows platform.

I also created a dev container and the test created a valid coverage result.

Image

coverlet-issue-1739_0.zip

@Bertk
Copy link
Collaborator

Bertk commented Mar 27, 2025

DataCollector Assembly Naming Convention

When test platform is looking for a DataCollector it will likely need to examine many
assemblies. As an optimization, test platform will only look at distinctly named
assemblies, specifically, a DataCollector must follow the naming convention
*collector.dll. By enforcing such a naming convention, test platform can speed up
locating a DataCollector assembly. Once located, test platform will load the data
collector for the entire run.

##Summary
coverlet.collector is a DataCollector assembly and will be loaded from test platform before test execution. During this phase all other *collector.dll assemblies will be loaded as well. This includes Collector.dll assembly and coverlet.collector fails to instrument the assembly because test platform is using the Collector.dll.

This assembly name problem cannot be resolved by coverlet because the limitation is introduced with DataCollector Assembly Naming Convention.

@Bertk Bertk added as-designed Expected behaviour and removed duplicate This issue or pull request already exists waiting for customer Waiting for customer action labels Mar 27, 2025
@alex-ntk
Copy link
Author

I cannot find any workaround for this so I suppose I should just not name the assembly this way. Thank you for finding this. This is very useful.

@alex-ntk
Copy link
Author

One question through: can coverlet instrument assembly before vstest loads/locks it and do modify it afterwards? This way it still can be processed and there will be no error, right?

@Bertk
Copy link
Collaborator

Bertk commented Mar 27, 2025

One question through: can coverlet instrument assembly before vstest loads/locks it and do modify it afterwards? This way it still can be processed and there will be no error, right?

Please use coverlet.msbuild nuget apckage 6.0.4.

@alex-ntk
Copy link
Author

Um... I'm not sure how to do that. I've tried:

mkdir MySolution
cd MySolution
dotnet new classlib -n MyLibrary
dotnet new console -n Collector
cd Collector
dotnet add reference ../MyLibrary/MyLibrary.csproj
cd ..
dotnet new nunit -n MyTests
cd MyTests
dotnet add reference ../Collector/Collector.csproj
dotnet add package coverlet.msbuild --version 6.0.4
cd ..
dotnet.exe test MyTests\MyTests.csproj --collect:"XPlat Code Coverage"

but it does not seem to make any difference.

@Bertk
Copy link
Collaborator

Bertk commented Mar 27, 2025

One question through: can coverlet instrument assembly before vstest loads/locks it and do modify it afterwards? This way it still can be processed and there will be no error, right?

Please use coverlet.msbuild nuget apckage 6.0.4.

The first link is the integration description. Here you find the integration documentation for coverlet.msbuild.

dotnet test /p:CollectCoverage=true

@alex-ntk
Copy link
Author

We finally made it work. Thank you for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
as-designed Expected behaviour
Projects
None yet
Development

No branches or pull requests

2 participants