Errors with sentry-native launching with Sentry.AspNetCore on Dockerized Native AOT builds on Linux #4127

ionite34 · 2025-04-21T19:49:18Z

Package

Sentry

.NET Flavor

.NET

.NET Version

9.0.201

OS

Linux

OS Version

alpine 3.21

Development Environment

Other

SDK Version

5.5.1

Self-Hosted Sentry Version

No response

Workload Versions

N/A

UseSentry or SentrySdk.Init call

N/A

Steps to Reproduce

Using any minimal api AOT example, such as:

var builder = WebApplication.CreateSlimBuilder(args);

builder.Services.AddSentry();

var app = builder.Build();

api.MapGet("test", () => "Test Ok");

await app.RunAsync();

Building with this Dockerfile:

FROM mcr.microsoft.com/dotnet/nightly/sdk:9.0.201-alpine3.21-aot AS build
WORKDIR /src

COPY --link . .
RUN dotnet publish -c Release -a linux-musl-x64 -o /app ./Project/Project.csproj
RUN rm /app/*.dbg /app/*.Development.json

FROM mcr.microsoft.com/dotnet/nightly/runtime-deps:9.0.3-alpine3.21-aot AS final
USER $APP_UID
EXPOSE 8080
EXPOSE 8081
ENV ASPNETCORE_URLS=http://+:8080

WORKDIR /app
COPY --link --from=build /app .
ENTRYPOINT ["./Project"]

Expected Result

Should be able to launch normally

Actual Result

Upon hitting endpoints we'll get a 500 error with:

Unhandled exception. System.DllNotFoundException: Unable to load shared library 'sentry-native' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable: 

Error loading shared library sentry-native.so: No such file or directory
Error loading shared library libsentry-native.so: No such file or directory
Error loading shared library sentry-native: No such file or directory
Error loading shared library libsentry-native: No such file or directory

   at System.Runtime.InteropServices.NativeLibrary.LoadLibErrorTracker.Throw(String) + 0x46
   at Internal.Runtime.CompilerHelpers.InteropHelpers.FixupModuleCell(InteropHelpers.ModuleFixupCell*) + 0x139
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvokeSlow(InteropHelpers.MethodFixupCell*) + 0x35
   at Sentry.Native.C.sentry_options_new() + 0x1f
   at Sentry.Native.C.Init(SentryOptions) + 0x1f
   at Sentry.SentrySdk.InitNativeSdk(SentryOptions) + 0x2d
   at Sentry.SentrySdk.InitHub(SentryOptions) + 0x185
   at Sentry.SentrySdk.Init(Action`1) + 0x3a
   at Program.<<Main>$>d__0.MoveNext() + 0xde

Might be related to #4116 and #4102

An option to disable this sentry-native usage might be helpful for now since this is a blocker for us deploying Sentry at all on our Native AOT containers just for request exceptions logging and traces.

We're currently using https://www.nuget.org/packages/Microsoft.ApplicationInsights.aspnetcore on these containers, and despite build time trim warnings it still works fine so far in regards to capturing exceptions and traces.

The text was updated successfully, but these errors were encountered:

bruno-garcia · 2025-04-22T17:45:51Z

Thanks for raising this.

to unblock you, there is an option in MSbuild you can use DisableSentryNative if u set to false stops sentry from adding it.

~~We might rename that to SentryDisableNative but we'll keep it backward compatible at least until version 6.0 of the sdk.~~

Looks like this option was released yet so we'll put a release out asap to unblock you.

Add DisableSentryNative build property #4107

Other than that we need to debug this as this shouldn't be happening.

Are you using a proper nuget dependency? like adding sentry via nuget? is something possibly deleting our native libraries from the obj/bin folders?

bruno-garcia · 2025-04-22T20:34:34Z

We'll ship a release tomorrow either with the option name. Or with the current solution so you can work around this.

But looking in more detail about the issue: /runtime-deps:9.0.3-alpine3.21-aot perhaps we don't have sentry-native in a way that's compatible with this? Is it a musl Linux situation? @vaind might have thoughts

Note the publish args: linux-musl-x64
We bundle these sentry-native static libs:

linux-x64 is there, should that work on musl linux?
Does .NET look at linux-x64 when resolving the directoy? Or does it look at linux-musl-x64 and hence we need to build a new version and add to a new directory? @filipnavara might know this one.

filipnavara · 2025-04-22T20:59:33Z

Does .NET look at linux-x64 when resolving the directoy? Or does it look at linux-musl-x64 and hence we need to build a new version and add to a new directory?

I think you might need to build a new one. Even if it looked for the directory (which can be checked with the RID graph) the native binaries are generally not portable between different libc implementations.

vaind · 2025-04-23T15:45:27Z

We specify the path explicitly, so after adding the build, we'll also need to update this code:

sentry-dotnet/src/Sentry/Platforms/Native/buildTransitive/Sentry.Native.targets

Lines 36 to 41 in 084e3b4

    
           <ItemGroup Condition="'$(FrameworkSupportsNative)' == 'true' and '$(RuntimeIdentifier)' == 'linux-x64'"> 
        
             <DirectPInvoke Include="sentry-native" /> 
        
             <NativeLibrary Include="$(MSBuildThisFileDirectory)..\sentry-native\linux-x64\libsentry-native.a" /> 
        
             <!-- See: https://github.com/dotnet/runtime/issues/97414 --> 
        
             <NativeSystemLibrary Include="curl" /> 
        
           </ItemGroup>

bruno-garcia · 2025-04-23T17:50:44Z

I think we need to make a release asap where we disable sentry-native if we detect RID is linux-musl

bruno-garcia · 2025-04-24T00:44:01Z

Release finally done: https://github.com/getsentry/sentry-dotnet/releases/tag/5.6.0

Sorry took a minute. We had issues with our release automation.
You can disable it with:

<SentryNative>false</SentryNative>

bruno-garcia · 2025-05-14T12:25:52Z

Good news, this got resolved in the native SDK:

support linux musl sentry-native#1232
Thanks @jpnurmi

Now, following his work on:

WIP: fix: support musl on Linux #4182

jpnurmi · 2025-05-14T12:48:32Z

Indeed, sentry-native has been fixed to support musl with all backends. The way sentry-dotnet builds sentry-native with -DSENTRY_BACKEND=inproc -DSENTRY_BUILD_SHARED_LIBS=0 makes it link to static libunwind.a to avoid deployment issues. The rest is about fixing the sentry-dotnet build system and CI to deal with linux-x64 vs. linux-musl-x64.

Edit: Fixes released in sentry-native 0.8.5.

ionite34 · 2025-05-20T19:22:03Z

Release finally done: 5.6.0 (release)

Sorry took a minute. We had issues with our release automation. You can disable it with:

false

Hm, just testing the disabling of sentry native now on 5.7.0, but I don't think it's working for a project depending on

<PackageReference Include="Sentry" Version="5.7.0" />
<PackageReference Include="Sentry.AspNetCore" Version="5.7.0" />

Tried the following:

<PropertyGroup>
    <DisableSentryNative>true</DisableSentryNative>
    <SentryNative>false</SentryNative>
</PropertyGroup>

<ItemGroup>
    <RuntimeHostConfigurationOption Include="Sentry.Native.IsEnabled" Value="false" />
</ItemGroup>

When we add the integration with

builder.Services.AddSentry();

Upon hitting any asp endpoint we still result in:

info: Microsoft.Hosting.Lifetime[14]
2025-05-20T19:15:47.895673863Z info: Microsoft.Hosting.Lifetime[0]
2025-05-20T19:15:47.895676531Z       Application started. Press Ctrl+C to shut down.
2025-05-20T19:15:47.895678952Z info: Microsoft.Hosting.Lifetime[0]
2025-05-20T19:15:47.895681456Z       Hosting environment: Production
2025-05-20T19:15:47.895683713Z info: Microsoft.Hosting.Lifetime[0]
2025-05-20T19:15:47.895686135Z       Content root path: /app
2025-05-20T19:15:50.443714670Z fail: Microsoft.AspNetCore.Server.Kestrel[13]
2025-05-20T19:15:50.443755105Z       Connection id "0HNCNQATQV04U", Request id "0HNCNQATQV04U:00000001": An unhandled exception was thrown by the application.
2025-05-20T19:15:50.443761672Z       System.DllNotFoundException: Unable to load shared library 'sentry-native' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable: 
2025-05-20T19:15:50.443765715Z       Error loading shared library sentry-native.so: No such file or directory
2025-05-20T19:15:50.443768372Z       Error loading shared library libsentry-native.so: No such file or directory
2025-05-20T19:15:50.443770701Z       Error loading shared library sentry-native: No such file or directory
2025-05-20T19:15:50.443772958Z       Error loading shared library libsentry-native: No such file or directory
2025-05-20T19:15:50.443775215Z       
2025-05-20T19:15:50.443777442Z          at System.Runtime.InteropServices.NativeLibrary.LoadLibErrorTracker.Throw(String) + 0x46
2025-05-20T19:15:50.443779709Z          at Internal.Runtime.CompilerHelpers.InteropHelpers.FixupModuleCell(InteropHelpers.ModuleFixupCell*) + 0x139
2025-05-20T19:15:50.443790821Z          at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvokeSlow(InteropHelpers.MethodFixupCell*) + 0x35
2025-05-20T19:15:50.443793335Z          at Sentry.Native.C.sentry_options_new() + 0x1f
2025-05-20T19:15:50.443797552Z          at Sentry.Native.C.Init(SentryOptions) + 0x1f
2025-05-20T19:15:50.443799850Z          at Sentry.SentrySdk.InitNativeSdk(SentryOptions) + 0x2d
2025-05-20T19:15:50.443802159Z          at Sentry.SentrySdk.InitHub(SentryOptions) + 0x176
2025-05-20T19:15:50.443805267Z          at Sentry.Extensions.Logging.Extensions.DependencyInjection.ServiceCollectionExtensions.<>c__0`1.<AddSentry>b__0_3(IServiceProvider c) + 0x41
2025-05-20T19:15:50.443807904Z          at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite, RuntimeResolverContext) + 0xf
2025-05-20T19:15:50.443810274Z          at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite, RuntimeResolverContext) + 0x64

jamescrosswell · 2025-05-21T04:28:06Z

@ionite34 that shouldn't be happening, logically. Setting <SentryNative>false</SentryNative> should result in FrameworkSupportsNative being set to false:

sentry-dotnet/src/Sentry/Platforms/Native/buildTransitive/Sentry.Native.targets

Line 25 in 084e3b4

    
           <FrameworkSupportsNative Condition="'$(SentryNative)' == 'false' or '$(SentryNative)' == 'disable'">false</FrameworkSupportsNative>

And that gets used to determine whether SentryNative is included... e.g. here:

sentry-dotnet/src/Sentry/Platforms/Native/buildTransitive/Sentry.Native.targets

Lines 36 to 41 in 084e3b4

    
           <ItemGroup Condition="'$(FrameworkSupportsNative)' == 'true' and '$(RuntimeIdentifier)' == 'linux-x64'"> 
        
             <DirectPInvoke Include="sentry-native" /> 
        
             <NativeLibrary Include="$(MSBuildThisFileDirectory)..\sentry-native\linux-x64\libsentry-native.a" /> 
        
             <!-- See: https://github.com/dotnet/runtime/issues/97414 --> 
        
             <NativeSystemLibrary Include="curl" /> 
        
           </ItemGroup>

Is it possible the logs that you're seeing are from an older build of your product that is still using an earlier version of the Sentry SDK?

If not, you could try running your build with the /bl parameter to get binary logs, so that you can dig into what's happening in your build. It may be that this property is being overwritten in a Directory.Build.props file or something somewhere else?

getsantry bot added the Waiting for: Product Owner label Apr 21, 2025

github-project-automation bot added this to GDX Apr 21, 2025

getsantry bot added this to GitHub Issues with 👀 3 Apr 21, 2025

getsantry bot moved this to Waiting for: Product Owner in GitHub Issues with 👀 3 Apr 21, 2025

ionite34 changed the title ~~Errors with sentry-native launching with Sentry.AspNetCore on Native AOT builds on Linux~~ Errors with sentry-native launching with Sentry.AspNetCore on Dockerized Native AOT builds on Linux Apr 21, 2025

getsantry bot removed the Waiting for: Product Owner label Apr 22, 2025

getsantry bot removed the status in GitHub Issues with 👀 3 Apr 22, 2025

getsantry bot added the Waiting for: Product Owner label Apr 22, 2025

getsantry bot moved this to Waiting for: Product Owner in GitHub Issues with 👀 3 Apr 22, 2025

getsantry bot removed the Waiting for: Product Owner label Apr 23, 2025

getsantry bot removed the status in GitHub Issues with 👀 3 Apr 23, 2025

bruno-garcia mentioned this issue May 8, 2025

support linux musl getsentry/sentry-native#1232

Closed

jpnurmi mentioned this issue May 12, 2025

WIP: fix: support musl on Linux #4182

Closed

getsantry bot added the Waiting for: Product Owner label May 14, 2025

getsantry bot moved this to Waiting for: Product Owner in GitHub Issues with 👀 3 May 14, 2025

jpnurmi linked a pull request May 14, 2025 that will close this issue

fix: support musl on Linux #4188

Draft

8 tasks

Flash0ver removed the Waiting for: Product Owner label May 15, 2025

getsantry bot removed the status in GitHub Issues with 👀 3 May 15, 2025

Flash0ver added Bug Something isn't working Native labels May 15, 2025

getsantry bot added the Waiting for: Product Owner label May 20, 2025

getsantry bot moved this to Waiting for: Product Owner in GitHub Issues with 👀 3 May 20, 2025

getsantry bot removed the Waiting for: Product Owner label May 21, 2025

getsantry bot removed the status in GitHub Issues with 👀 3 May 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Errors with sentry-native launching with Sentry.AspNetCore on Dockerized Native AOT builds on Linux #4127

Errors with sentry-native launching with Sentry.AspNetCore on Dockerized Native AOT builds on Linux #4127

ionite34 commented Apr 21, 2025 •

edited

Loading

bruno-garcia commented Apr 22, 2025 •

edited

Loading

bruno-garcia commented Apr 22, 2025

filipnavara commented Apr 22, 2025 •

edited

Loading

vaind commented Apr 23, 2025 •

edited

Loading

bruno-garcia commented Apr 23, 2025

bruno-garcia commented Apr 24, 2025

bruno-garcia commented May 14, 2025

jpnurmi commented May 14, 2025 •

edited

Loading

ionite34 commented May 20, 2025

jamescrosswell commented May 21, 2025

Errors with sentry-native launching with Sentry.AspNetCore on Dockerized Native AOT builds on Linux #4127

Errors with sentry-native launching with Sentry.AspNetCore on Dockerized Native AOT builds on Linux #4127

Comments

ionite34 commented Apr 21, 2025 • edited Loading

Package

.NET Flavor

.NET Version

OS

OS Version

Development Environment

SDK Version

Self-Hosted Sentry Version

Workload Versions

UseSentry or SentrySdk.Init call

Steps to Reproduce

Expected Result

Actual Result

bruno-garcia commented Apr 22, 2025 • edited Loading

bruno-garcia commented Apr 22, 2025

filipnavara commented Apr 22, 2025 • edited Loading

vaind commented Apr 23, 2025 • edited Loading

bruno-garcia commented Apr 23, 2025

bruno-garcia commented Apr 24, 2025

bruno-garcia commented May 14, 2025

jpnurmi commented May 14, 2025 • edited Loading

ionite34 commented May 20, 2025

jamescrosswell commented May 21, 2025

ionite34 commented Apr 21, 2025 •

edited

Loading

bruno-garcia commented Apr 22, 2025 •

edited

Loading

filipnavara commented Apr 22, 2025 •

edited

Loading

vaind commented Apr 23, 2025 •

edited

Loading

jpnurmi commented May 14, 2025 •

edited

Loading