Add support for building CoreCLR for MacCatalyst/iOS simulator #109928

filipnavara · 2024-11-18T15:29:35Z

Re-open and rebase of #98127

Build instructions:

Build the runtime pack and tools: ./build.sh clr+clr.runtime+libs+packs -os [iossimulator/maccatalyst] -arch [x64/arm64] -cross -c Release
Run the sample app: ./dotnet.sh publish src/mono/sample/iOS/Program.csproj -c Release /p:TargetOS=maccatalyst /p:TargetArchitecture=arm64 /p:DeployAndRun=true /p:UseMonoRuntime=false /p:RunAOTCompilation=false /p:MonoForceInterpreter=false

Related work:

Counter-part Xamarin changes needed to build full workload and run MAUI apps: [net10.0] Enable CoreCLR for all platforms. macios#21627 / https://github.com/xamarin/xamarin-macios/compare/net8.0...filipnavara:xamarin-macios:net8.0-coreclr-ioslike?expand=1
SDK would need to be updated with RIDs in KnownRuntimeFramework to use this
tvOS support is dropped in this version of the PR. The OS doesn't expose Mach exceptions and it would need larger modification in the runtime to add the support.
We don't use the new JIT API (Investigate use of new Apple JIT APIs #108423). That's orthogonal problem but would be a necessity to support MacCatalyst/arm64 in production.

Notably, this doesn't include CI scripts to build this or the runtime packs. I am open to suggestions on how to better split this into more digestible/reviewable chunks.

src/coreclr/hosts/inc/coreclrhost.h

src/coreclr/pal/src/CMakeLists.txt

src/coreclr/pal/src/include/pal/dbgmsg.h

src/coreclr/pal/src/map/virtual.cpp

src/native/corehost/hostpolicy/hostpolicy_context.cpp

src/native/libs/System.Native/entrypoints.c

src/native/libs/System.Security.Cryptography.Native.Apple/entrypoints.c

janvorli · 2024-11-19T00:09:04Z

src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt

@@ -228,6 +228,9 @@ endif(CLR_CMAKE_TARGET_WIN32)

 # add the install targets
 install_clr(TARGETS coreclr DESTINATIONS . sharedFramework COMPONENT runtime)
+if(CLR_CMAKE_HOST_MACCATALYST OR CLR_CMAKE_HOST_IOS)
+    install_clr(TARGETS coreclr_static DESTINATIONS . sharedFramework COMPONENT runtime)


Why do we need to install this?

iOS generally restricts linking to static libraries or dynamic frameworks distributed with the app itself. The aim was to include statically built CoreCLR in the runtime pack.

I wonder if that means that the single file is the only deployment mechanism on iOS. If it is the case, then I am not sure what would be the scenario where developers would explicitly use the static version of coreclr.

Linking with a static library typically results in smaller apps, which is why we've always linked Mono statically by default.

Linking dynamically can make the build a little bit faster (from past experience in Xamarin, we never ported this to .NET when we migrated due to time constraints).

The single file is a host statically linked to all the native libraries including coreclr. The scenario when the developers would need static coreclr library would be when they want to use their own host. Thinking about it more, I guess distributing the static coreclr version actually makes sense to enable such scenarios.

Yes, that sounds like what we want.

Note that .dylib won't do (Apple doesn't allow them in iOS apps), each dynamic library has to be made into a .framework (which works).

FWIW Apple has recommended using no more than 6-8 .frameworks because otherwise it affects startup performance.

src/coreclr/hosts/inc/coreclrhost.h

src/coreclr/pal/src/exception/seh-unwind.cpp

src/coreclr/pal/src/map/virtual.cpp

src/coreclr/pal/src/misc/dbgmsg.cpp

src/native/corehost/apphost/static/CMakeLists.txt

src/native/libs/System.Globalization.Native/CMakeLists.txt

src/native/libs/System.Native/entrypoints.c

src/tests/build.proj

ivanpovazan · 2025-01-23T08:51:41Z

/azp run runtime-extra-platforms

azure-pipelines · 2025-01-23T08:52:02Z

Azure Pipelines successfully started running 1 pipeline(s).

kotlarmilos · 2025-01-23T10:58:21Z

/azp run runtime-extra-platforms

azure-pipelines · 2025-01-23T10:58:34Z

Azure Pipelines successfully started running 1 pipeline(s).

filipnavara · 2025-01-23T15:35:55Z

CI has few timeouts and couple of known issues but the rest passed.

matouskozak · 2025-01-23T15:41:27Z

CI has few timeouts and couple of known issues but the rest passed.

I've triggered re-run for the failing ones to hopefully remove any infrastructure noise (the timeouts were on jobs that usually don't time out)

ivanpovazan

LGTM! Thanks a lot for all the hard work!

I resolved all my comments related to build integration files and sample changes as I will cover them in a separate PR as part of: #111745

steveisok

Thanks!

kotlarmilos

LGTM!

kotlarmilos · 2025-01-24T08:43:40Z

/azp run runtime-extra-platforms

azure-pipelines · 2025-01-24T08:43:57Z

Azure Pipelines successfully started running 1 pipeline(s).

kotlarmilos · 2025-01-24T09:15:46Z

The official builds have passed.

akoeplinger

LGTM apart from two small comments that can be addressed in a follow-up PR, thanks!

akoeplinger · 2025-01-24T12:12:56Z

src/tasks/AppleAppBuilder/Templates/runtime-coreclr.m

+
+#import "util.h"
+
+#define APPLE_RUNTIME_IDENTIFIER "iossimulator-arm64"


this should use the template string like runtime.m

Good catch. I assume we can handle that as part of the follow-ups tracked in #111745.

akoeplinger · 2025-01-24T12:14:05Z

src/coreclr/pal/src/thread/process.cpp

@@ -2212,6 +2210,9 @@ PROCCreateCrashDump(
    INT cbErrorMessageBuffer,
    bool serialize)
 {
+#if defined(TARGET_IOS)


we should disable this on tvos too

I didn't enable tvOS in this PR. It requires more work and testing due to lack of Mach signal API.

(Previous versions of this PR enabled the tvOS compilation at one point. I may resubmit them separately at some point for a more thorough review. Mixing them with the rest of the iOS bring up made it unreviewable.)

yes I just wanted to call out so we don't forget to update this spot

vitek-karas · 2025-01-24T15:22:46Z

Thanks a lot Filip - this was a lot of work!!!

mikem8361 · 2025-02-02T21:07:56Z

src/coreclr/debug/daccess/CMakeLists.txt

@@ -47,10 +47,10 @@ target_link_libraries(daccess PRIVATE cdacreader_api)

 add_dependencies(daccess eventing_headers)

-if(CLR_CMAKE_HOST_FREEBSD OR CLR_CMAKE_HOST_NETBSD OR CLR_CMAKE_HOST_SUNOS)
+if(CLR_CMAKE_HOST_FREEBSD OR CLR_CMAKE_HOST_NETBSD OR CLR_CMAKE_HOST_SUNOS OR CLR_CMAKE_HOST_APPLE)


@hoyosjs, I think it was this change that broke debugging/diagnostics repo tests on MacOS x64. Adding CLR_CMAKE_HOST_APPLE here will cause the runtime DAC table to be generated in the old "rva" way.

I don't think the change was intentional. I traced it back to the first commit on my previous branch but it could have been a result of some rebase.

Do you want to remove/fix this?

Seems like at some point after the initial patch the conditional compilation of machoreader.cpp was added. This still uses the old CLR_CMAKE_TARGET_OSX condition. I'll send a PR to fix this.

@hoyosjs, I think it was this change that broke debugging/diagnostics repo tests on MacOS x64. Adding CLR_CMAKE_HOST_APPLE here will cause the runtime DAC table to be generated in the old "rva" way.

Thanks Mike for drawing attention to this!

Which pipeline is running these tests?
I am thinking about ways to improve test coverage for PRs like this, so that we can catch such regressions earlier.

These failing tests are part of the diagnostics repo. There is a manual way of running them against a local runtime build, but there currently no way for the runtime repo to use them. There is Jeremy created this issue dotnet/diagnostics#5213 to track this work.

Thanks a lot for the pointers.

mikem8361 · 2025-02-02T21:14:01Z

src/coreclr/dlls/mscordac/CMakeLists.txt

        generate_exports_file(${DEF_SOURCES} ${EXPORTS_FILE})
-    endif(CLR_CMAKE_HOST_OSX OR CLR_CMAKE_HOST_FREEBSD OR CLR_CMAKE_HOST_NETBSD OR CLR_CMAKE_HOST_SUNOS OR CLR_CMAKE_HOST_HAIKU)
+    endif(CLR_CMAKE_HOST_APPLE OR CLR_CMAKE_HOST_FREEBSD OR CLR_CMAKE_HOST_NETBSD OR CLR_CMAKE_HOST_SUNOS OR CLR_CMAKE_HOST_HAIKU)

    if(CORECLR_SET_RPATH AND CLR_CMAKE_HOST_OSX AND CLR_CMAKE_HOST_ARCH_ARM64)


Has CLR_CMAKE_HOST_OSX been completed replaced by CLR_CMAKE_HOST_APPLE? If so, then the RPATH in the DAC isn't be added.

@hoyosjs

Never mind. CLR_CMAKE_HOST_OSX is still defined if not maccatalyst.

The idea is to use same parameters for all Apple platforms (macOS, iOS, tvOS) whenever possible. RPATH may still need some tweaks for each platform due to different bundle structure on macOS/MacCatalyst and iOS/tvOS.

I'll need to do another pass over the CLR_CMAKE_HOST_OSX usages. The RPath ones are likely wrong for some platforms. We don't produce packages for those platforms yet, so the code has never been tested. As for other occurrences - createdump is macOS only (but maybe MacCatalyst too?), guards in test code are likely unnecessary because _DARWIN_C_SOURCE is now set globally (to be verified).

filipnavara requested review from janvorli, AaronRobinsonMSFT, a team, steveisok and vitek-karas as code owners November 18, 2024 15:29

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Nov 18, 2024

dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Nov 18, 2024

AaronRobinsonMSFT reviewed Nov 18, 2024

View reviewed changes

tommcdon requested review from thaystg and mikem8361 November 18, 2024 16:46

jkotas added area-Infrastructure-coreclr and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Nov 18, 2024

matouskozak added os-ios Apple iOS os-maccatalyst MacCatalyst OS and removed os-ios Apple iOS labels Nov 18, 2024

janvorli reviewed Nov 19, 2024

View reviewed changes

This was referenced Nov 19, 2024

chrome-DebuggerTests.GetPropertiesTests timing out #109070

Open

[MT][Firefox] Wasm test failure: System.IO.FileNotFoundException : Could not load file or assembly xunit.assert #109945

Open

filipnavara mentioned this pull request Nov 22, 2024

Remove usage of HP libunwind on Apple platforms #110023

Closed

build-analysis bot mentioned this pull request Nov 29, 2024

Error reported in diagnostic logs. Please examine the log for more details dotnet/dnceng#1928

Open

3 tasks

Remove extra NativeDependencies property (bad merge)

5098d47

ivanpovazan reviewed Jan 22, 2025

View reviewed changes

src/tests/build.proj Outdated Show resolved Hide resolved

This was referenced Jan 23, 2025

[WASI] Sockets - unknown handle index #108726

Open

Test failure: baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.cmd #110173

Open

ivanpovazan mentioned this pull request Jan 23, 2025

[ios/android] Improve internal support for iOS/Android testing and verification #111745

Open

8 tasks

Add back the UseNativeAOTRuntime property to unblock CI

f6aae30

ivanpovazan approved these changes Jan 23, 2025

View reviewed changes

ivanpovazan requested review from akoeplinger and kotlarmilos January 23, 2025 16:54

steveisok approved these changes Jan 23, 2025

View reviewed changes

Merge branch 'main' into coreclr-ioslike-2

7686181

kotlarmilos approved these changes Jan 24, 2025

View reviewed changes

build-analysis bot mentioned this pull request Jan 24, 2025

Infra timeout causing "Unable to obtain kernel buffer" dotnet/dnceng#4547

Open

3 tasks

akoeplinger approved these changes Jan 24, 2025

View reviewed changes

ivanpovazan merged commit 717d3b1 into dotnet:main Jan 24, 2025
217 of 225 checks passed

mikem8361 reviewed Feb 2, 2025

View reviewed changes

filipnavara mentioned this pull request Feb 2, 2025

[daccess] Do not use USE_DAC_TABLE_RVA on Apple platforms #112076

Merged

github-actions bot locked and limited conversation to collaborators Mar 8, 2025

filipnavara deleted the coreclr-ioslike-2 branch April 2, 2025 20:07


		#import "util.h"

		#define APPLE_RUNTIME_IDENTIFIER "iossimulator-arm64"

Add support for building CoreCLR for MacCatalyst/iOS simulator #109928

Add support for building CoreCLR for MacCatalyst/iOS simulator #109928

Conversation

filipnavara commented Nov 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ivanpovazan commented Jan 23, 2025

azure-pipelines bot commented Jan 23, 2025

kotlarmilos commented Jan 23, 2025

azure-pipelines bot commented Jan 23, 2025

filipnavara commented Jan 23, 2025

matouskozak commented Jan 23, 2025

ivanpovazan left a comment

Choose a reason for hiding this comment

steveisok left a comment

Choose a reason for hiding this comment

kotlarmilos left a comment

Choose a reason for hiding this comment

kotlarmilos commented Jan 24, 2025

azure-pipelines bot commented Jan 24, 2025

kotlarmilos commented Jan 24, 2025

akoeplinger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vitek-karas commented Jan 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment