Merge pull request #1365 from NiklasGustafsson/main

Moving to libtorch 2.4.0

Author: NiklasGustafsson

DEVGUIDE.md

@@ -52,7 +52,7 @@ Build with
 
 ## Packages
 
-An ephemeral feed of packages from Azure DevOps CI is available for those 
+An ephemeral feed of packages from Azure DevOps CI is available for those
 
 * View link: https://dotnet.visualstudio.com/TorchSharp/_packaging?_a=feed&feed=SignedPackages
 * Nuget feed: https://dotnet.pkgs.visualstudio.com/TorchSharp/_packaging/SignedPackages/nuget/v3/index.json
@@ -77,7 +77,7 @@ To change the TorchSharp package version update this [file](https://github.com/d
 The TorchSharp package is pushed to nuget.org via Azure DevOps CI release pipeline.  Assuming you're not building or updating the LibTorch packages
 (`BuildLibTorchPackages` is `false` in [azure-pipelines.yml](azure-pipelines.yml)) this is pretty simple once you have the permissions:
 
-1. Update the version number in [./build/BranchInfo.props](./build/BranchInfo.props) and in the [Release Notes](./RELEASENOTES.md) file and then submit a PR. 
+1. Update the version number in [./build/BranchInfo.props](./build/BranchInfo.props) and in the [Release Notes](./RELEASENOTES.md) file and then submit a PR.
 
    Updating the major or minor version number should only be done after a discussion with repo admins. The patch number should be incremented by one each release and set to zero after a change to the major or minor version.
 2. Integrate code to main and wait for CI to process
@@ -149,7 +149,7 @@ For this reason, we do the following
 This project grabs LibTorch and makes a C API wrapper for it, then calls these from C#. When updating to a newer
 version of PyTorch then quite a lot of careful work needs to be done.
 
-0. Make sure you have plenty of disk space, e.g. 15GB. 
+0. Make sure you have plenty of disk space, e.g. 15GB.
 
 1. Clean and reset to main
 
@@ -163,7 +163,7 @@ version of PyTorch then quite a lot of careful work needs to be done.
        https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.2.0%2Bcpu.zip
 
    Don't download anything yet, or manually. The downloads are acquired automatically in step 2.
-   
+
    To update the version, update this in [Dependencies.props](build/Dependencies.props):
 
        <LibTorchVersion>2.2.0</LibTorchVersion>
@@ -179,8 +179,8 @@ version of PyTorch then quite a lot of careful work needs to be done.
 On Windows:
 
         dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
-        dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=mac /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
-        dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true 
+        dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=mac /p:TargetArchitecture=arm64 /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
+        dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
         dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Debug /t:Build /p:IncludeLibTorchCpuPackages=true
 
         dotnet build src\Redist\libtorch-cuda-12.1\libtorch-cuda-12.1.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCudaPackages=true
@@ -190,8 +190,8 @@ On Windows:
 On Linux / Mac:
 
         dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
-        dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=mac /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
-        dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true 
+        dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=mac /p:TargetArchitecture=arm64 /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
+        dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
         dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Debug /t:Build /p:IncludeLibTorchCpuPackages=true
 
         dotnet build src/Redist/libtorch-cuda-12.1/libtorch-cuda-12.1.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCudaPackages=true
@@ -213,7 +213,7 @@ On Linux / Mac:
        dir bin\obj\x64.Release\libtorch-cpu\libtorch-cxx11-abi-shared-with-deps-2.2.0cpu\libtorch\lib\*.so*
 
    You may also need to precisely refactor the binaries into multiple parts so each package ends up under ~300MB. Before release 2.2.0 of libtorch, this really only affected the CUDA packagages, but it is now also affecting the CPU packages on Linux and OSX. Windows CPU is still small enough to be contained in just one package. The NuGet gallery does not allow packages larger than 250MB, so if files are 300MB, after compression, they are likely to be smaller than 250MB. However, you have to look out: if the compression is poor, then packages may end up larger. Note that it is 250 million
-   bytes that is the limit, **not** 250*1024*1024. In other words, it is 250 MB, not 250 MiB. Note that Windows Explorer will show file sizes in KiB, not thousands of bytes. Use 'dir' from a CMD window to get the exact size in bytes for each file. For example -- the file `libtorch_cpu.so` shows up as 511,872 KB in Windows Explorer, but 524,156,144 bytes in CMD. The 2.4% difference can be significant. Getting the partitioning right requires precision. 
+   bytes that is the limit, **not** 250*1024*1024. In other words, it is 250 MB, not 250 MiB. Note that Windows Explorer will show file sizes in KiB, not thousands of bytes. Use 'dir' from a CMD window to get the exact size in bytes for each file. For example -- the file `libtorch_cpu.so` shows up as 511,872 KB in Windows Explorer, but 524,156,144 bytes in CMD. The 2.4% difference can be significant. Getting the partitioning right requires precision.
 
    If the combined size of the files going into a part is smaller than 250MB, then everything is fine, and there is no need to split the part. It can be singular. If that is not the case, then the part should be fragmented into two or more parts that are linked together by their names.
 
@@ -229,12 +229,12 @@ On Linux / Mac:
    They must all be called either 'primary,' which should be the first fragment, or 'fragmentN' where 'N' is the ordinal number of the fragment, starting with '1'. The current logic allows for as many as 10 non-primary fragments. If more are needed, the code in [FileRestitcher.cs](pkg/FileRestitcher/FileRestitcher/FileRestitcher.cs) and [RestitchPackage.targets](pkg/common/RestitchPackage.targets) needs to be updated. Note that the size of each fragment is expressed in bytes, and that fragment start must be
    the sum of the size of all previous fragments. A '-1' should be used for the last fragment (and only for the last fragment): it means that the fragment size will be based on how much there is still left of the file.
 
-   Each part, whether singular or fragmented, should have its own .nupkgproj file in its own folder under pkg. The folder and file should have the same name as the part. If you need to add new fragments, it is straightforward to just copy an existing fragment folder and rename it as well as the project file to the new fragment. 
-   
+   Each part, whether singular or fragmented, should have its own .nupkgproj file in its own folder under pkg. The folder and file should have the same name as the part. If you need to add new fragments, it is straightforward to just copy an existing fragment folder and rename it as well as the project file to the new fragment.
+
    __Important:__
 
-   If you must fragment a previously singular part, it is best to rename the existing folder and file to '-fragment1' and then copy a '-primary' folder and rename with the right part name. This is because the primary .nupkgproj files look different from others. 
-   
+   If you must fragment a previously singular part, it is best to rename the existing folder and file to '-fragment1' and then copy a '-primary' folder and rename with the right part name. This is because the primary .nupkgproj files look different from others.
+
    Specifically, they include different build targets:
 
     ```xml
@@ -295,8 +295,8 @@ On Linux / Mac:
 
        <LibTorchPackageVersion>2.0.1.1</LibTorchPackageVersion>
 
-       dotnet pack -c Release -v:n /p:SkipNative=true /p:SkipTests=true /p:IncludeTorchSharpPackage=true /p:IncludeLibTorchCpuPackages=true /p:IncludeLibTorchCudaPackages=true 
-       dotnet pack -c Release -v:n /p:SkipNative=true /p:SkipTests=true /p:TargetOS=linux /p:IncludeTorchSharpPackage=true /p:IncludeLibTorchCpuPackages=true /p:IncludeLibTorchCudaPackages=true        
+       dotnet pack -c Release -v:n /p:SkipNative=true /p:SkipTests=true /p:IncludeTorchSharpPackage=true /p:IncludeLibTorchCpuPackages=true /p:IncludeLibTorchCudaPackages=true
+       dotnet pack -c Release -v:n /p:SkipNative=true /p:SkipTests=true /p:TargetOS=linux /p:IncludeTorchSharpPackage=true /p:IncludeLibTorchCpuPackages=true /p:IncludeLibTorchCudaPackages=true
 
     Once these finish, the output can be found in `bin\packages\Release`. Look at the file sizes -- if anything is larger than 250,000,000 bytes, you need to go back to #3 above and redefine the package contents and fragmentation scheme. It maybe necessary to introduce new fragments.
 

Directory.Build.props

@@ -20,7 +20,8 @@
     <SourceDir>$(RepoRoot)src/</SourceDir>
     <PkgDir>$(RepoRoot)pkg/</PkgDir>
 
-    <LibTorchPackageVersion>2.2.1.1</LibTorchPackageVersion>
+    <LibTorchPackageVersion>2.4.0.0</LibTorchPackageVersion>
+    <LibTorchPackageVersion Condition="'$(TargetOS)' == 'mac' and '$(TargetArchitecture)' == 'x64'">2.2.2.0</LibTorchPackageVersion>
 
     <!-- when building on local machines the massive downloads get placed up one directory -->
     <!-- so we can clean without triggering a re-download-->
@@ -55,7 +56,6 @@
 
     <TargetRuntimeID Condition="'$(TargetOS)' == 'windows'">win-x64</TargetRuntimeID>
     <TargetRuntimeID Condition="'$(TargetOS)' == 'linux'">linux-x64</TargetRuntimeID>
-    <TargetRuntimeID Condition="'$(TargetOS)' == 'mac'">osx-x64</TargetRuntimeID>
     <TargetRuntimeID Condition="'$(TargetPlatform)' == 'mac-arm64'">osx-arm64</TargetRuntimeID>
 
     <TargetRuntimeID Condition="'$(TargetOS)' == 'windows'">win-x64</TargetRuntimeID>
@@ -86,7 +86,8 @@
   <!-- use stable versions for libtorch packages based on LibTorch version number scheme-->
   <!-- we manually update these -->
   <PropertyGroup Condition="'$(MSBuildProjectName.IndexOf(`libtorch-`))' != '-1'">
-    <LibTorchPackageVersion>2.2.1.1</LibTorchPackageVersion>
+    <LibTorchPackageVersion>2.4.0.0</LibTorchPackageVersion>
+    <LibTorchPackageVersion Condition="'$(TargetOS)' == 'mac' and '$(TargetArchitecture)' == 'x64'">2.2.2.0</LibTorchPackageVersion>
     <EnablePackageValidation>false</EnablePackageValidation>
     <VersionPrefix>$(LibTorchPackageVersion)</VersionPrefix>
     <VersionSuffix></VersionSuffix>
@@ -135,17 +136,14 @@
 
   <PropertyGroup>
     <LibTorchArchiveSource>pytorch</LibTorchArchiveSource>
-    <LibTorchArchiveSource Condition="'$(TargetPlatform)' == 'mac-arm64'">conda</LibTorchArchiveSource>
-    <CondaArchivePlatformName Condition="'$(TargetPlatform)' == 'mac-arm64'">osx-arm64</CondaArchivePlatformName>
     <LibTorchCpuArchiveNameSuffix Condition="'$(TargetOS)' != 'mac'">%252Bcpu</LibTorchCpuArchiveNameSuffix>
-    <LibTorchCpuArchiveNameSuffix Condition="'$(LibTorchArchiveSource)' == 'conda'">-py3.10_0</LibTorchCpuArchiveNameSuffix>
     <LibTorchCudaArchiveNameSuffix>%252Bcu$(CudaVersionNoDot)</LibTorchCudaArchiveNameSuffix>
     <LibTorchCpuLocalNameSuffix>cpu</LibTorchCpuLocalNameSuffix>
     <LibTorchCudaLocalNameSuffix>cu$(CudaVersionNoDot)</LibTorchCudaLocalNameSuffix>
     <LibTorchArchiveCoreName Condition="'$(TargetOS)' == 'windows'">libtorch-win-shared-with-deps$(LibTorchDebug)</LibTorchArchiveCoreName>
     <LibTorchArchiveCoreName Condition="'$(TargetOS)' == 'linux'">libtorch-cxx11-abi-shared-with-deps</LibTorchArchiveCoreName>
     <LibTorchArchiveCoreName Condition="'$(TargetOS)' == 'mac'">libtorch-macos-x86_64</LibTorchArchiveCoreName>
-    <LibTorchArchiveCoreName Condition="'$(LibTorchArchiveSource)' == 'conda'">pytorch</LibTorchArchiveCoreName>
+    <LibTorchArchiveCoreName Condition="'$(TargetPlatform)' == 'mac-arm64'">libtorch-macos-arm64</LibTorchArchiveCoreName>
     <LibTorchCpuArchiveBase>$(LibTorchArchiveCoreName)-$(LibTorchVersion)$(LibTorchCpuArchiveNameSuffix)</LibTorchCpuArchiveBase>
     <LibTorchCudaArchiveBase>$(LibTorchArchiveCoreName)-$(LibTorchVersion)$(LibTorchCudaArchiveNameSuffix)</LibTorchCudaArchiveBase>
     <LibTorchCpuLocalBase>$(LibTorchArchiveCoreName)-$(LibTorchVersion)$(LibTorchCpuLocalNameSuffix)</LibTorchCpuLocalBase>

Directory.Build.targets

@@ -24,35 +24,36 @@
 
   <!-- Windows CUDA 11.3 libtorch binary list used for examples and testing -->
   <ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64' and '$(OS)' == 'Windows_NT' and '$(TestUsesLibTorch)' == 'true' and ('$(TestCuda)' == 'true' and '$(SkipCuda)' != 'true') and '$(SkipNative)' != 'true' ">
-    <NativeAssemblyReference Include="c10" Variant="cuda\" />
     <NativeAssemblyReference Include="asmjit" Variant="cuda\" />
+    <NativeAssemblyReference Include="c10" Variant="cuda\" />
     <NativeAssemblyReference Include="c10_cuda" Variant="cuda\" />
     <NativeAssemblyReference Include="caffe2_nvrtc" Variant="cuda\" />
     <NativeAssemblyReference Include="cublas64_12" Variant="cuda\" />
     <NativeAssemblyReference Include="cublasLt64_12" Variant="cuda\" />
     <NativeAssemblyReference Include="cudart64_12" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn_adv_infer64_8" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn_adv_train64_8" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn_cnn_infer64_8" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn_cnn_train64_8" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn_ops_infer64_8" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn_ops_train64_8" Variant="cuda\" />
-    <NativeAssemblyReference Include="cudnn64_8" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_adv64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_cnn64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_engines_precompiled64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_engines_runtime_compiled64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_graph64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_heuristic64_9" Variant="cuda\" />
+    <NativeAssemblyReference Include="cudnn_ops64_9" Variant="cuda\" />
     <NativeAssemblyReference Include="cufft64_11" Variant="cuda\" />
     <NativeAssemblyReference Include="cufftw64_11" Variant="cuda\" />
-    <NativeAssemblyReference Include="curand64_10" Variant="cuda\" />
     <NativeAssemblyReference Include="cupti64_2023.1.1" Variant="cuda\" />
+    <NativeAssemblyReference Include="curand64_10" Variant="cuda\" />
     <NativeAssemblyReference Include="cusolver64_11" Variant="cuda\" />
     <NativeAssemblyReference Include="cusolverMg64_11" Variant="cuda\" />
     <NativeAssemblyReference Include="cusparse64_12" Variant="cuda\" />
     <NativeAssemblyReference Include="fbgemm" Variant="cuda\" />
     <NativeAssemblyReference Include="fbjni" Variant="cuda\" />
     <NativeAssemblyReference Include="libiomp5md" Variant="cuda\" />
     <NativeAssemblyReference Include="libiompstubs5md" Variant="cuda\" />
-    <NativeAssemblyReference Include="nvrtc64_120_0" Variant="cuda\" />
-    <NativeAssemblyReference Include="nvrtc-builtins64_121" Variant="cuda\" />
-    <NativeAssemblyReference Include="nvToolsExt64_1" Variant="cuda\" />
     <NativeAssemblyReference Include="nvJitLink_120_0" Variant="cuda\" />
+    <NativeAssemblyReference Include="nvToolsExt64_1" Variant="cuda\" />
+    <NativeAssemblyReference Include="nvrtc-builtins64_121" Variant="cuda\" />
+    <NativeAssemblyReference Include="nvrtc64_120_0" Variant="cuda\" />
     <NativeAssemblyReference Include="pytorch_jni" Variant="cuda\" />
     <NativeAssemblyReference Include="torch" Variant="cuda\" />
     <NativeAssemblyReference Include="torch_cpu" Variant="cuda\" />
@@ -81,10 +82,14 @@
 	<!-- Mac arm64 libtorch binary list used for examples and testing -->
   <ItemGroup Condition="'$(NativeTargetArchitecture)' == 'arm64'and $([MSBuild]::IsOSPlatform('osx')) and '$(TestUsesLibTorch)' == 'true'  and '$(SkipNative)' != 'true' ">
     <NativeAssemblyReference Include="c10" />
+    <NativeAssemblyReference Include="fbjni" />
+    <NativeAssemblyReference Include="omp" />
+    <NativeAssemblyReference Include="pytorch_jni" />
     <NativeAssemblyReference Include="shm" />
     <NativeAssemblyReference Include="torch" />
     <NativeAssemblyReference Include="torch_cpu" />
     <NativeAssemblyReference Include="torch_global_deps" />
+    <NativeAssemblyReference Include="torch_python" />
   </ItemGroup>
 
   <!-- Linux CPU libtorch binary list used for examples and testing -->

README.md

@@ -11,9 +11,10 @@
 <br/>
 Please check the [Release Notes](RELEASENOTES.md) file for news on what's been updated in each new release.
 
-__TorchSharp is now in the .NET Foundation!__
 
-If you are using TorchSharp from NuGet, you should be using a version >= 0.98.3 of TorchSharp, and >= 1.12.0 of the libtorch-xxx redistributable packages. We recommend using one of the 'bundled' packages: TorchSharp-cpu, TorchSharp-cuda-windows, or TorchSharp-cuda-linux. They will pull in the right LibTorch backends.
+__TorchSharp no longer supports MacOS on Intel hardware.__
+
+With libtorch release 2.4.0, Intel HW support was deprecated for libtorch. This means that the last version of TorchSharp to work on Intel Macintosh hardware is 0.102.8. Starting with 0.103.0, only Macs based on Apple Silicon are supported.
 
 __TorchSharp examples has their own home!__
 
@@ -105,7 +106,7 @@ Otherwise, you also need one of the LibTorch backend packages: https://www.nuget
 
 * `libtorch-cpu-win-x64` (CPU, Windows)
 
-* `libtorch-cpu-osx-x64` (CPU, OSX)
+* `libtorch-cpu-osx-arm64` (CPU, OSX)
 
 * `libtorch-cpu` (CPU, references all three, larger download but simpler)
 

RELEASENOTES.md

@@ -2,6 +2,10 @@
 
 Releases, starting with 9/2/2021, are listed with the most recent release at the top.
 
+# NuGet Version 0.103.0
+
+Move to libtorch 2.4.0.
+
 # NuGet Version 0.102.8
 
 __Bug Fixes__: