automate offload, part 2 - clang calls #149202

ZuseZ4 · 2025-11-22T07:01:40Z

This automates steps 2+3 (the clang invocations) of the rust offload usage pipeline.
Now all that remains is a clang-linker-invocation after this step.

r? oli-obk

ZuseZ4 · 2025-12-09T14:22:17Z

compiler/rustc_codegen_llvm/src/back/write.rs

+            let host_out_c = path_to_c_string(device_pathbuf.as_path());
+
+            // 2) Finalize host: lib.bc + host.out -> host.offload.o (host TM created in C++)
+            let llmod2 = llvm::LLVMCloneModule(module.module_llvm.llmod());


My feeling here is that embedding a device artifact into a host module would break incremental compilation, so I create a copy. It's likely not great for compile times, but better correct than fast. Also at least amd is currently using build-std, so compile times are a lost cause anyway.

we could do the loading in a query in the future, then it is tracked in incremental just like include_bytes could be

ZuseZ4 · 2025-12-09T14:23:34Z

compiler/rustc_llvm/llvm-wrapper/RustWrapper.cpp


+Expected<std::unique_ptr<Module>>
+loadHostModuleFromBitcode(LLVMContext &Ctx, StringRef LibBCPath) {
+  auto MBOrErr = MemoryBuffer::getFile(LibBCPath);


We don't have MemoryBuffer, but ModuleBuffer exposed in our RustWrappers. However, they don't seem to be the same. I had some issues exposing MemoryBuffers, especially a constructor for them in Rust, so I left 25 lines of C++.

ZuseZ4 · 2025-12-09T14:25:24Z

compiler/rustc_session/src/options.rs

+            let variant = match key {
+                "Host" => {
+                    if let Some(p) = arg {
+                        Offload::Host(p.to_string())


I could already do the argument verification here - E.g. is this an absolute path, does it point to a file called host.out, and does the full path with file exist. However, we don't have the error contexts here, so I left them for rustc_cg_llvm, even if that's a bit late.

This would have to happen later. We may want to do it earlier than the backend at some point. For now this is fine

ZuseZ4 · 2025-12-09T14:25:38Z

src/doc/rustc-dev-guide/src/offload/usage.md

-"clang-21" "-cc1as" "-triple" "x86_64-unknown-linux-gnu" "-filetype" "obj" "-main-file-name" "lib.rs" "-target-cpu" "x86-64" "-mrelocation-model" "pic" "-o" "host.o" "host.s"
-
-"clang-linker-wrapper" "--should-extract=gfx90a" "--device-compiler=amdgcn-amd-amdhsa=-g" "--device-compiler=amdgcn-amd-amdhsa=-save-temps=cwd" "--device-linker=amdgcn-amd-amdhsa=-lompdevice" "--host-triple=x86_64-unknown-linux-gnu" "--save-temps" "--linker-path=/ABSOlUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/lld/bin/ld.lld" "--hash-style=gnu" "--eh-frame-hdr" "-m" "elf_x86_64" "-pie" "-dynamic-linker" "/lib64/ld-linux-x86-64.so.2" "-o" "bare" "/lib/../lib64/Scrt1.o" "/lib/../lib64/crti.o" "/ABSOLUTE_PATH_TO/crtbeginS.o" "-L/ABSOLUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/llvm/bin/../lib/x86_64-unknown-linux-gnu" "-L/ABSOLUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/llvm/lib/clang/21/lib/x86_64-unknown-linux-gnu" "-L/lib/../lib64" "-L/usr/lib64" "-L/lib" "-L/usr/lib" "host.o" "-lstdc++" "-lm" "-lomp" "-lomptarget" "-L/ABSOLUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/llvm/lib" "-lgcc_s" "-lgcc" "-lpthread" "-lc" "-lgcc_s" "-lgcc" "/ABSOLUTE_PATH_TO/crtendS.o" "/lib/../lib64/crtn.o"
+"clang-linker-wrapper" "--should-extract=gfx90a" "--device-compiler=amdgcn-amd-amdhsa=-g" "--device-compiler=amdgcn-amd-amdhsa=-save-temps=cwd" "--device-linker=amdgcn-amd-amdhsa=-lompdevice" "--host-triple=x86_64-unknown-linux-gnu" "--save-temps" "--linker-path=/ABSOlUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/lld/bin/ld.lld" "--hash-style=gnu" "--eh-frame-hdr" "-m" "elf_x86_64" "-pie" "-dynamic-linker" "/lib64/ld-linux-x86-64.so.2" "-o" "bare" "/lib/../lib64/Scrt1.o" "/lib/../lib64/crti.o" "/ABSOLUTE_PATH_TO/crtbeginS.o" "-L/ABSOLUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/llvm/bin/../lib/x86_64-unknown-linux-gnu" "-L/ABSOLUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/llvm/lib/clang/21/lib/x86_64-unknown-linux-gnu" "-L/lib/../lib64" "-L/usr/lib64" "-L/lib" "-L/usr/lib" "target/<GPU_DIR>/release/host.o" "-lstdc++" "-lm" "-lomp" "-lomptarget" "-L/ABSOLUTE_PATH_TO/rust/build/x86_64-unknown-linux-gnu/llvm/lib" "-lgcc_s" "-lgcc" "-lpthread" "-lc" "-lgcc_s" "-lgcc" "/ABSOLUTE_PATH_TO/crtendS.o" "/lib/../lib64/crtn.o"


I can't wait to get rid of this one too.

ZuseZ4 · 2025-12-09T14:27:23Z

compiler/rustc_codegen_llvm/src/back/write.rs

+        {
+            let device_pathbuf = PathBuf::from(device_path);
+            if device_pathbuf.is_relative() {
+                panic!("Absolute path is needed");


I'm currently creating proper errors for all panic/assertions.

ZuseZ4 · 2025-12-09T14:30:07Z

r? @oli-obk

I got the c++ code down to 25 lines! Also Rustc does the new writing to file, as you suggested in your last PR.
I left some comments, ~~and I'm just writing a follow-up commit to replace the panics/asserts with proper errors.~~ Done.
Any other thoughts?

rustbot · 2025-12-09T14:30:11Z

oli-obk is not on the review rotation at the moment.
They may take a while to respond.

rustbot · 2025-12-09T14:30:14Z

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @jieyouxu, @Kobzol, @tshepang

oli-obk · 2025-12-09T15:20:52Z

compiler/rustc_codegen_llvm/src/back/write.rs

+            let host_out_c = path_to_c_string(device_pathbuf.as_path());
+
+            // 2) Finalize host: lib.bc + host.out -> host.offload.o (host TM created in C++)
+            let llmod2 = llvm::LLVMCloneModule(module.module_llvm.llmod());


we could do the loading in a query in the future, then it is tracked in incremental just like include_bytes could be

rustbot · 2025-12-10T02:06:20Z

Requested reviewer is already assigned to this pull request.

Please choose another assignee.

ZuseZ4 mentioned this pull request Nov 22, 2025

automate gpu offloading - part 1 #149170

Merged