ICE: Architecture x86_64 does not support GpuKernel calling convention
#138738
Assignees
Labels
A-ABI
Area: Concerning the application binary interface (ABI)
A-LLVM
Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.
C-bug
Category: This is a bug.
F-abi_ptx
`#![feature(abi_ptx)]`
I-ICE
Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️
S-bug-has-test
Status: This bug is tracked inside the repo by a `known-bug` test.
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
matthiaskrgr
added
C-bug
rustbot
added
the
needs-triage
|
bisects to is #136807 |
|
@workingjubilee looks like you forgot a check |
|
smaller #![feature(abi_ptx)]
fn main() {
let a = unsafe { core::mem::transmute::<usize, extern "ptx-kernel" fn(i32)>(4) }(2);
} |
jieyouxu
added
A-LLVM
matthiaskrgr
added
the
S-bug-has-test
This was referenced Jun 2, 2025
rust-bors bot
added a commit
that referenced
this issue
Jun 6, 2025
Reject `extern "{abi}"` when the target does not support it
## What
Promote [`unsupported_fn_ptr_calling_conventions`] from a warning to a hard error, making sure edge-cases will not escape. We now emit hard errors for every case we would return `Invalid` from `AbiMap::canonize_abi` during AST to HIR lowering. In particular, these architecture-specific ABIs now only compile on their architectures[^1]:
- amdgpu: "gpu-kernel"
- arm: "aapcs", "C-cmse-nonsecure-entry"
- avr: "avr-interrupt", "avr-non-blocking-interrupt"
- msp430: "msp430-interrupt"
- nvptx64: "gpu-kernel", "ptx-kernel"
- riscv32 and riscv64: "riscv-interrupt-machine", "riscv-interrupt-supervisor"
- x86: "thiscall"
- x86 and x86_64: "x86-interrupt"
- x86_64: "sysv64", "win64"
The panoply of ABIs that are logically x86-specific but actually permitted on all Windows targets remain supported on Windows, as they were before. For non-Windows targets they error if the architecture does not match.
Moving the check into AST lowering **is itself a breaking change in rare cases**, above and beyond the cases rustc currently warns about. See "Why or Why Not" for details.
## How
We modify rustc_ast_lowering to prevent unsupported ABIs from leaking through the HIR without being checked for target support. Previously ad-hoc checking on various HIR items required making sure we check every HIR item which could contain an `extern "{abi}"` string. This is a losing proposition compared to gating the lowering itself.
As a consequence, unsupported ABI strings will now hard-error instead of triggering the FCW `unsupported_fn_ptr_calling_conventions`.
However, per #86232 this does cause errors for rare usages of `extern "{abi}"` that were theoretically possible to write in Rust source, without previous warning or error. For instance, trait declarations without impls were never checked. These are the exact kinds of leakages that this new approach prevents.
This differs from the following PRs:
- #141435 is orthogonal, as it adds a new lint for ABIs we have not warned on and are not touched by this PR
- #141877 is subsumed by this, in that this simply cuts out bad functionality instead of adding epicycles for stable code
## Why or Why Not
We already made the decision to issue the `unsupported_fn_ptr_calling_conventions` future compatibility warning. It has warned in dependencies since #135767, which reached stable with Rust 1.87. That was released on 2025 May 17, and it is now June. As we already had erred on these ABI strings in most other positions, and warn on stable for function pointer types, this breakage has had reasonable foreshadowing.
Upgrading the warning to an error addresses a real problem. In some cases the Rust compiler can attempt to actually compute the ABI for calling a function. We could accept this case and compute unsupported ABIs according to some other ABI, silently[^0]. However, this obviously exposes Rust to errors in codegen. We cannot lower directly to the "obvious" ABI and then trust code generators like LLVM to reliably error on these cases, either.
Refactoring the compiler so we could defer more ABI computations would be possible, but seems weakly motivated. Even if we succeeded, we would at minimum risk:
- exposing the "whack-a-mole" problem but "approaching linking" instead of "leaving AST"
- making it harder to reason about functions we *can* lower further
- complicating the compiler for no clear benefit
A deprecation cycle for the edge-cases could be implemented first, but it is not very useful for such marginal cases, like this trait declaration without a definition:
```rust
pub trait UsedToSneakBy {
pub extern "gpu-kernel" fn sneaky();
}
```
Upon any impl, even for provided fn within trait declarations, e.g. `pub extern "gpu-kernel" fn sneaky() {}`, different HIR types were used which would, in fact, get checked. Likewise with anything with function pointers. Thus we would be discussing deprecation cycles for code that is impotent or forewarned[^2].
Implementing a deprecation cycle _is_ possible, but it would likely require emitting multiple of a functionally identical warning or error on code that would not have multiple warnings or errors before. It is also not clear to me we would not find **another**, even more marginal edge-case that slipped through, as "things slip through" is the motivation for checking earlier. Additional effort spent on additional warnings should require committing to a hard limit first.
r? lang
Fixes #86232
Fixes #132430
Fixes #138738
Fixes #142107
[`unsupported_fn_ptr_calling_conventions`]: #130260
[^1]: Some already will not compile, due to reaching ICEs or LLVM errors.
[^0]: We already do this for all `AbiStr` we cannot parse, pretending they are `ExternAbi::Rust`, but we also emit an error to prevent reaching too far into codegen.
[^2]: It actually did appear in two cases in rustc's test suite because we are a collection of Rust edge-cases by the simple fact that we don't care if the code actually runs. These cases were excised in c1db989.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
auto-reduced (treereduce-rust):
original:
Version information
Possibly related line of code:
rust/compiler/rustc_codegen_llvm/src/abi.rs
Lines 674 to 686 in 70237a8
Command:
/home/matthias/.rustup/toolchains/master/bin/rustc -Zcrate-attr=feature(abi_ptx)Program output
@rustbot label +F-abi_ptx
The text was updated successfully, but these errors were encountered: