Rustc fails to compile a program with ThinLTO and split-debuginfo = "packed"

[zamazan4ik]

Hi!

A person on Reddit posted a strange build error when ThinLTO is enabled. I performed additional build tests in different configurations and I can confirm the error on current Rustc compilers (see the versions below). My environment is Fedora 41 + AMD Ryzen 5900x (x86-64). The test project is https://github.com/anza-xyz/agave on the master branch with 144925eda5eba98ef28a47a659be68b93211cdb2. The test command is cargo +stable test --profile release-with-debug/cargo +nightly test --profile release-with-debug.

The original report is when ThinLTO is enabled, some binaries fail to be built with the duplicate split compilation unit error. I performed the build in multiple configurations and here are my results:

Build ok:

[profile.release-with-debug]
inherits = "release"
debug = true
#split-debuginfo = "packed"
lto = "thin"

Build ok:

[profile.release-with-debug]
inherits = "release"
debug = true
split-debuginfo = "packed"
#lto = "thin"

Build ok:

[profile.release-with-debug]
inherits = "release"
debug = true
split-debuginfo = "packed"
lto = "fat"

Build ok:

[profile.release-with-debug]
inherits = "release"
debug = true
split-debuginfo = "unpacked"
lto = "thin"

Build fails:

[profile.release-with-debug]
inherits = "release"
debug = true
split-debuginfo = "packed"
lto = "thin"

Build fails:

[profile.release-with-debug]
inherits = "release"
debug = true
split-debuginfo = "packed"
lto = "thin"
codegen-units = 1

According to the tests, the buggy is only the combination of ThinLTO and split-debuginfo = "packed". Disabling one of these options or enabling Fat LTO instead of ThinLTO resolves the issue.

I expected to see this happen: the build with lto = "thin" + split-debuginfo = "packed" is successful

Instead, this happened: the build fails with the duplicate split compilation unit error

Meta

The issue is reproduced on both Rustc versions: stable and current nightly

rustc --version --verbose:

rustc 1.82.0 (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: x86_64-unknown-linux-gnu
release: 1.82.0
LLVM version: 19.1.1

rustc +nightly --version --verbose

rustc 1.84.0-nightly (fbab78289 2024-11-04)
binary: rustc
commit-hash: fbab78289dd8c6e8860034e0048cfb538f217700
commit-date: 2024-11-04
host: x86_64-unknown-linux-gnu
release: 1.84.0-nightly
LLVM version: 19.1.3

Providing RUST_BACKTRACE=1 doesn't give more information so no backtraces here.

[workingjubilee]

@zamazan4ik The build error was deleted from the Reddit post, so we do not have a record of the complete error, it seems?

[workingjubilee]

Actual errors:

   Compiling tikv-jemallocator v0.4.1
error: duplicate split compilation unit (0x10ff6e7b9c803596)

error: could not compile `agave-transaction-view` (lib test) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
error: duplicate split compilation unit (0xcda36cd0a109c421)

error: could not compile `solana-zk-keygen` (bin "solana-zk-keygen" test) due to 1 previous error
error: could not compile `solana-clap-v3-utils` (lib test) due to 1 previous error
error: could not compile `solana-keygen` (bin "solana-keygen" test) due to 1 previous error
error: could not compile `solana-sdk` (lib test) due to 1 previous error
error: could not compile `solana-program` (lib test) due to 1 previous error

[workingjubilee]

If anyone wants this worked on, then ideally we'd minimize away at least the parts of this repo that aren't necessary to reproduce the error.

I see a lot of cdylibs in the repo, so it seems possible this bug is a dupe of rust-lang/thorin#26 though I can't say for sure.

[workingjubilee]

cc @ksolana @willhickey Thought you might want to know.

[jobafr]

I'm running into this too and have produced a minimal example:
https://github.com/jobafr/repro-rust-132677

It's a crate with a bin and a lib (cdylib) and a function in each. The bin imports the lib and calls a function from there (the error goes away if it doesn't).

The error is the same on stable (1.84), nightly (rustc 1.86.0-nightly (99db2737c 2025-01-16)) and whatever old stable i was using before trying to reproduce this.