`build_from_source` makes assumptions on internal Cargo details, breaking `cargo-miri` with new `build-dir` features

[epage]

Original report at rust-lang/cargo#14125 (comment)

miri (13e8790949 2025-05-04) fails to run with

$ cargo miri clean
$ cargo miri test
Preparing a sysroot for Miri (target: x86_64-unknown-linux-gnu)... fatal error: failed to build sysroot: failed to read cargo out dir

Caused by:
    No such file or directory (os error 2)

when

[build]
build-dir = "{cargo-cache-home}/build/{workspace-path-hash}"

[unstable]
build-dir=true

is set in ~/.cargo/config.toml.

After disabling these lines, miri can create the sysroot and works when these lines are reenabled.

The problem is

rustc-build-sysroot/src/lib.rs

Lines 513 to 516 in 93fd0c1

	let out_dir = build_target_dir
	.join(&target_name)
	.join(DEFAULT_SYSROOT_PROFILE)
	.join("deps");

According to the Cargo docs

Cargo also creates several other directories and files needed for the build process. Their layout is considered internal to Cargo, and is subject to change. Some of these directories are:

target/debug/deps/

With rust-lang/cargo#14125 (docs), users will be allowed to move deps and many other directories out of target-dir into their `build-dir of choice.

Our expectation is that we will eventually change the default location of build-dir from target-dir to {cargo-cache-home}/build/{workspace-path-hash}.

[RalfJung]

Thanks for bringing this up!

We need some way to identify where the build results (including all the rlibs for the dependencies) have been put so that we can copy them to the sysroot. Is there a "less cursed" way of doing that? We don't have to support arbitrary user's choice; we set up our own little workspace for the build anyway so we can configure that to place the build results in a particular location.

Would it be enough to just set CARGO_BUILD_BUILD_DIR to a fixed value?

[epage]

Providing your own CARGO_BUILD_BUILD_DIR would ensure you don't rely on the users or the systems.

This is also prep work for reorganizing the content within the build directory (rust-lang/cargo#15010).

We need some way to identify where the build results (including all the rlibs for the dependencies) have been put so that we can copy them to the sysroot.

Unsure how officially this is supported but if you specify all packages, including dependencies, with --package, then their rlibs will be uplifted into the target-dir.

[RalfJung]

Unsure how officially this is supported but if you specify all packages, including dependencies, with --package, then their rlibs will be uplifted into the target-dir.

We can't hard-code the dependencies of std, so that's not going to work.

rustc bootstrap (./x) must have a similar problem when it prepares the sysroot, but I don't know what they are doing. I assume this is currently equally broken by setting build-dir.

[RalfJung]

Providing your own CARGO_BUILD_BUILD_DIR would ensure you don't rely on the users or the systems.

We have already set CARGO_TARGET_DIR to force cargo to put the files into a given location, I guess we have to set even more env vars to all the same value to overwrite all the ways in which users could screw us by moving things to a different location. ;)