Support JSON target specs in bootstrap

[ehuss]

JSON target specs were destabilized in #150151 and #151534. However, this broke trying to build rustc itself with a JSON target spec. This is because in a few places bootstrap is manually calling rustc without the ability for the user to provide additional flags (primarily, -Zunstable-options to enable JSON targets).

There's a few different ways to fix this. One would be to change these calls to rustc to include flags provided by the user (such as RUSTFLAGS_NOT_BOOTSTRAP). Just to keep things simple, this PR proposes to just unconditionally pass -Zunstable-options.

Another consideration here is how maintainable this is. A possible improvement here would be to have a function somewhere (BootstrapCommand, TargetSelection, free function) that would handle appropriately adding the --target flag. For example, that's what cargo does in CompileKind::add_target_arg.

I have only tested building the compiler and a few tools like rustdoc. I have not tested doing things like building other tools, running tests, etc.

This would be much easier if there was a Docker image for testing the use case of building rustc with a custom target spec (and even better if that ran in CI).

After the next beta branch, using target JSON specs will become more cumbersome because target specs with the .json extension will now require passing -Zjson-target-spec (from
rust-lang/cargo#16557). This does not affect target specs without the .json extension (such as those from RUST_TARGET_PATH). From my testing, it should be sufficient to pass CARGOFLAGS_NOT_BOOTSTRAP="-Zjson-target-spec". I think that should be fine, since this is not a particularly common use case AFAIK. We could extend bootstrap to auto-detect if the target is a file path, and pass -Zjson-target-spec appropriately. I tried something similar in ehuss@f0bdd35, which could be adapted if desired.

It would be nice if all of this is documented somewhere. https://rustc-dev-guide.rust-lang.org/building/new-target.html does not really say how to build the compiler with a custom json target.

Fixes #151729

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: bootstrap
• bootstrap expanded to 6 candidates
• Random selection from Mark-Simulacrum, clubby789, jieyouxu

[ehuss]

cc @davidtwco

I think this should either be backported to beta, or #150151 reverted on beta. Otherwise 1.94 will not be buildable. I might recommend reverting, so that all of the destabilization changes are in 1.95.

[davidtwco]

LGTM. Happy for #150151 to be reverted on beta.

View changes since this review

[jieyouxu]

Thanks.

• On the bootstrap part, tiny nit re. backlinking to a cleanup issue #152709.
• Compiler wise, I would lean towards reverting #150151 on beta so the destabilization changes can land together (IIUC).

You can r= david and me after the nit.

View changes since this review

[ehuss]

@jieyouxu I realized I made a mistake. This was missing RUSTC_BOOTSTRAP. This would otherwise fail when the channel is set to stable. Do the new changes look correct to you?

[ehuss]

Posted a beta revert at #152721.

[jieyouxu]

@jieyouxu I realized I made a mistake. This was missing RUSTC_BOOTSTRAP. This would otherwise fail when the channel is set to stable. Do the new changes look correct to you?

This seems okay.

Thanks
@bors r+ rollup=never p=1

[rust-bors]

📌 Commit b4e645f has been approved by jieyouxu

It is now in the queue for this repository.

[jieyouxu]

@bors r=davidtwco,jieyouxu

[rust-bors]

📌 Commit b4e645f has been approved by davidtwco,jieyouxu

It is now in the queue for this repository.

[rust-bors]

☀️ Test successful - CI
Approved by: davidtwco,jieyouxu
Duration: 4h 1m 44s
Pushing 8387095 to main...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing dfbfbf7 (parent) -> 8387095 (this PR)

Test differences

Show 2 test diffs

2 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 8387095803f21a256a9a772ac1f9b41ed4d5aa0a --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. dist-apple-various: 2h 11m -> 1h 47m (-18.1%)
2. i686-gnu-1: 2h 21m -> 1h 58m (-16.1%)
3. pr-check-1: 33m 33s -> 28m 14s (-15.9%)
4. x86_64-gnu-llvm-20: 1h 18m -> 1h 6m (-15.5%)
5. x86_64-gnu-distcheck: 2h 21m -> 2h (-15.2%)
6. test-various: 2h 12m -> 1h 52m (-15.0%)
7. tidy: 2m 54s -> 2m 31s (-12.9%)
8. x86_64-gnu-gcc: 1h 10m -> 1h 1m (-11.8%)
9. x86_64-gnu: 2h 22m -> 2h 6m (-11.3%)
10. x86_64-gnu-tools: 1h 2m -> 55m 40s (-11.3%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (8387095): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary -2.5%, secondary 5.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	5.9%	[5.9%, 5.9%]	1
Improvements ✅ (primary)	-2.5%	[-2.5%, -2.5%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-2.5%	[-2.5%, -2.5%]	1

Cycles

Results (primary 2.2%, secondary -2.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.2%	[2.2%, 2.2%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.9%	[-2.9%, -2.9%]	1
All ❌✅ (primary)	2.2%	[2.2%, 2.2%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 482.346s -> 480.396s (-0.40%)
Artifact size: 397.84 MiB -> 397.88 MiB (0.01%)