Skip to content

Share the compiler repository with rustfmt of the same release - #4443

Open
TrapsterDK wants to merge 1 commit into
bazelbuild:mainfrom
TrapsterDK:rustfmt-reuse-rustc
Open

TrapsterDK wants to merge 1 commit into
bazelbuild:mainfrom
TrapsterDK:rustfmt-reuse-rustc

Conversation

@TrapsterDK

Copy link
Copy Markdown
Contributor

Closes #4441.

Since #1769, rustfmt_toolchain_tools_repository downloads and unpacks the rustc component next to rustfmt, because rustfmt loads librustc_driver relative to its own path. That's about 400 MB per execution platform. When rustfmt_version is the release the Rust toolchains compile with by default, the same rustc is already in the shared <name>__<channel>__rustc repository from #4397, so a build that runs rustfmt holds two copies.

This change:

  • rust_register_toolchains passes that compiler repository to the rustfmt repository as a new rustc_anchor attribute. It only does this when rustfmt_version is one of versions and is the default channel's release (stable, or the only version given). A nightly rustfmt in a stable build still downloads its own rustc, because the nightly compiler repository also holds its standard library and llvm-tools.
  • With rustc_anchor set, the rustfmt repository links the compiler's lib/ beside rustfmt, the way the Clippy repository already does, and skips the rustc download. rustc_lib names the links, so remote executors get the libraries where rustfmt looks for them. The toolchain's rustc points at the compiler repository.
  • rust_repository_set now gets compact_windows_names from rust_register_toolchains, so the shared repository names computed for rustfmt match the ones the set declares. The name logic moved into _shared_repository_name so both places use it.

Without an anchor, the behaviour is the same as before.

Test: a new integration module, test/integration/rustfmt_shared_compiler, sets rustfmt_version to its only version. It runs a rustfmt_test, plus an analysis test that checks the rustfmt toolchain's rustc comes from the compiler repository and its rustc_lib from rustfmt's own. It runs as a post_shell_commands step of the existing rustfmt_failure task, because presubmit is already at BazelCI's limit of 128 tasks.

  • Without the change: the analysis test fails.
  • With the change: bazel test //... passes (2/2), and the rustfmt repository is 7 MB with no rustc of its own.

bazel test //test/rustfmt/... at the root still passes (20/20).

Related: #1768.

Assisted by Claude.

`rustfmt_toolchain_tools_repository` downloaded and unpacked its own
`rustc` (about 400 MB unpacked) because rustfmt loads `librustc_driver`
relative to its own path. When `rustfmt_version` is the release the Rust
toolchains of the same execution platform compile with by default, that
compiler is already in their `<name>__<channel>__rustc` repository.

`rust_register_toolchains` now passes that repository as `rustc_anchor`,
and the rustfmt repository links its `lib/` beside `rustfmt` (as the
Clippy repository does) instead of downloading `rustc`. `rustc_lib`
names the links, so remote executors get the libraries where rustfmt
looks for them. The compiler repository of a non-default channel is not
shared, since it also holds its standard library and llvm-tools and is
otherwise only fetched with `--@rules_rust//rust/toolchain/channel`.

`rust_repository_set` now gets `compact_windows_names` from
`rust_register_toolchains`, so the shared repository names computed for
rustfmt match the ones the set declares.

The new `test/integration/rustfmt_shared_compiler` module sets
`rustfmt_version` to its only version, runs a `rustfmt_test`, and checks
that the rustfmt toolchain's `rustc` comes from the compiler repository
and its `rustc_lib` from rustfmt's own. The rustfmt repository there is
7 MB with no `rustc` of its own.

Assisted-by: Claude

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rustfmt toolchain repository downloads a second rustc

1 participant