Bootstrap config: libgccjit libs dir

[antoyo]

r? @Kobzol

[rustbot]

This PR changes how GCC is built. Consider updating src/bootstrap/download-ci-gcc-stamp.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

[rustbot]

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

[antoyo]

@Kobzol: I'm sending this PR to gather some feedback from you or others.

How can I add tests for this new bootstrap config?
Where should I document this new config?

[bors]

☔ The latest upstream changes (presumably #149348) made this pull request unmergeable. Please resolve the merge conflicts.

[Kobzol]

I will focus on #149426 first, as I think that we should first deal with the compiler/cg_gcc side of things, before we go modify bootstrap.

[Kobzol]

This makes sense, although the implementation needs some changes. I'd be happy to play with this if you don't want to deal with doing these bootstrap changes.

View changes since this review

[bors]

☔ The latest upstream changes (presumably #149426) made this pull request unmergeable. Please resolve the merge conflicts.

[Kobzol]

I refactored the code a bit to more explicitly model that we have multiple target libgccjit dylibs per host, and also to enable both downloading from CI and using prebuilt libgccjit.so files (so that people don't also have to build their x64 libgccjit.so from scratch). Please let me know if it makes sense to you.

[antoyo]

Please let me know if it makes sense to you.

This commit makes sense to me.
For the other commit, I do not understand bootstrap enough to understand what's happening here. Would that commit help keep the m68k libgccjit.so file that was deleted? Do you want me to try to cross-compile rustc locally with that commit?

[Kobzol]

Do you want me to try to cross-compile rustc locally with that commit?

If you already have it prepared, yeah, you could try it. Although CI failed, so maybe there's something else wrong.

[antoyo]

If you already have it prepared, yeah, you could try it. Although CI failed, so maybe there's something else wrong.

Good. I try that right away.

[antoyo]

If you already have it prepared, yeah, you could try it. Although CI failed, so maybe there's something else wrong.

The command finished running and the issue with the m68k libgccjit.so file being deleted still seems there. Any ideas of how to fix this? Do the steps I posted above give you any hint?

[Kobzol]

I suspect I know where is the issue. bootstrap hacks around this for LLVM, but I don't like how it is done (I want to refactor how LLVM is handled in bootstrap, as it's super messy right now, but it's very hard).

In that case, I would actually suggest to sidestep this issue entirely and use a different directory structure: <sysroot>/lib/rustlib/<host-target>/codegen-backends/lib/<target>/libgccjit.so. This I think also makes it clearer that cg_gcc is actually invoked on <host-target>, and it will use libraries from its lib subdirectory, rather than from completely different directories in <sysroot>/lib/rustlib/....

This will run into an issue with Rustup later, but we would have the exact same issue even with the current layout, so it doesn't really matter.

What do you think?

[antoyo]

What do you think?

I'm OK with this directory structure.
But since this is still in rustlib, won't that get deleted too?

[Kobzol]

I certainly hope not 😆 Bootstrap should only be deleting rustlib/<target>/lib, I think. If not, then I misunderstood the issue.

[antoyo]

I certainly hope not 😆 Bootstrap should only be deleting rustlib/<target>/lib, I think. If not, then I misunderstood the issue.

Ok. I'll try this new directory structure locally to see if that works. If so, I'll push it here.

[antoyo]

I remember @bjorn3 had some concerns about a similar directory structure.
@bjorn3: Could you please confirm that this won't result in having multiple directories in /usr/lib with different target triples when doing a global install?

[antoyo]

There seems to be one issue: the native m68k libgccjit.so library is not copied in the sysroot. Instead, it takes a native x86 libgccjit.so:

rust/build/m68k-unknown-linux-gnu/stage2/lib/rustlib/m68k-unknown-linux-gnu/codegen-backends/lib/m68k-unknown-linux-gnu/libgccjit.so:       ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=7fb2bb8bf0436221a668bcd67d6042c067463bd6, with debug_info, not stripped

@Kobzol: Any idea what could be wrong here?

Edit: Could it be the step "Uplifting library (stage1:x86_64-unknown-linux-gnu -> stage2:m68k-unknown-linux-gnu)" that does that?

[antoyo]

My investigation leads me to believe that we don't fill properly dylibs here.

My understanding is that for a specific stage, we need:

• target of the built rustc (usually x86-64 for stage1, but here is m68k for stage2) -> target of the build rustc (here would be x86-64 -> x86-64 for stage1 and m68k -> m68k for stage2).
• and possibly target of the build rustc (usually x86-64 for stage1) -> other targets configured (here is m68k) (here would be x86-64 -> m68k for stage1).

But that code seems to consider the host target (the one used to build the rustc) instead.

[Kobzol]

It fascinates me how complicated is the handling of target pairs. We can be really thankful to LLVM for being able to cross-compile to multiple targets, otherwise bootstrap would have to handle yet another configuration axis 😆 This is kinda insane.

I pushed a change that should build libgccjit for more targets. But it's quite difficult to debug without having access to a repro. If you could send me the whole libgcc-libs-dir prepared for the m68k cross-compilation, that would be useful.

[antoyo]

It fascinates me how complicated is the handling of target pairs. We can be really thankful to LLVM for being able to cross-compile to multiple targets, otherwise bootstrap would have to handle yet another configuration axis 😆 This is kinda insane.

Indeed.

I pushed a change that should build libgccjit for more targets. But it's quite difficult to debug without having access to a repro. If you could send me the whole libgcc-libs-dir prepared for the m68k cross-compilation, that would be useful.

Thanks.
I'll try to figure out what files (if any) are necessary beyond libgccjit.so and gcc and create an archive.
I'll try your PR locally and see if that fixes the issue.

In the mean time, I believe you could try using --dry-run with some print messages to get an idea of what's going on.

[antoyo]

I believe this now copies the correct file, so we're getting there!
I'll finish my tests locally to confirm that everything works.

One thing I noticed that can be confusing is that if the file for m68k is not found, it will try to build GCC, which will likely fail. For instance:

libgccjit.so for `m68k-unknown-linux-gnu -> m68k-unknown-linux-gnu` was not found at `/path/to/libgccjit-libs-dir/m68k-unknown-linux-gnu/m68k-unknown-linux-gnu/libgccjit.so`
GCC CI download is only available for the `x86_64-unknown-linux-gnu` host/target
Building GCC for m68k-unknown-linux-gnu -> m68k-unknown-linux-gnu

I believe it would be better to abort with an error in this case. What are your thoughts on this?

[Kobzol]

I think that it might be useful to have the ability to provide some prebuilt GCCs, but also to build the ones that you don't have prebuilt.

That being said, bootstrap can't currently build GCC (T1, T2) where T1 != T2, and likely also where T1 != host. So in these cases we should indeed error out.

For the case where T1 != T2, but T1 = host, is there some flag we could use to build libgccjit for (T1, T2)? That's not strictly cross-compiling, so if it's easy to do, we could support it in bootstrap.

[antoyo]

I finished my testing and everything seems to work 🥳 .
Thanks a lot for your help here.
So, I guess the only missing part is to error when we cannot find the file in the libgccjit-libs-dir. Do I understand correctly that you suggest we should only error when we cannot build the lib?

For the case where T1 != T2, but T1 = host, is there some flag we could use to build libgccjit for (T1, T2)? That's not strictly cross-compiling, so if it's easy to do, we could support it in bootstrap.

Do you mean creating a cross-compiling libgccjit? For instance, from x86-64 to m68k?

[Kobzol]

When we try to resolve libgccjit for a target pair (T1, T2), three situations can happen:

1. T1 = host target of the PC that runs bootstrap, and T1 =T2. That's the easy default case, a native build of a GCC that will do native builds. Bootstrap already handles this.
2. T1 = host target and T1 != T2. This is a native build of a GCC that will cross-compile.
3. T1 != host target. This is a cross-compiled build of GCC.

We can already do 1. I assume that 3. is out of scope, and we should eagerly error out in that case (unless it's easy to cross-compile GCC, which I assume isn't the case).

I was wondering whether we can support 2. For example, if there is some flag for GCC that tells it to build itself so that it can compile code for T2 (something like --codegen-target=T2), we could do that in bootstrap.

[antoyo]

Doing number 2 is also not trivial. Currently, I have this repo where I create such cross-compilers in the CI. This uses a tool called crosstool-ng since it makes it much easier to create a cross-compiling GCC.
I guess if we're opened to use an external tool here, it should not be too hard.

But if using an external tool is not an option, this is going to be very hard. Look at this article which explains how to build a GCC cross-compiler if you want to have an idea.