[DONOTMERGE] Switch llvm source build to v20 #57658

Zentrik · 2025-03-06T16:25:59Z

This is for testing with pkgeval before we have built on yggdrasil

@nanosoldier runtests(configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1", "USE_BINARYBUILDER_LLVM=0"],), vs_configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

maleadt · 2025-03-09T06:37:23Z

PkgEval got restarted:

@nanosoldier runtests(configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1", "USE_BINARYBUILDER_LLVM=0"],), vs_configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

Only one job ahead of this, so it should report within 2 days.

maleadt · 2025-03-10T08:03:34Z

The build failed:

CMake 3.20.0 or higher is required.  You are running version 3.18.4

PkgEval is currently using the latest rootfs, https://github.com/JuliaCI/rootfs-images/releases/tag/v7.10. I guess we'll need to upgrade from bullseye to bookworm, https://github.com/JuliaCI/rootfs-images/blob/e7be8086ff1285b0f333ca72884977d01aed0e1f/src/build_img/debian.jl#L17.

giordano · 2025-03-10T08:07:21Z

You can download cmake with contrib/download_cmake.sh, but no idea how do that for pkgeval invocations (if doable at all). But maybe the Makefile here can be hacked to force that.

nanosoldier · 2025-03-10T08:14:54Z

Your job failed. Consult the server logs for more details (cc @maleadt).

Zentrik · 2025-03-10T09:20:12Z

I forgot we added an invocation to the download cmake script for the source build, JuliaCI/julia-buildkite@552037b.

Zentrik · 2025-03-10T11:44:30Z

@maleadt would it be easy to get this fixed otherwise I'll start the llvm yggy build?

giordano · 2025-03-10T11:46:36Z

Can't you run the download cmake script as part of the build script in this PR? This is just for testing anyway.

Zentrik · 2025-03-10T11:49:40Z

Can't you run the download cmake script as part of the build script in this PR? This is just for testing anyway.

Yep, ignore me then tim, I'll do that.

Zentrik · 2025-03-10T13:44:55Z

Let's see if this works

@nanosoldier runtests(configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1", "USE_BINARYBUILDER_LLVM=0"],), vs_configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

src/llvm-late-gc-lowering.cpp

nanosoldier · 2025-03-12T15:52:33Z

Your job failed. Consult the server logs for more details (cc @maleadt).

Zentrik · 2025-03-12T16:11:27Z

I suspect it's missing some package, the source build uses the llvm-passes rootfs image.

maleadt · 2025-03-12T16:20:38Z

/usr/local/bin/../lib/gcc/x86_64-linux-gnu/9.1.0/../../../../x86_64-linux-gnu/bin/ld: warning: libz.so.1, needed by ../../lib/libLLVM.so.20.1jl, not found (try using -rpath or -rpath-link)
/usr/local/bin/../lib/gcc/x86_64-linux-gnu/9.1.0/../../../../x86_64-linux-gnu/bin/ld: ../../lib/libLLVM.so.20.1jl: undefined reference to `compress2'
/usr/local/bin/../lib/gcc/x86_64-linux-gnu/9.1.0/../../../../x86_64-linux-gnu/bin/ld: ../../lib/libLLVM.so.20.1jl: undefined reference to `crc32'
/usr/local/bin/../lib/gcc/x86_64-linux-gnu/9.1.0/../../../../x86_64-linux-gnu/bin/ld: ../../lib/libLLVM.so.20.1jl: undefined reference to `compressBound@ZLIB_1.2.0'
/usr/local/bin/../lib/gcc/x86_64-linux-gnu/9.1.0/../../../../x86_64-linux-gnu/bin/ld: ../../lib/libLLVM.so.20.1jl: undefined reference to `uncompress'
collect2: error: ld returned 1 exit status

Zentrik · 2025-03-12T20:23:28Z

Hmm, this just seems like a llvm bug where it's not specifying the path to zlib when linking with libLLVM.

deps/llvm.mk

Zentrik · 2025-03-13T14:56:06Z

Let's try this one more time

@nanosoldier runtests(configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1", "USE_BINARYBUILDER_LLVM=0"],), vs_configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

nanosoldier · 2025-03-14T01:27:29Z

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

Report summary

❗ Packages that crashed

2 packages crashed only on the current version.

A segmentation fault happened: 2 packages

✖ Packages that failed

7 packages failed only on the current version.

Package fails to precompile: 1 packages
Package has test failures: 3 packages
Test duration exceeded the time limit: 2 packages
Test log exceeded the size limit: 1 packages

673 packages failed on the previous version too.

✔ Packages that passed tests

10 packages passed tests only on the current version.

Other: 10 packages

3665 packages passed tests on the previous version too.

~ Packages that at least loaded

1 packages successfully loaded only on the current version.

Other: 1 packages

1987 packages successfully loaded on the previous version too.

➖ Packages that were skipped altogether

4034 packages were skipped on the previous version too.

giordano · 2025-03-14T01:29:32Z

That's remarkably few new failures.

Zentrik · 2025-03-14T16:03:46Z

All looked good, but let's confirm on the newer commit from our llvm fork.

@nanosoldier runtests(configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1", "USE_BINARYBUILDER_LLVM=0"],), vs_configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

nanosoldier · 2025-03-14T21:28:08Z

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

Report summary

❗ Packages that crashed

2 packages crashed only on the current version.

A segmentation fault happened: 2 packages

✖ Packages that failed

7 packages failed only on the current version.

Package has test failures: 4 packages
Networking-related issues were detected: 1 packages
Test duration exceeded the time limit: 2 packages

606 packages failed on the previous version too.

✔ Packages that passed tests

4 packages passed tests only on the current version.

Other: 4 packages

3334 packages passed tests on the previous version too.

~ Packages that at least loaded

1737 packages successfully loaded on the previous version too.

➖ Packages that were skipped altogether

4691 packages were skipped on the previous version too.

benlorenz · 2025-03-15T16:02:13Z

Might make sense to re-run PkgEval once #57785 is merged?
Because currently 4691 packages were skipped compared to 1357 two weeks ago (when the PkgEval package installation issue was originally introduced).

IanButterworth · 2025-03-15T19:06:08Z

Good to rerun now

Filed llvm/llvm-project#131119

Zentrik · 2025-03-16T00:37:45Z

@nanosoldier runtests(configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1", "USE_BINARYBUILDER_LLVM=0"],), vs_configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

Zentrik · 2025-03-16T13:01:02Z

Hmm, this x86 darwin failure looks real given it doesn't occur on #57792.

 clang++ -mmacosx-version-min=10.14 -m64  -shared -fPIC -L[buildroot]/usr/lib/julia -L[buildroot]/usr/lib -L[buildroot]/usr/lib -o [buildroot]/usr/lib/julia/sysbase.dylib -Xlinker -all_load [buildroot]/usr/lib/julia/sysbase-o.a  -ljulia-internal -ljulia $([ Darwin = WINNT ] && echo '' -lopenlibm -lssp --disable-auto-import --disable-runtime-pseudo-reloc)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#1.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#1.o)
duplicate symbol '___truncdfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#1.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#2.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#2.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#3.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#3.o)
duplicate symbol '___truncdfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#3.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#4.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#4.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#5.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#5.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#6.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#6.o)
duplicate symbol '___truncdfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#6.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#7.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#7.o)
duplicate symbol '___truncdfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#7.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#8.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#8.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#9.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#9.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#10.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#10.o)
duplicate symbol '___extendhfsf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#11.o)
duplicate symbol '___truncsfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#11.o)
duplicate symbol '___truncdfhf2' in:
    [buildroot]/usr/lib/julia/sysbase-o.a(text#0.o)
    [buildroot]/usr/lib/julia/sysbase-o.a(text#11.o)
ld: 27 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [[buildroot]/usr/lib/julia/sysbase.dylib] Error 1

nanosoldier · 2025-03-16T18:11:24Z

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

Report summary

❗ Packages that crashed

2 packages crashed only on the current version.

An internal error was encountered: 1 packages
A segmentation fault happened: 1 packages

1 packages crashed on the previous version too.

✖ Packages that failed

19 packages failed only on the current version.

Package has test failures: 6 packages
Package tests unexpectedly errored: 1 packages
Test duration exceeded the time limit: 11 packages
Test log exceeded the size limit: 1 packages

1449 packages failed on the previous version too.

✔ Packages that passed tests

20 packages passed tests only on the current version.

Other: 20 packages

5049 packages passed tests on the previous version too.

~ Packages that at least loaded

16 packages successfully loaded only on the current version.

Other: 16 packages

2923 packages successfully loaded on the previous version too.

➖ Packages that were skipped altogether

1 packages were skipped only on the current version.

Package could not be installed: 1 packages

905 packages were skipped on the previous version too.

Try to debug issue at JuliaLang#57658 (comment)

Zentrik added DO NOT MERGE Do not merge this PR! external dependencies Involves LLVM, OpenBLAS, or other linked libraries labels Mar 6, 2025

maleadt mentioned this pull request Mar 10, 2025

Bump Linux glibc images to Debian bookworm JuliaCI/rootfs-images#259

Open

Zentrik commented Mar 11, 2025

View reviewed changes

src/llvm-late-gc-lowering.cpp Outdated Show resolved Hide resolved

Zentrik commented Mar 13, 2025

View reviewed changes

deps/llvm.mk Outdated Show resolved Hide resolved

Zentrik commented Mar 13, 2025

View reviewed changes

deps/llvm.mk Outdated Show resolved Hide resolved

Zentrik force-pushed the llvm-20-test-pkgeval branch 2 times, most recently from ea8c8b5 to 3d11791 Compare March 13, 2025 14:54

Zentrik force-pushed the llvm-20-test-pkgeval branch from 3d11791 to f12db52 Compare March 14, 2025 15:52

[DONOTMERGE] Switch llvm source build to v20

07fcedb

Zentrik added 20 commits March 15, 2025 22:45

Hack

5588c5e

Fix bug

1a03169

Try Fix bug

5afe17b

Try again

3f20c7c

Try again

3363757

Fix typo

117901e

Fix another deprecation

67b6545

Fix more deprecations

be06b1e

Fix another deprecation

6ba6814

hopefully last one

746447f

Remove release banner override

879c573

Fix more deprecations

fb9033a

Update src/llvm-late-gc-lowering.cpp

8f9fbde

Include checksum and remove hack

ebb0b07

Hack to get llvm to link against correct zlib

c172ce2

Filed llvm/llvm-project#131119

Update deps/llvm.mk

6a1fc48

Update deps/llvm.mk

acbed08

Bump to latest llvm

96e6af3

Add checksums

6cad7f8

Update test

48fdcdb

Zentrik force-pushed the llvm-20-test-pkgeval branch from 1b962d6 to 48fdcdb Compare March 15, 2025 22:45

minor fixes

444b00a

Zentrik closed this Mar 16, 2025

Zentrik mentioned this pull request Mar 18, 2025

[LLVM_full] Add LLVM_full, LLVM_full_assert 20 recipes JuliaPackaging/Yggdrasil#10747

Open

Zentrik added a commit to Zentrik/julia that referenced this pull request Mar 18, 2025

Force x86 darwin build to use llvm from yggy CI

954b757

Try to debug issue at JuliaLang#57658 (comment)

Zentrik mentioned this pull request Mar 18, 2025

[DONOTMERGE] Debug x86 darwin issue #57821

Closed

gbaraldi mentioned this pull request Mar 19, 2025

Code that uses a libcall marks internal symbol as exported when making object file llvm/llvm-project#132055

Open

[DONOTMERGE] Switch llvm source build to v20 #57658

[DONOTMERGE] Switch llvm source build to v20 #57658

Conversation

Zentrik commented Mar 6, 2025

maleadt commented Mar 9, 2025

maleadt commented Mar 10, 2025

giordano commented Mar 10, 2025 • edited Loading

nanosoldier commented Mar 10, 2025

Zentrik commented Mar 10, 2025

Zentrik commented Mar 10, 2025

giordano commented Mar 10, 2025

Zentrik commented Mar 10, 2025

Zentrik commented Mar 10, 2025

nanosoldier commented Mar 12, 2025

Zentrik commented Mar 12, 2025

maleadt commented Mar 12, 2025

Zentrik commented Mar 12, 2025 • edited Loading

Zentrik commented Mar 13, 2025

nanosoldier commented Mar 14, 2025

❗ Packages that crashed

✖ Packages that failed

✔ Packages that passed tests

~ Packages that at least loaded

➖ Packages that were skipped altogether

giordano commented Mar 14, 2025

Zentrik commented Mar 14, 2025

nanosoldier commented Mar 14, 2025

❗ Packages that crashed

✖ Packages that failed

✔ Packages that passed tests

~ Packages that at least loaded

➖ Packages that were skipped altogether

benlorenz commented Mar 15, 2025

IanButterworth commented Mar 15, 2025

Zentrik commented Mar 16, 2025

Zentrik commented Mar 16, 2025

nanosoldier commented Mar 16, 2025

❗ Packages that crashed

✖ Packages that failed

✔ Packages that passed tests

~ Packages that at least loaded

➖ Packages that were skipped altogether

giordano commented Mar 10, 2025 •

edited

Loading

Zentrik commented Mar 12, 2025 •

edited

Loading