libunwind build system not picking up LDFLAGS causing build failure on systems without zlib

[NickCao]

LDFLAGS is required for the linker to find bundled libraries

julia/Make.inc

Line 728 in 378f192

LDFLAGS += -L$(build_libdir) -Wl,-rpath,$(build_libdir)

But libunwind build system is not passing it to the linker, causing configure/build failure on systems without zlib, for example the nixpkgs stdenv.

configure:3928: checking whether the C compiler works
configure:3950: gcc -m64   -U_FORTIFY_SOURCE -fPIC -lz     -Wl,-rpath,'$$ORIGIN' -Wl,-z,origin -Wl,-rpath-link,/build/julia-1.10.5/usr/lib -Wl,--enable-new-dtags  conftest.c  >&5
/nix/store/x7yyxvwy1f9hlx72rzrgx069jyf7hxwr-binutils-2.42/bin/ld: cannot find -lz: No such file or directory

Additionally, in #42782, -lz was added to LIBUNWIND_CFLAGS in deps/unwind.mk:

julia/deps/unwind.mk

Line 6 in 378f192

LIBUNWIND_CFLAGS := -U_FORTIFY_SOURCE $(fPIC) -lz $(SANITIZE_OPTS)

But -lz is already handled by libunwind configure script:
https://github.com/libunwind/libunwind/blob/v1.5.0/configure.ac#L316

[giordano]

@NickCao would you be able to give #55624 a try?

[giordano]

LDFLAGS is required for the linker to find bundled libraries

julia/Make.inc

Line 728 in 378f192

LDFLAGS += -L$(build_libdir) -Wl,-rpath,$(build_libdir)

Why do you claim setting LDFLAGS is necessary? Autotools should be able to find libraries in the prefix, no? That's what we do with BinaryBuilder (which also uses a non-standard prefix where to install the packages) and we never have to set LDFLAGS separately to point the linker to the prefix. And we already set the prefix in

julia/deps/tools/common.mk

Line 7 in 378f192

CONFIGURE_COMMON = --prefix=$(abspath $(build_prefix)) --build=$(BUILD_MACHINE) --libdir=$(abspath $(build_libdir)) --bindir=$(abspath $(build_depsbindir)) $(CUSTOM_LD_LIBRARY_PATH)

and you can see that's set also for you in #55624 (comment)

[NickCao]

Why do you claim setting LDFLAGS is necessary? Autotools should be able to find libraries in the prefix, no?

I believe the --prefix option is only about the installation prefix of the package currently being configured (libunwind)? It's not passed to the underlying compiler/linker by autotools, so the linker would still follow its default search paths.

[giordano]

I'm not sure: in BinaryBuilder we don't have the prefix's libdir (/workspace/destdir/lib) anywhere in the default search path of the compiler (see LIBRARY_PATH):

sandbox:${WORKSPACE} # SUPER_VERBOSE=1 gcc -v
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -frandom-seed=0x75262c83 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -Wl,-rpath-link,/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -march=x86-64 -mtune=generic -v
Using built-in specs.
COLLECT_GCC=/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/lto-wrapper
Target: x86_64-linux-gnu
Configured with: /workspace/srcdir/gcc-5.2.0/configure --prefix=/workspace/destdir --target=x86_64-linux-gnu --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-linux-gnu/sys-root --program-prefix=x86_64-linux-gnu- --disable-bootstrap --with-arch=x86-64 --enable-languages=c,c++,fortran,objc,obj-c++
Thread model: posix
gcc version 5.2.0 (GCC) 
COMPILER_PATH=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/:/opt/x86_64-linux-gnu/bin/../libexec/gcc/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/bin/
LIBRARY_PATH=/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/:/opt/x86_64-linux-gnu/bin/../lib/gcc/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=1' '-frandom-seed=0x75262c83' '-L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64' '-march=x86-64' '-mtune=generic' '-v'
 /opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/collect2 -plugin /opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/liblto_plugin.so -plugin-opt=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/cchfGdiM.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crt1.o /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crti.o /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/crtbegin.o -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -L/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0 -L/opt/x86_64-linux-gnu/bin/../lib/gcc -L/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib/../lib64 -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/../lib64 -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64 -L/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib -rpath-link /opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/crtend.o /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crtn.o
/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crt1.o: In function `_start':
/workspace/srcdir/glibc-2.17/csu/../sysdeps/x86_64/start.S:119: undefined reference to `main'
collect2: error: ld returned 1 exit status

and yet we can find all the dependencies of libunwind just by setting the prefix (which I believe is used to look for the dependencies as well): https://github.com/JuliaPackaging/Yggdrasil/blob/84c1061962644addde53ef51f2ada406be61978b/L/LibUnwind/LibUnwind%401.8.1/build_tarballs.jl. LDFLAGS isn't set anywhere

[NickCao]

That's since libunwind mostly only depends on zlib, which is available on nearly every system.

[giordano]

That's since libunwind mostly only depends on zlib, which is available on nearly every system.

Not in the binarybuilder environment (which is why I'm insisting on taking it as an example).

[NickCao]

That's since libunwind mostly only depends on zlib, which is available on nearly every system.

Not in the binarybuilder environment (which is why I'm insisting on taking it as an example).

Except there is, tested with:

--- a/L/LibUnwind/[email protected]/build_tarballs.jl
+++ b/L/LibUnwind/[email protected]/build_tarballs.jl
@@ -29,6 +29,14 @@ if [[ ${bb_full_target} == *-sanitize+memory* ]]; then
     cp -rL ${libdir}/linux/* /opt/x86_64-linux-musl/lib/clang/*/lib/linux/
 fi

+cat > test.c <<EOF
+int main() {}
+EOF
+
+$CC -Wl,--verbose test.c -lz
+
+exit 1
+
 export CFLAGS="-DPI -fPIC"
 ./configure \
     --prefix=${prefix} \

attempt to open /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/local/lib/libz.so succeeded
-lz (/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/local/lib/libz.so

[giordano]

Except there is

That's not a libz from the sysroot (otherwise for most targets it'd be a file for a foreign architecture or OS) but it's the libz coming from the dependencies installed in the prefix (/opt/${target}/${target}/sys-root/usr/local is a symlink to ${prefix}).

That said, the libdir in the prefix (${prefix}/lib == /workspace/destdir/lib) is indeed in the linker search path:

sandbox:${WORKSPACE} # ld --verbose | grep SEARCH_DIR | tr -s ' ;' '\n'
SEARCH_DIR("=/workspace/destdir/x86_64-linux-musl/lib64")
SEARCH_DIR("=/workspace/destdir/lib64")
SEARCH_DIR("=/usr/local/lib64")
SEARCH_DIR("=/lib64")
SEARCH_DIR("=/usr/lib64")
SEARCH_DIR("=/workspace/destdir/x86_64-linux-musl/lib")
SEARCH_DIR("=/workspace/destdir/lib")
SEARCH_DIR("=/usr/local/lib")
SEARCH_DIR("=/lib")
SEARCH_DIR("=/usr/lib")

[NickCao]

That said, the libdir in the prefix (${prefix}/lib == /workspace/destdir/lib) is indeed in the linker search path:

But how does it know to add that? Is the linker configured specially or patched?

[Zentrik]

I think this is related to us adding zlib to the image we use for source builds, as the unwind build failed finding it. JuliaCI/rootfs-images#185

[Zentrik]

LDFLAGS is required for the linker to find bundled libraries

julia/Make.inc

Line 728 in 378f192
LDFLAGS += -L$(build_libdir) -Wl,-rpath,$(build_libdir)

fwiw, this seems to only be set for freebsd.

[Zentrik]

I'm not sure what's going on with BinaryBuilder here, but this doesn't seem to work elsewhere so I've created a fix that sets the LDFLAGS appropriately #57798. The prefix per configure --help specifies where some files get installed and per configure --help we should be setting the LDFLAGS.

Excerpts from configure --help

Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
                          [/usr/local]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
                          [PREFIX]

--with-sysroot[=DIR]    Search for dependent libraries within DIR (or the
                        compiler's sysroot if not specified).

Some influential environment variables:
  CC          C compiler command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  LIBS        libraries to pass to the linker, e.g. -l<library>
  CPPFLAGS    (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
              you have headers in a nonstandard directory <include dir>
  CXX         C++ compiler command
  CXXFLAGS    C++ compiler flags
  LT_SYS_LIBRARY_PATH
              User-defined run-time library search path.
  CPP         C preprocessor
  CXXCPP      C++ preprocessor
  CCAS        assembler compiler command (defaults to CC)
  CCASFLAGS   assembler compiler flags (defaults to CFLAGS)

Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.

Specifying the sysroot doesn't seem to actually do anything unfortunately. Though I wouldn't think we would want to set that.

On the binarybuilder issue, zlib is a build dep of libunwind so wouldn't we expect it to be in the search path of the compiler?

[Zentrik]

@NickCao once #57792 merges, you should be able to drop the zlib build dependency in nixpkgs. The libxml2 dep seems unnecessary as well. cacert seems to be used to workaround nix setting SSL_CERT_FILE to /no-cert-file.crt which prevents julia from using the cacert it bundles, not sure if this is desired.

[NickCao]

@NickCao once #57792 merges, you should be able to drop the zlib build dependency in nixpkgs. The libxml2 dep seems unnecessary as well. cacert seems to be used to workaround nix setting SSL_CERT_FILE to /no-cert-file.crt which prevents julia from using the cacert it bundles, not sure if this is desired.

Thanks! I will try once it reaches a release.