Repository navigation
Fix relocate stripping RPATH on cross-compiled builds (#37) - #352
Merged
Merged
Conversation
On a cross build (e.g. amd64 host building aarch64), ``handle_elf``'s
original ``ldd`` call fails -- the host dynamic linker either refuses to
load a foreign-arch ELF ("not a dynamic executable") or exits non-zero
with empty stdout. The parser silently iterates zero lines,
``needs_rpath`` stays ``False``, and the else-branch calls
``remove_rpath(path)`` on every single cross-built binary. Net effect:
RPATH gets stripped from an entire cross-compiled onedir, which then
can't find its own bundled libs at runtime. Issue is from 2022-11; it
has survived because CI's ``build-cross`` job is currently commented
out and nobody exercises the path.
Replace the host ``ldd`` with a DT_NEEDED parse of ``readelf -d``:
* ``parse_readelf_d_needed(stdout)`` -- extract SONAMEs from the dynamic
section's ``NEEDED`` entries. Pure parser over text we already know
how to produce.
* ``read_needed(path)`` -- call site that routes through the existing
``_get_readelf_binary()`` helper, so toolchain-prefixed ``readelf``
(e.g. ``aarch64-linux-gnu-readelf``) is used when ``RELENV_BUILDENV``
points at a cross toolchain. ``readelf`` just parses ELF metadata,
so it's arch-agnostic by construction.
``handle_elf`` rewritten to:
1. Pull DT_NEEDED SONAMEs via ``read_needed`` (toolchain-aware).
2. For each SONAME, skip the glibc allow-list, then check if a file of
that name exists under ``libs/`` (or anywhere under ``root/`` as a
shallow fallback for arch-specific subdirs). Found -> needs_rpath.
3. In ``rpath_only=True`` mode (the default, what finalize uses) a
missing file is just a warning -- the binary links to something we
don't bundle; leave it to the system loader. No host ``ldd``
involvement at all.
4. In ``rpath_only=False`` mode (the manual ``relenv relocate`` CLI on
already-installed native onedirs) we still fall back to host ``ldd``
to resolve the SONAME for copying, since that's the only way to find
the source file on a native host. Cross builds never use this path.
Tests in ``tests/test_relocate.py``:
* New ``test_parse_readelf_d_needed`` + ``_empty`` + ``_malformed``
covering the parser.
* Existing ``test_handle_elf*`` updated to mock ``read_needed`` instead
of ``subprocess.run`` (which now goes to ``readelf`` + an optional
``ldd`` fallback). Semantics unchanged.
* New ``test_handle_elf_keeps_rpath_on_cross_build`` -- the specific
gh-37 regression: simulate host ``ldd`` exiting non-zero with empty
stdout, assert the binary's RPATH is patched (not removed) because
the DT_NEEDED entries show a bundled dependency.
All 28 relocate tests pass locally. CI's native builds will keep
exercising the native path; cross CI remains out of scope here but the
bug it was always going to surface is now fixed.
2 of 3 tasks
This PR replaced handle_elf's ldd shell-out with a readelf -d DT_NEEDED parse via read_needed(), but the existing unit test was still shimming subprocess.run for ldd and raising AssertionError on anything else. Stub read_needed directly so the test exercises the new arch-agnostic path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #37.
Problem
On a cross build (e.g. amd64 host building aarch64),
relenv.relocate.handle_elfshells out to the hostlddon every built ELF. The host dynamic linker refuses to load a foreign-arch ELF -- it exits non-zero with empty stdout, or printsnot a dynamic executable. The parser'sfor line in proc.stdout.decode().splitlines()iterates zero lines,needs_rpathstaysFalse, andhandle_elffalls through toremove_rpath(path).Net effect: RPATH gets stripped from every single cross-compiled binary, so the cross-built onedir can't locate its own bundled libraries at runtime.
Issue filed 2022-11; survived because
build-crossis commented out of CI so this path never gets exercised in practice.Fix
Replace the host
lddwith a DT_NEEDED parse ofreadelf -d.readelfjust parses ELF metadata (arch-agnostic by construction), and_get_readelf_binary()already prefers the target toolchain'sreadelfwhenRELENV_BUILDENVpoints at a cross toolchain -- so even an x86_64 host building aarch64 picks up the right tool.In
rpath_only=True(the default -- what finalize uses) we don't need the host linker's path resolution at all: we just need to know which DT_NEEDED SONAMEs refer to libraries already laid down underlibs/. That's a filesystem-local check that works identically for native and cross builds.New helpers in
relenv/relocate.py:parse_readelf_d_needed(stdout)-- extract SONAMEs fromNEEDEDlines.read_needed(path)-- routes through_get_readelf_binary().handle_elfrewritten to use them. The non-rpath-only branch (manualrelenv relocateCLI on native onedirs) retains thelddfallback for copying missing libs from the host, which is only reachable from a native invocation.Tests
test_parse_readelf_d_needed+_empty+_malformedunit tests on the parser.test_handle_elf*updated to mockread_neededinstead ofsubprocess.run(semantic contract unchanged).test_handle_elf_keeps_rpath_on_cross_build-- the specific relocate fails on cross compiled builds #37 regression. Simulates hostlddexiting non-zero with empty stdout, asserts the binary's RPATH is patched (not removed) because DT_NEEDED shows a bundled dep.All 28
test_relocate.pytests pass locally.