Skip to content

add2virtualenv logic breaks builds of packages with modern build backend #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dfn-certling opened this issue Apr 17, 2024 · 1 comment
Open

add2virtualenv logic breaks builds of packages with modern build backend #98

dfn-certling opened this issue Apr 17, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@dfn-certling
Copy link

The logic in add2virtualenv of adding paths to the front of sys.path breaks installing packages in the virtualenv that use a PEP517 build. In preparing the build environment pip tries to determine the necessary sys.path using a "sitecustomize.py" written in pip's _internal/build_env.py, which gets confused with paths not being added to the end of sys.path. The result is that the lib-dynload path of the Python environment gets removed from sys.path and pip install fails with a traceback like this

# pip install .
Processing /Devel/foo
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 534, in collect_root_requirements
    reqs = list(
           ^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 490, in _make_requirements_from_install_req
    cand = self._make_base_candidate_from_link(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 228, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 290, in __init__
    super().__init__(
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 54, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 124, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 101, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 745, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/pip-build-env-pitma8s7/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 70, in do_override
    ensure_local_distutils()
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 53, in ensure_local_distutils
    importlib.import_module('distutils')
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 90, in find_spec
    return method()
           ^^^^^^^^
  File "/root/.virtualenvs/foo/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 97, in spec_for_distutils
    import importlib.abc
  File "/usr/lib64/python3.11/importlib/abc.py", line 19, in <module>
    from .resources.abc import ResourceReader, Traversable, TraversableResources
  File "/usr/lib64/python3.11/importlib/resources/__init__.py", line 3, in <module>
    from ._common import (
  File "/usr/lib64/python3.11/importlib/resources/_common.py", line 3, in <module>
    import tempfile
  File "/usr/lib64/python3.11/tempfile.py", line 45, in <module>
    from random import Random as _Random
  File "/usr/lib64/python3.11/random.py", line 49, in <module>
    from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
ModuleNotFoundError: No module named 'math'

This can be reproduced with a fresh virtualenv that contains a couple - in this setup it was three, but in other environments two were enough - paths added via add2virtualenv

mkvirtualenv foo
mkdir dummy1 dummy2 dummy3
add2virtualenv dummy1 dummy2 dummy3

and then installing a basically empty package with a minimal pyproject.toml

echo "[project]
name = 'bar'
version = '1.0.0'

[build-system]
requires = ['setuptools']
build-backend = 'setuptools.build_meta'" > pyproject.toml
pip install .
@dfn-certling
Copy link
Author

Version numbers.

#  pip show virtualenvwrapper
Name: virtualenvwrapper
Version: 6.1.0
Summary: 
Home-page: 
Author: 
Author-email: Doug Hellmann <[email protected]>, Jason Myers <[email protected]>
License: MIT
Location: /usr/lib/python3.11/site-packages
Requires: stevedore, virtualenv, virtualenv-clone
Required-by: 

# pip --version
pip 22.3.1 from /usr/lib/python3.11/site-packages/pip (python 3.11)

@dhellmann dhellmann added the bug Something isn't working label Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dhellmann @dfn-certling