Open
Description
- I have searched the issue tracker and believe that this is not a duplicate.
Make sure you run commands with -v
flag before pasting the output.
Steps to reproduce
Twice in the past 24 hours, I have seen PDM exhibit a memory corruption related crash in my CI jobs.
I do not yet know how to reproduce this issue reliably. Re-running the same job a second time can work successfully; this problem is not deterministic. Filing it in case there might be something suspicious that you can think of, and to provide a place to keep track of further instances to collect more information.
In a Python 3.11 CI job yesterday:
$ if [ -f pdm.lock ]; then pdm sync --no-editable -v; else pip install .; fi
STATUS: Resolving packages from lockfile...
STATUS: Fetching hashes for resolved packages...
Synchronizing working set with resolved packages: 114 to add, 0 to update, 0 to remove
realloc(): invalid next size
/usr/bin/bash: line 277: 214 Aborted (core dumped) pdm sync --no-editable -v
In a Python 3.12 CI job today (for a different project):
$ if [ -f pdm.lock ]; then pdm sync --no-editable -v; else pip install .; fi
STATUS: Resolving packages from lockfile...
Synchronizing working set with resolved packages: 122 to add, 0 to update, 0 to remove
double free or corruption (fasttop)
/usr/bin/bash: line 278: 194 Aborted (core dumped) pdm sync --no-editable -v
Actual behavior
PDM sometimes dumps core due to memory corruption issues.
Expected behavior
PDM doesn't crash and installs the files.
Environment Information
Here is how PDM is installed and set up in these CI jobs:
$ PIP_INDEX_URL= pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in /usr/local/lib/python3.11/site-packages (23.2.1)
Collecting pip
Obtaining dependency information for pip from https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl.metadata
Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.11/site-packages (65.5.1)
Collecting setuptools
Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/de/88/70c5767a0e43eb4451c2200f07d042a4bcd7639276003a9c54a68cfcc1f8/setuptools-70.0.0-py3-none-any.whl.metadata
Downloading setuptools-70.0.0-py3-none-any.whl.metadata (5.9 kB)
Requirement already satisfied: wheel in /usr/local/lib/python3.11/site-packages (0.42.0)
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata
Downloading wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Downloading pip-24.0-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 31.0 MB/s eta 0:00:00
Downloading setuptools-70.0.0-py3-none-any.whl (863 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 863.4/863.4 kB 31.2 MB/s eta 0:00:00
Downloading wheel-0.43.0-py3-none-any.whl (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.8/65.8 kB 5.8 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip
Attempting uninstall: wheel
Found existing installation: wheel 0.42.0
Uninstalling wheel-0.42.0:
Successfully uninstalled wheel-0.42.0
Attempting uninstall: setuptools
Found existing installation: setuptools 65.5.1
Uninstalling setuptools-65.5.1:
Successfully uninstalled setuptools-65.5.1
Attempting uninstall: pip
Found existing installation: pip 23.2.1
Uninstalling pip-23.2.1:
Successfully uninstalled pip-23.2.1
Successfully installed pip-24.0 setuptools-70.0.0 wheel-0.43.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ if [ -f pdm.lock ]; then pip install pdm~=$PDM_VERSION; fi
Looking in indexes: https://gitlab-ci-token:****@gitlab.<REDACTED>
Collecting pdm~=2.15.0
Downloading pdm-2.15.4-py3-none-any.whl.metadata (10 kB)
Collecting blinker (from pdm~=2.15.0)
Downloading blinker-1.8.2-py3-none-any.whl.metadata (1.6 kB)
Collecting packaging!=22.0,>=20.9 (from pdm~=2.15.0)
Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting platformdirs (from pdm~=2.15.0)
Downloading platformdirs-4.2.2-py3-none-any.whl.metadata (11 kB)
Collecting rich>=12.3.0 (from pdm~=2.15.0)
Downloading rich-13.7.1-py3-none-any.whl.metadata (18 kB)
Collecting virtualenv>=20 (from pdm~=2.15.0)
Downloading virtualenv-20.26.2-py3-none-any.whl.metadata (4.4 kB)
Collecting pyproject-hooks (from pdm~=2.15.0)
Downloading pyproject_hooks-1.1.0-py3-none-any.whl.metadata (1.3 kB)
Collecting unearth>=0.15.0 (from pdm~=2.15.0)
Downloading unearth-0.15.4-py3-none-any.whl.metadata (3.8 kB)
Collecting dep-logic<1.0,>=0.2.0 (from pdm~=2.15.0)
Downloading dep_logic-0.2.0-py3-none-any.whl.metadata (2.7 kB)
Collecting findpython<1.0.0a0,>=0.6.0 (from pdm~=2.15.0)
Downloading findpython-0.6.1-py3-none-any.whl.metadata (5.3 kB)
Collecting tomlkit<1,>=0.11.1 (from pdm~=2.15.0)
Downloading tomlkit-0.12.5-py3-none-any.whl.metadata (2.7 kB)
Collecting shellingham>=1.3.2 (from pdm~=2.15.0)
Downloading shellingham-1.5.4-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting python-dotenv>=0.15 (from pdm~=2.15.0)
Downloading python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)
Collecting resolvelib>=1.0.1 (from pdm~=2.15.0)
Downloading resolvelib-1.0.1-py2.py3-none-any.whl.metadata (4.0 kB)
Collecting installer<0.8,>=0.7 (from pdm~=2.15.0)
Downloading installer-0.7.0-py3-none-any.whl.metadata (936 bytes)
Collecting truststore (from pdm~=2.15.0)
Downloading truststore-0.9.1-py3-none-any.whl.metadata (4.5 kB)
Collecting hishel<0.1.0,>=0.0.24 (from pdm~=2.15.0)
Downloading hishel-0.0.27-py3-none-any.whl.metadata (10 kB)
Collecting msgpack>=1.0 (from pdm~=2.15.0)
Downloading msgpack-1.0.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.1 kB)
Collecting pbs-installer (from pdm~=2.15.0)
Downloading pbs_installer-2024.4.24-py3-none-any.whl.metadata (990 bytes)
Collecting httpx<1,>0.20 (from httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading httpx-0.27.0-py3-none-any.whl.metadata (7.2 kB)
Collecting filelock>=3.13 (from pdm~=2.15.0)
Downloading filelock-3.15.1-py3-none-any.whl.metadata (2.8 kB)
Collecting typing-extensions>=4.8.0 (from hishel<0.1.0,>=0.0.24->pdm~=2.15.0)
Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting anyio (from httpx<1,>0.20->httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading anyio-4.4.0-py3-none-any.whl.metadata (4.6 kB)
Collecting certifi (from httpx<1,>0.20->httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading certifi-2024.6.2-py3-none-any.whl.metadata (2.2 kB)
Collecting httpcore==1.* (from httpx<1,>0.20->httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)
Collecting idna (from httpx<1,>0.20->httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting sniffio (from httpx<1,>0.20->httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>0.20->httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
Collecting socksio==1.* (from httpx[socks]<1,>0.20->pdm~=2.15.0)
Downloading socksio-1.0.0-py3-none-any.whl.metadata (6.1 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=12.3.0->pdm~=2.15.0)
Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=12.3.0->pdm~=2.15.0)
Downloading pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Collecting distlib<1,>=0.3.7 (from virtualenv>=20->pdm~=2.15.0)
Downloading distlib-0.3.8-py2.py3-none-any.whl.metadata (5.1 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=12.3.0->pdm~=2.15.0)
Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Downloading pdm-2.15.4-py3-none-any.whl (258 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 258.5/258.5 kB 13.6 MB/s eta 0:00:00
Downloading dep_logic-0.2.0-py3-none-any.whl (26 kB)
Downloading filelock-3.15.1-py3-none-any.whl (15 kB)
Downloading findpython-0.6.1-py3-none-any.whl (20 kB)
Downloading hishel-0.0.27-py3-none-any.whl (38 kB)
Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 7.0 MB/s eta 0:00:00
Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 6.4 MB/s eta 0:00:00
Downloading socksio-1.0.0-py3-none-any.whl (12 kB)
Downloading installer-0.7.0-py3-none-any.whl (453 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 453.8/453.8 kB 28.8 MB/s eta 0:00:00
Downloading msgpack-1.0.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (409 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 409.3/409.3 kB 24.1 MB/s eta 0:00:00
Downloading packaging-24.1-py3-none-any.whl (53 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.0/54.0 kB 4.9 MB/s eta 0:00:00
Downloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)
Downloading resolvelib-1.0.1-py2.py3-none-any.whl (17 kB)
Downloading rich-13.7.1-py3-none-any.whl (240 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.7/240.7 kB 15.8 MB/s eta 0:00:00
Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)
Downloading tomlkit-0.12.5-py3-none-any.whl (37 kB)
Downloading unearth-0.15.4-py3-none-any.whl (47 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.0/47.0 kB 3.1 MB/s eta 0:00:00
Downloading virtualenv-20.26.2-py3-none-any.whl (3.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 39.0 MB/s eta 0:00:00
Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB)
Downloading blinker-1.8.2-py3-none-any.whl (9.5 kB)
Downloading pbs_installer-2024.4.24-py3-none-any.whl (42 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.3/42.3 kB 2.1 MB/s eta 0:00:00
Downloading pyproject_hooks-1.1.0-py3-none-any.whl (9.2 kB)
Downloading truststore-0.9.1-py3-none-any.whl (17 kB)
Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 22.1 MB/s eta 0:00:00
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 4.9 MB/s eta 0:00:00
Downloading pygments-2.18.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 32.9 MB/s eta 0:00:00
Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Downloading anyio-4.4.0-py3-none-any.whl (86 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 5.1 MB/s eta 0:00:00
Downloading idna-3.7-py3-none-any.whl (66 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB 4.4 MB/s eta 0:00:00
Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Downloading certifi-2024.6.2-py3-none-any.whl (164 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 164.4/164.4 kB 11.0 MB/s eta 0:00:00
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 4.6 MB/s eta 0:00:00
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Installing collected packages: resolvelib, distlib, typing-extensions, truststore, tomlkit, socksio, sniffio, shellingham, python-dotenv, pyproject-hooks, pygments, platformdirs, pbs-installer, packaging, msgpack, mdurl, installer, idna, h11, filelock, certifi, blinker, virtualenv, markdown-it-py, httpcore, findpython, dep-logic, anyio, rich, httpx, unearth, hishel, pdm
Successfully installed anyio-4.4.0 blinker-1.8.2 certifi-2024.6.2 dep-logic-0.2.0 distlib-0.3.8 filelock-3.15.1 findpython-0.6.1 h11-0.14.0 hishel-0.0.27 httpcore-1.0.5 httpx-0.27.0 idna-3.7 installer-0.7.0 markdown-it-py-3.0.0 mdurl-0.1.2 msgpack-1.0.8 packaging-24.1 pbs-installer-2024.4.24 pdm-2.15.4 platformdirs-4.2.2 pygments-2.18.0 pyproject-hooks-1.1.0 python-dotenv-1.0.1 resolvelib-1.0.1 rich-13.7.1 shellingham-1.5.4 sniffio-1.3.1 socksio-1.0.0 tomlkit-0.12.5 truststore-0.9.1 typing-extensions-4.12.2 unearth-0.15.4 virtualenv-20.26.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ python -m venv $VENV_NAME
$ source ${VENV_NAME}/bin/activate
$ if [ -f pdm.lock ]; then pdm use --ignore-remembered --first $VENV_NAME/bin/python; fi
Using Python interpreter: /builds/<REDACTED>/.venv/bin/python (3.11)
$ pip install setuptools wheel
Looking in indexes: https://gitlab-ci-token:****@gitlab.<REDACTED>
Requirement already satisfied: setuptools in ./.venv/lib/python3.11/site-packages (65.5.0)
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata
Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Using cached wheel-0.43.0-py3-none-any.whl (65 kB)
Installing collected packages: wheel
Successfully installed wheel-0.43.0
[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
$ if [ -f pdm.lock ]; then pdm sync --no-editable -v; else pip install .; fi
STATUS: Resolving packages from lockfile...
STATUS: Fetching hashes for resolved packages...
Synchronizing working set with resolved packages: 114 to add, 0 to update, 0 to remove
realloc(): invalid next size
/usr/bin/bash: line 277: 214 Aborted (core dumped) pdm sync --no-editable -v