Avoid `pip install --dry-run` downloading full wheels #13482

pelson · 2025-07-15T03:12:58Z

Removed "more preparation" (downloading) from the resolver to prevent downloading before dry-run validation
Added distribution caching to InstallRequirement with set_dist() and get_dist() methods to preserve metadata-only distributions
Set download_info during metadata-only fetching to ensure it's available for commands like pip lock and --report without requiring full downloads

pelson · 2025-07-15T03:16:18Z

src/pip/_internal/resolution/resolvelib/resolver.py

            req_set.add_named_requirement(ireq)

-        reqs = req_set.all_requirements
-        self.factory.preparer.prepare_linked_requirements_more(reqs)


This is the key change - the resolver no longer triggers additional preparation. Instead, we move this responsibility to consumers of the resolve result.

Technically, I could have made the preparer aware of the fact that we don't want to download anything, but the preparer is already a grab-bag of flags, and it felt cleaner to do it this way.

Is there any sort of documentation attached to the preparer where we could document this contract? While this change makes sense, it is also a bit arbitrary.

notatallshaw · 2025-07-15T03:46:21Z

Thanks for opening the pull request with pip, I'm certainly happy to see this PR, but please be aware it might take a moment for a pip maintainer to review this, as we all do this on a volunteer basis.

* Removed "more preparation" (downloading) from the resolver to prevent downloading before dry-run validation * Added distribution caching to `InstallRequirement` with `set_dist()` and `get_dist()` methods to preserve metadata-only distributions * Set `download_info` during metadata-only fetching to ensure it's available for commands like `pip lock` and `--report` without requiring full downloads Closes pypa#12603.

notatallshaw · 2025-08-02T00:49:05Z

Thanks again for raising this PR, I've put it on my list of things to review, though it may be a few weeks before I have enough time to give full feedback.

notatallshaw

Overall I am very happy with this approach, the news item needs updating, and I am going to merge in the latest main to check no changes elsewhere have caused anything to fail.

It would be nice to have at least one integration test, but if you can not provide this I will add one in a follow up PR.

notatallshaw · 2025-09-13T16:03:19Z

news/12603.feature.rst

@@ -0,0 +1 @@
+When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install.


This needs to be updated slightly, in-particular all sdists are still downloaded (at least for now), either specifically call out wheels or change to something like:

When PEP-658 metadata is available for a distribution the full file is no longer downloaded

Or

When PEP-658 metadata is available, full wheel download no longer occurs

Further, also add that pip lock no longer downloads distributions.

If there's no update and no objections in the next few days I will update this news items myself.

Technically pip has no problem fetching metadata for sdists already 😉. So this PR does make --dry-run avoid an sdist download/build 🎉 (this is a pretty substantial benefit as it means you don't need to build the sdist to figure out the dependencies... if only we had any repositories with sdist metadata served ala 658).

To prove this, I did the following:

mkdir -p ./delme3-project; cd ./delme3-project cat << EOF > pyproject.toml [project] name = "delme3" version = "0.1.0" requires-python = ">=3.11" EOF python -m venv ./venv source ./venv/bin/activate python -m pip install build python -m build --sdist . mkdir -p dummy-repo/delme3 mv dist/delme3-0.1.0.tar.gz ./dummy-repo/delme3/ tar -xOf ./dummy-repo/delme3/delme3-0.1.0.tar.gz delme3-0.1.0/PKG-INFO > ./dummy-repo/delme3/delme3-0.1.0.tar.gz.metadata cat << EOF > ./dummy-repo/delme3/index.html <!DOCTYPE html><html><body> <a href="/delme3/delme3-0.1.0.tar.gz" data-core-metadata="true">delme3-0.1.0.tar.gz</a><br/> </body></html> EOF cd dummy-repo python -m http.server

On the pip side, I ran:

$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/ Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes) Would install delme3-0.1.0

With only the following http.server output:

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/ HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -

(i.e. the sdist is never downloaded)

Whereas on master (f2b9231):

$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/ Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes) Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz (646 bytes) Running command pip subprocess to install build dependencies Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) Looking in indexes: http://localhost:8000/ ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none) ERROR: No matching distribution found for setuptools>=40.8.0 Ensuring compatibility with Acc-Py error: subprocess-exited-with-error × pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. full command: /media/important/github/pypa/pip/venv/bin/python /media/important/github/pypa/pip/src/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-ohiuyo5e/overlay --no-warn-script-location --disable-pip-version-check --no-compile --target '' -v --no-binary :none: --only-binary :none: -i http://localhost:8000/ --trusted-host acc-py-repo.cern.ch -- 'setuptools>=40.8.0' cwd: [inherit] Installing build dependencies ... error error: subprocess-exited-with-error × pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip.

With the http log:

$ python -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/ HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:56] code 404, message File not found 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /setuptools/ HTTP/1.1" 404 -

I proceeded to try pip lock out on the above setup, but got a failure:

$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/ Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning. Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes) ERROR: Exception: Traceback (most recent call last): File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 107, in _run_wrapper status = _inner_run() ^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 98, in _inner_run return self.run(options, args) ^^^^^^^^^^^^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/cli/req_command.py", line 70, in wrapper return func(self, options, args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/commands/lock.py", line 162, in run pylock_toml = Pylock.from_install_requirements( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 179, in from_install_requirements packages=sorted( ^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 181, in <genexpr> Package.from_install_requirement(ireq, base_dir) File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 138, in from_install_requirement raise NotImplementedError() NotImplementedError

I think that is because my package repository didn't include hashes...

This was the changed line in question:

$ cat ./delme3/index.html | grep sha256 <a href="/delme3/delme3-0.1.0.tar.gz#sha256=713524ababddfe0944ccc8bef33535a13e5ddcb92156dddfc37fc838c8e91acc" data-core-metadata="true">delme3-0.1.0.tar.gz foo</a><br/>

When I added them it correctly generated the lock file without downloading:

$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/ Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning. Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)

$ python -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/ HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -

🎉

Would you like to mention anything about performance in the what's new? Whilst pip doesn't quite reach uv performance, it is still very reasonable when the cache is cold! The narrative that pip is much slower might be nice to counter here 😉

$ time uv pip install --dry-run torch --no-cache --index-url https://pypi.org/simple/ Using Python 3.11.11 environment at: venv Resolved 25 packages in 371ms Would download 24 packages Would install 24 packages + filelock==3.19.1 + fsspec==2025.9.0 + jinja2==3.1.6 + markupsafe==3.0.2 + mpmath==1.3.0 + networkx==3.5 + nvidia-cublas-cu12==12.8.4.1 + nvidia-cuda-cupti-cu12==12.8.90 + nvidia-cuda-nvrtc-cu12==12.8.93 + nvidia-cuda-runtime-cu12==12.8.90 + nvidia-cudnn-cu12==9.10.2.21 + nvidia-cufft-cu12==11.3.3.83 + nvidia-cufile-cu12==1.13.1.3 + nvidia-curand-cu12==10.3.9.90 + nvidia-cusolver-cu12==11.7.3.90 + nvidia-cusparse-cu12==12.5.8.93 + nvidia-cusparselt-cu12==0.7.1 + nvidia-nccl-cu12==2.27.3 + nvidia-nvjitlink-cu12==12.8.93 + nvidia-nvtx-cu12==12.8.90 + sympy==1.14.0 + torch==2.8.0 + triton==3.4.0 + typing-extensions==4.15.0 real 0m0.492s user 0m0.132s sys 0m0.039s

$ time python -m pip lock torch --no-cache --index-url https://pypi.org/simple/ WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning. Looking in indexes: https://pypi.org/simple/ Collecting torch Downloading torch-2.8.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (30 kB) Collecting filelock (from torch) Downloading filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB) Collecting typing-extensions>=4.10.0 (from torch) Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB) Collecting sympy>=1.13.3 (from torch) Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch) Downloading networkx-3.5-py3-none-any.whl.metadata (6.3 kB) Collecting jinja2 (from torch) Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB) Collecting fsspec (from torch) Downloading fsspec-2025.9.0-py3-none-any.whl.metadata (10 kB) Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch) Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch) Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch) Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch) Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cublas-cu12==12.8.4.1 (from torch) Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cufft-cu12==11.3.3.83 (from torch) Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-curand-cu12==10.3.9.90 (from torch) Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch) Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch) Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cusparselt-cu12==0.7.1 (from torch) Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB) Collecting nvidia-nccl-cu12==2.27.3 (from torch) Downloading nvidia_nccl_cu12-2.27.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB) Collecting nvidia-nvtx-cu12==12.8.90 (from torch) Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch) Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cufile-cu12==1.13.1.3 (from torch) Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting triton==3.4.0 (from torch) Downloading triton-3.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB) Collecting setuptools>=40.8.0 (from triton==3.4.0->torch) Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB) Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch) Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Collecting MarkupSafe>=2.0 (from jinja2->torch) Downloading MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) real 0m1.555s user 0m0.807s sys 0m0.104s

Interesting about the sdists, I wasn't aware of that impact, I'll try and review later this week.

The Windows test failure looks unrelated. I'll try and make, or review, a PR to fix as soon as I can.

pelson

Thank you for the review! News updated

pelson · 2025-09-24T11:02:00Z

news/12603.feature.rst

@@ -0,0 +1 @@
+When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install.


Technically pip has no problem fetching metadata for sdists already 😉. So this PR does make --dry-run avoid an sdist download/build 🎉 (this is a pretty substantial benefit as it means you don't need to build the sdist to figure out the dependencies... if only we had any repositories with sdist metadata served ala 658).

To prove this, I did the following:

mkdir -p ./delme3-project; cd ./delme3-project cat << EOF > pyproject.toml [project] name = "delme3" version = "0.1.0" requires-python = ">=3.11" EOF python -m venv ./venv source ./venv/bin/activate python -m pip install build python -m build --sdist . mkdir -p dummy-repo/delme3 mv dist/delme3-0.1.0.tar.gz ./dummy-repo/delme3/ tar -xOf ./dummy-repo/delme3/delme3-0.1.0.tar.gz delme3-0.1.0/PKG-INFO > ./dummy-repo/delme3/delme3-0.1.0.tar.gz.metadata cat << EOF > ./dummy-repo/delme3/index.html <!DOCTYPE html><html><body> <a href="/delme3/delme3-0.1.0.tar.gz" data-core-metadata="true">delme3-0.1.0.tar.gz</a><br/> </body></html> EOF cd dummy-repo python -m http.server

On the pip side, I ran:

$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/ Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes) Would install delme3-0.1.0

With only the following http.server output:

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/ HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:06] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -

(i.e. the sdist is never downloaded)

Whereas on master (f2b9231):

$ python -m pip install delme3 -v --no-cache --dry-run --index-url http://localhost:8000/ Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes) Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz (646 bytes) Running command pip subprocess to install build dependencies Using pip 25.3.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) Looking in indexes: http://localhost:8000/ ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none) ERROR: No matching distribution found for setuptools>=40.8.0 Ensuring compatibility with Acc-Py error: subprocess-exited-with-error × pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. full command: /media/important/github/pypa/pip/venv/bin/python /media/important/github/pypa/pip/src/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-ohiuyo5e/overlay --no-warn-script-location --disable-pip-version-check --no-compile --target '' -v --no-binary :none: --only-binary :none: -i http://localhost:8000/ --trusted-host acc-py-repo.cern.ch -- 'setuptools>=40.8.0' cwd: [inherit] Installing build dependencies ... error error: subprocess-exited-with-error × pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip.

With the http log:

$ python -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/ HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /delme3/delme3-0.1.0.tar.gz HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:43:56] code 404, message File not found 127.0.0.1 - - [24/Sep/2025 12:43:56] "GET /setuptools/ HTTP/1.1" 404 -

I proceeded to try pip lock out on the above setup, but got a failure:

$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/ Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning. Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes) ERROR: Exception: Traceback (most recent call last): File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 107, in _run_wrapper status = _inner_run() ^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/cli/base_command.py", line 98, in _inner_run return self.run(options, args) ^^^^^^^^^^^^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/cli/req_command.py", line 70, in wrapper return func(self, options, args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/commands/lock.py", line 162, in run pylock_toml = Pylock.from_install_requirements( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 179, in from_install_requirements packages=sorted( ^^^^^^^ File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 181, in <genexpr> Package.from_install_requirement(ireq, base_dir) File "/media/important/github/pypa/pip/src/pip/_internal/models/pylock.py", line 138, in from_install_requirement raise NotImplementedError() NotImplementedError

I think that is because my package repository didn't include hashes...

This was the changed line in question:

$ cat ./delme3/index.html | grep sha256 <a href="/delme3/delme3-0.1.0.tar.gz#sha256=713524ababddfe0944ccc8bef33535a13e5ddcb92156dddfc37fc838c8e91acc" data-core-metadata="true">delme3-0.1.0.tar.gz foo</a><br/>

When I added them it correctly generated the lock file without downloading:

$ python -m pip lock delme3 -v --no-cache --index-url http://localhost:8000/ Using pip 25.2.dev0 from /media/important/github/pypa/pip/src/pip (python 3.11) WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning. Looking in indexes: http://localhost:8000/ Collecting delme3 Obtaining dependency information for delme3 from http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata Downloading http://localhost:8000/delme3/delme3-0.1.0.tar.gz.metadata (74 bytes)

$ python -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/ HTTP/1.1" 200 - 127.0.0.1 - - [24/Sep/2025 12:56:44] "GET /delme3/delme3-0.1.0.tar.gz.metadata HTTP/1.1" 200 -

🎉

pelson · 2025-09-24T11:08:34Z

news/12603.feature.rst

@@ -0,0 +1 @@
+When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install.


Would you like to mention anything about performance in the what's new? Whilst pip doesn't quite reach uv performance, it is still very reasonable when the cache is cold! The narrative that pip is much slower might be nice to counter here 😉

$ time uv pip install --dry-run torch --no-cache --index-url https://pypi.org/simple/ Using Python 3.11.11 environment at: venv Resolved 25 packages in 371ms Would download 24 packages Would install 24 packages + filelock==3.19.1 + fsspec==2025.9.0 + jinja2==3.1.6 + markupsafe==3.0.2 + mpmath==1.3.0 + networkx==3.5 + nvidia-cublas-cu12==12.8.4.1 + nvidia-cuda-cupti-cu12==12.8.90 + nvidia-cuda-nvrtc-cu12==12.8.93 + nvidia-cuda-runtime-cu12==12.8.90 + nvidia-cudnn-cu12==9.10.2.21 + nvidia-cufft-cu12==11.3.3.83 + nvidia-cufile-cu12==1.13.1.3 + nvidia-curand-cu12==10.3.9.90 + nvidia-cusolver-cu12==11.7.3.90 + nvidia-cusparse-cu12==12.5.8.93 + nvidia-cusparselt-cu12==0.7.1 + nvidia-nccl-cu12==2.27.3 + nvidia-nvjitlink-cu12==12.8.93 + nvidia-nvtx-cu12==12.8.90 + sympy==1.14.0 + torch==2.8.0 + triton==3.4.0 + typing-extensions==4.15.0 real 0m0.492s user 0m0.132s sys 0m0.039s

$ time python -m pip lock torch --no-cache --index-url https://pypi.org/simple/ WARNING: pip lock is currently an experimental command. It may be removed/changed in a future release without prior warning. Looking in indexes: https://pypi.org/simple/ Collecting torch Downloading torch-2.8.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (30 kB) Collecting filelock (from torch) Downloading filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB) Collecting typing-extensions>=4.10.0 (from torch) Downloading typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB) Collecting sympy>=1.13.3 (from torch) Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch) Downloading networkx-3.5-py3-none-any.whl.metadata (6.3 kB) Collecting jinja2 (from torch) Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB) Collecting fsspec (from torch) Downloading fsspec-2025.9.0-py3-none-any.whl.metadata (10 kB) Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch) Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch) Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch) Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch) Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cublas-cu12==12.8.4.1 (from torch) Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cufft-cu12==11.3.3.83 (from torch) Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-curand-cu12==10.3.9.90 (from torch) Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch) Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch) Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cusparselt-cu12==0.7.1 (from torch) Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB) Collecting nvidia-nccl-cu12==2.27.3 (from torch) Downloading nvidia_nccl_cu12-2.27.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB) Collecting nvidia-nvtx-cu12==12.8.90 (from torch) Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch) Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cufile-cu12==1.13.1.3 (from torch) Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting triton==3.4.0 (from torch) Downloading triton-3.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB) Collecting setuptools>=40.8.0 (from triton==3.4.0->torch) Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB) Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch) Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Collecting MarkupSafe>=2.0 (from jinja2->torch) Downloading MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) real 0m1.555s user 0m0.807s sys 0m0.104s

ichard26

This is an awesome optimization. I love to see it!

I'll defer to @notatallshaw's review, but I have my own drive-by comments. Damian, when you deem this PR ready, feel free to merge.

ichard26 · 2025-10-12T22:57:01Z

src/pip/_internal/commands/download.py

                preparer.save_linked_requirement(req)
                downloaded.append(req.name)

-        preparer.prepare_linked_requirements_more(requirement_set.requirements.values())


I'm not very familiar with the preparer, could you explain why we need to prepare requirements earlier now?

I believe this is because the resolver no longer downloads the files and later save_linked_requirements() needs req.local_file_path to be set.

ichard26 · 2025-10-12T22:59:33Z

src/pip/_internal/resolution/resolvelib/resolver.py

            req_set.add_named_requirement(ireq)

-        reqs = req_set.all_requirements
-        self.factory.preparer.prepare_linked_requirements_more(reqs)


Is there any sort of documentation attached to the preparer where we could document this contract? While this change makes sense, it is also a bit arbitrary.

news/12603.feature.rst

Co-authored-by: Richard Si <[email protected]>

notatallshaw · 2025-10-14T03:38:29Z

Is there any sort of documentation attached to the preparer where we could document this contract? While this change makes sense, it is also a bit arbitrary.

I agree that this whole section of pip needs extensive documentation, especially on what are the expectations, but I don't think this should hold this PR up.

notatallshaw

LGTM!

@pelson while I do agree there are performance improvements for certain situations I don't want to set the wrong expectations here, I think it's enough to say full distribution files aren't downloaded. I really appreciate your work here.

pelson · 2025-10-14T09:28:48Z

I'd be happy to follow-up on the developer documentation aspect in slower time (for clarity, I don't think this is likely that I will be able to do it before a release). There are other follow-ups in relation to performance and reduction of tech-debt that I'm interested to explore too.

I don't want to over-promise though 😉, my workload at the moment means that it is unlikely there will be any major activity from me before xmas 🎄.

notatallshaw · 2025-10-17T23:51:13Z

Thanks @pelson for your PR to pip!

pelson · 2025-10-18T03:53:25Z

Thank you for the reviews, and I want to say that I greatly appreciate the clarity of your messaging in relation to the challenge of maintainer time.

psf-chronographer bot added the bot:chronographer:provided label Jul 15, 2025

pelson commented Jul 15, 2025

View reviewed changes

pelson force-pushed the feature/no-download-for-dry-run branch from ff4715b to 196f992 Compare July 15, 2025 03:49

notatallshaw added this to the 25.3 milestone Sep 11, 2025

notatallshaw approved these changes Sep 13, 2025

View reviewed changes

notatallshaw and others added 2 commits September 13, 2025 12:05

Merge branch 'main' into feature/no-download-for-dry-run

a844f3b

Refine the news

fc2eb8d

pelson commented Sep 24, 2025

View reviewed changes

Merge branch 'main' into feature/no-download-for-dry-run

11d2a6f

ichard26 reviewed Oct 12, 2025

View reviewed changes

notatallshaw and others added 2 commits October 13, 2025 23:32

Update news/12603.feature.rst

62a5d16

Co-authored-by: Richard Si <[email protected]>

Merge branch 'main' into feature/no-download-for-dry-run

afa16f2

notatallshaw approved these changes Oct 14, 2025

View reviewed changes

notatallshaw merged commit 5a36b1c into pypa:main Oct 17, 2025
28 checks passed

pelson deleted the feature/no-download-for-dry-run branch October 18, 2025 03:51

		@@ -0,0 +1 @@
		When PEP-658 metadata is available, full distribution download no longer occurs when using dry-run mode on install.

Avoid pip install --dry-run downloading full wheels #13482

Avoid pip install --dry-run downloading full wheels #13482

Conversation

pelson commented Jul 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

notatallshaw commented Jul 15, 2025

Uh oh!

notatallshaw commented Aug 2, 2025

Uh oh!

notatallshaw left a comment

Choose a reason for hiding this comment

Uh oh!

notatallshaw Sep 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pelson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ichard26 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

notatallshaw commented Oct 14, 2025

Uh oh!

notatallshaw left a comment

Choose a reason for hiding this comment

Uh oh!

pelson commented Oct 14, 2025

Uh oh!

Uh oh!

notatallshaw commented Oct 17, 2025

Uh oh!

pelson commented Oct 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Avoid `pip install --dry-run` downloading full wheels #13482

Avoid `pip install --dry-run` downloading full wheels #13482

notatallshaw Sep 13, 2025 •

edited

Loading