Pip/pipx compatibiliy

[canatella]

I'd like my end user to be able to install my tool without having to clone the repo and everything, mostly with pipx. I use the publish plugin to export my tool to a private repo and I'm able to install it with pipx. So far so good. A problem arises when trying to run the tool. It seem pipx did not install the dependencies versions specified in the lock file, but used another version which breaks my tool: pdm.lock has

[[package]]
name = "pyparsing"
version = "2.4.7"
requires_python = ">=2.6,!=3.0.*,!=3.1.*,!=3.2.*"
summary = "Python parsing module"

Whereas pipx installed 3.0.0a0 which breaks my code. Am I missing something ?

Thanks!

[pawamoy]

The lock file is not used by package managers other than PDM. What matters is what you have specified in pyproject's dependencies. The deps end up in the wheel's metadata (wheel that is built by PDM), and are what pipx (or any other installation tool) uses to install dependencies.

Could you show your pyproject deps so we can help?

[canatella]

dependencies = [
    "adafruit-circuitpython-ssd1306==2.12.0",
    "Appium-Python-Client==1.1.0",
    "Pillow==8.3.1",
    "pytest==6.2.2",
    "RPi.GPIO==0.7.1a4; sys_platform == 'linux'",
    "firebase-admin~=5.0",
    "google-cloud-storage~=1.42",
    "google-cloud-logging~=2.6",
    "google-cloud-resource-manager==0.30.3",
    "docopt~=0.6",
]

So its a dep from a dep from a dep... Thanks for the help btw, greatly appreciated, I feel so lost with python packaging... Each time I think I understood, it comes to bites me again.

[pawamoy]

Yes, Python packaging is not easy to grasp 😅
But this is improving with latest PEPs and great tools like PDM 🙂

So, back to the issue. One of your deps probably allows pre-release versions of pyparsing, hence why pipx grabbed it. I wonder why PDM did not use it. It's maybe a simple timing issue? Version 3.0.0a0 was released after you locked and before you installed with pipx? Or maybe you passed --pip-args=--pre to pipx, and it contaminated subdependencies? I would find this surprising.

[canatella]

Yes that's it. I guess I'm stuck to generating the requirements file for now ?

[frostming]

Yeah, the lock file format is tool-specific and only consumed by the corresponding tool. Fortunately, Python Packaging is working on a standard lockfile format: PEP 665, hope that will help the eco-system.

[canatella]

Thanks, that PEP seems to be exactly what's missing indeed, lets hope its accepted and implemented soon in all tools.