Release 25.1 (April) #13185
Comments
ichard26
added
the
type: maintenance
|
I'm in agreement that having different maintainers wear the RM hat is important, both for different maintainers being familiar with the process and making sure the release instructions are correct. Also, I'd be happy to volunteer for a different release later in the year. |
|
I'd like to do a release sometime, just to check the new process on Windows. My open source time has been severely limited recently, but I can try to do 25.1 (and if @ichard26 or @notatallshaw want a blow-by-blow account of the process I used, I'm happy to do some form of running commentary or walkthrough). |
|
Sure, we can do that. It'd be nice to shadow an experienced RM for a release cycle before being a RM. In my original post, I got the release months mixed up. 25.2 (July) is probably the best time for me be the RM (unless I go on vacation then, but I haven't decided yet). I'm still available for 25.1 if availabilities change. I've assigned the issue to you to mark you as our tentative RM for pip 25.1.
Yes please! The mechanical work is likely probably well-documented (updating get-pip...?) but it's the decision-making and things to look out for I'd appreciate. (Perhaps we could turn your commentary into informal checklists?) |
|
One thing that needs to be decided is whether to drop support for Python 3.8 (#12989). My main concern is that pip can't vendor urllib 2.x until Python 3.10+, and I don't know the timeline for urllib 1.x support ending, the docs currently say "continue supporting v1.26.x releases with both security and bug fixes for the forseeable future": https://urllib3.readthedocs.io/en/1.26.6/v2-roadmap.html, but 2.x has been out for nearly two years now. |
|
We can simply ask @sethmlarson. Do you have plans to drop support for urllib3 1.x? Either entirely, or only on Python 3.8 specifically? |
|
If we get consensus on #12989 to drop support for Python 3.8, and someone does the work to create the necessary PR, that's fine by me with my RM hat on. It's not related to urllib3 (although maybe getting consensus will be affected by the urllib3 position - I doubt it, but it's possible). As far as I can see, with regard to urllib3 version 2, we're blocked until we drop support for Python 3.9 - at least that seems to be what this comment suggests. That's not going to be until pip 26.0 at the absolute earliest (Python 3.9 goes EOL just after pip 25.3 will be released). Regardless, anything related to the urllib3 migration should be discussed on #12857. And I'm going to say that barring something unexpected, it won't be happening in 25.1.
I'm pretty strongly against shipping two vendored copies of urllib3 so that we can pick which one to use based on the Python version. And having different wheels for different Python versions is a big enough change that it would need its own issue to track it (and I'd probably say no to including it in 25.1 anyway). |
|
I wasn't implying that we'd vendor two copies of urllib3. I was making the distinction since if urllib3 1.x is going to be completely unsupported, then we have a bigger problem that just Python 3.8 support since we aren't a position to properly support an unmaintained HTTP library. If urllib3 1.x is planning to drop Python 3.8 support, then pip can simply move in the same direction and also drop Python 3.8 support.
I had a PR open to drop Python 3.8 support (#13190) , but it got no attention so I gave up and closed it. I don't care strongly enough to push for it.
It won't because there is no point. We aren't planning to drop Python 3.9 for a while and before then, upgrading to urllib3 2.x is blocked. I'm honestly pretty confused here. I don't want us to consider anything BUT potentially dropping support for Python 3.8 this release. That is all, nothing more. |
I wasn't a huge fan of the size of the PR. That may be part of the reason you got little interest - 197 files changed is way too much to review. I don't think we normally run pyupgrade like this when we drop support, we just leave the code unchanged and gradually use newer constructs when appropriate. Also, why deliberately break people who want to continue using Python 3.8 (unsupported, of course)? Sorry - I probably could have commented to that effect at the time.
Me too. Let's assume urllib3 is irrelevant (for 25.1, at least). The relevant issue for dropping 3.8 support is #12989, which feels inconclusive to me right now. So we won't be dropping 3.8 in 25.1 unless someone pushes there for consensus. But personally, I don't think supporting 3.8 is causing us any issues, and download numbers (8.3% in December isn't that small) make me inclined to be cautious. |
We did for Python 3.7. It was much a smaller change, however (see the pyupgrade commit in #11944).
It's not so much that I'm trying to break people stuck on Python $version, but that if we're doing to make the formal designation that $version is unsupported, we should fully take advantage of the new/improved constructs. The codebase is going to quickly regress without automated testing, regardless. It'd only take one modern typing import in some unrelated change to cause pip to immediately crash on an unsupported Python version. Now, I'm actually pretty ambivalent about switching the codebase to 3.8+ typing constructs wholesale specifically. I do like the changes, but if we don't want to review it, then meh, I don't care. We'll just need to disable of the pyupgrade rules in Ruff. I'm pretty sure we're talking past each other at this point, so all else I'm going to say is that I don't care if we drop Python 3.8 this release or not. I won't be picking up that task, and frankly, pip is foundational enough where it'd be nice to maintain official support when it's not a burden (and it isn't). |
That's probably why I missed it 🙂
IMO, not so much talking past each other as debating philosophy 🙂 I also don't care if we drop Python 3.8 support - I won't be doing the work, but I won't object if someone wants to. It sounds like a big part of the pyupgrade stuff was typing changes. I'm a typing skeptic, and find the rapid pace of change in type annotations frustrating and annoying. So I tend to view things like this as evidence that the cost/benefit case for type annotations isn't necessarily as favourable as people like to claim. But as long as I can continue to ignore the whole thing and let someone else deal with it, I'm not going to worry. |
Sorry, I started this additional discussion, I should of been more explicit in my reasoning, my concern about urllib3 1.x support and Python 3.8 is that pip is unlikely to drop Python 3.9 support until some amount of time has passed since Python 3.8 support has been dropped, so it's more of a scheduling thing than a direct consequence thing.
There's no way to avoid that when we increase the minimum supported Python while we have the upgrade rules turned on: #12936 We would either need to disable the pyupgrade rules or have a different minimum version of Python in ruff only to 3.8.
For me I was waiting for either release manager input or a consensues on #12989 before reviewing the PR, at a cursory glance it looked fine. But reading your response I'll confine any further discussion to #12989.
I would agree with this if updating the typing syntax was manual, but almost all updates tend to be done by ruffs auto fixer. And while ruff is a relatively young tool, it is widely deployed by open source and closed source projects, I have a lot of trust that if the fixes are automatic that they have a high degree of safety. My main point for reviewing a PR like this would be seeing there are distinct commits where auto fixing is used and where manual fixes need to be applied. |
|
I created #13236 to discuss the pyupgrade rules, and whether we should automatically apply them when we desupport a Python version. |
|
As a quick note, I'm planning on aiming for mid-April for the release, probably the weekend of 19/20 April. That's about 6 weeks from now. My plan is that I'll release what's on Once the release is done, I'll allow a week or two for any showstopper issues to appear, but I will expect to declare 25.1 complete by the end of April. After that point, fixes for any further issues that might arise will need to go into 25.2. If anyone has anything they really want to get released in 25.1, get it completed and merged as soon as possible 🙂 |
|
There are two PRs I'd like to review. The former may/probably will be deferred in favour of waiting until the packaging wheel filename APIs are stabilized. The latter is conditionally approved, and waiting on my review. I expect some back and forth on it, but unless I find major deficiencies, I'll try to get it in for 25.1.
Looking at the release milestone, there are a number of deprecations scheduled for pip 25.1:
2 is trivially easy (there's already a PR), although honestly, I'm leaning towards simply removing the flag from the help output, removing the tests, and calling it a day. I don't want to cause breakage over small thing such as this flag. 3 is really @sbidoul's project. I'm unsure how ready we are to formally remove support for 1 is the tricky one. Currently, there hasn't been much progress on actually removing support for legacy editable installs. I did do some early prototyping work, but I didn't make much progress, and frankly, I am not experienced enough to see it through completion. Unless @sbidoul (who did some related work not too long ago) wants to drive this forward, it will probably be pushed to 25.2.1 Otherwise, there are some other (smaller) changes I'd like to get in, but they aren't time sensitive, so I won't spend the time to enumerate them here. If they happen, great! If not, so be it. Footnotes
|
|
I'll happily take care of legacy editables (#11457) and egg ditributions (#12330). Either do it or postpone. If there are opinions on whether or not the time has come to remove these I'm all ears. Regarding legacy editables, and removal of setuptools specific code paths in general, I'm personally in favor of pushing those actively, as I think the cases where pip falls back to the legacy code paths tend to create more and more confusion. And in practice the Regarding egg distributions, I don't know. While pkg_resources is vendored, it is not a big maintenance burden. It does affect performance a little bit, though, due to having to import |
|
I'm also happy to see removal of legacy editables moving forward. I'd like to see those code paths removed, and honestly I doubt much will change if we wait. At best, setuptools might improve their config settings support, but frankly I doubt it, and I think people no longer having the fallback of legacy editables is more likely to push them to improve things than anything else we can do. Your suggestion of how to handle egg distributions seems reasonable to me. It doesn't feel urgent to remove it, although if someone wants to get it done sooner, I won't object. |
|
I just want to say that I'm aware of the recent discussion. I'll deal with the deprecations and the resume PR when I get the chance. Life has been busy. |
|
OK, I've just done the first full run through the 25.1 milestone, and added comments to issues/PRs where needed. I don't see any major problems, so 19/20 April remains the plan. If anyone wants to add anything to the 25.1 milestone at this point, then:
|
|
Also, I tried doing a vendoring update. I couldn't find any documentation on the correct command to use 🙁 We should probably have some. I think the correct command should be However, that command failed, with an error because our pkg_resources patch no longer applies. I don't have much appetite for dealing with that (we're moving away from With that extra skip, we are upgrading platformdirs, rich and typing_extensions. I've make that into a vendoring PR (#13313), and added documentation of the correct command to use as part of that PR. |
+1 to freeze pkg_resources until we stop supporting these code paths. See also #13010 (comment) |
|
Wait - the tag is there! Did it somehow get created when I merged the PR for release/25.1? I'm going to just assume that the documentation is wrong when it says you need to push the tag as a separate action. |
|
Apart from that glitch, the new release process is really neat! Thanks to those who set it up 🙂 Pip 25.1 is now released, and on PyPI. I'll now do get-pip, the release announcement, and the CPython PR, and we're done :-) |
I think it's just a warning, as you have permission to bypass the tag protection rule. |
|
Ah, yes. The line you quoted is clearer than the one I was focused on, "Cannot create ref due to creations being restricted"... |
|
Hmm, there's a problem with get-pip. When we dropped support for Python 3.8, we should have also updated the get-pip repo to preserve a "last pip that worked for Python 3.8" version (similar to I'll see if I can work out how to fix the get-pip repo, but can we make a note that this needs to be done in future when we drop support for an old version? |
|
OK. The logic of the get-pip repository is a nightmare. I had to manually change the generate script to make the needed change for 3.8, then regenerate everything and commit. I couldn't just change the script because the CI checks that what's in the repo matches what's generated by the script. Once I did that, the release script fails because there's "nothing to commit". Because I just generated it all. So I think we have a valid set of files in the get-pip repo. We had a bad 25.1 tag (presumably from my first aborted attempt), which I've had to delete. I've somehow managed to create a new tag - I think it's OK, but the commit it points to is not called "Release 25.1" which might confuse people. Sorry about that. At some point, someone needs to review and fix the get-pip automation, so that it doesn't fail so badly. It's great when it works, but being easy to fix when things go wrong is also important. Probably another reason why we should tie "updating get-pip" to the PR that drops support for a Python version, and not leave it until the release. |
|
Good morning!
@sbidoul I'm talking about #13157 (non-bare project names in an
Yeah, I agree it's awkward. This is why we should always create dedicated new issues for where the userbase is expected to engage. There isn't much I can do now as the release is cut and the URL fixed. Also, I moved the content to the issue description because people may miss your comment entirely. If the information is in the description, it should be harder to miss. If you want, I can include that it was written by the entire pip team (thank you @sbidoul for writing the first draft!) somewhere in the description if you think that helps. |
|
CPython PR now created: python/cpython#132997 I won't merge this immediately, if we need a 25.1.1, I can update the PR with the bugfix. |
|
And the announcement is up on Discourse, so I think that's everything (until the bug reports start roling in 😨) |
|
That's the fun part 🙃 But thank you for cutting the release! Other then the get-pip mess, I'm glad it went well for you. I agree that we should document get-pip better. I forgot about it TBH. |
Everyone forgets about get-pip TBH. Unfortunately, I don't think it's documentation alone that's the issue - the design is just hard to deal with. We check in a bunch of generated files, and therefore we have CI to check that the generated files match the code that generates them. Which in principle is fine, but it means we can't decouple changing the generation code (for example, when desupporting a Python version) from regenerating the files (for a release). And thanks for your "what's new" post, by the way - it's really awesome to have these! |
|
I've reworked #13157's description to explain and offer guidance for those hitting the deprecation. With that finished, this concludes all of the communication work for this release on my end.
Thank you! I write these because A) it's fun, and B) I strongly believe that the pip project would benefit from communicating better with its users. (Until recently?) The majority of Python developers are exposed to pip at some level at some point. There are so many places where pip intersects with the packaging ecosystem and the Python developer experience, and yet people don't really know what the pip project is doing. We get a bunch of attention whenever we deprecate something, but that leaves a sour taste in people's mouths. We don't put much effort into reaching out to our users. Which is totally fine! I appreciate the casual/relaxed culture of the project. However, pip is still evolving. We are deprecating legacy features and gaining new features over time (albeit slowly). It's better if our users—if they're interested—can follow pip's development and understand why changes were made. In addition, it's better if our users can learn about pip from an official and reliable source (instead of reading gossip and hearsay from interested-but-not-always-correct third-parties).1 In practice, my posts receive low enough traffic2 that it is likely only the most interested developers who are reading them, but I still think that having resources like my release posts simply available is valuable. And finally, it's nice to celebrate our progress! A huge part of working on foundational (ecosystem) infrastructure is "working in the shadows" but being in the spotlight when things are good is nice, too 🙂 Footnotes |
|
Is anyone aware of any issues that warrant a 25.1.1 bugfix release? The only two issues I've seen reported are #13359 (which I believe @ichard26 doesn't think warrants a bugfix release) and #13353 (which only affects the legacy resolver). I'm ambivalent, personally. Neither of the issues seem particularly critical to me, but a bigfix release isn't that hard to do. Either way, I'd like to make a decision by the weekend, so that if we do release a 25.1.1, it can go into CPython 3.14b1 (due on Tuesday). I don't intend to keep the 25.1 release cycle open beyond that point, as if there were any serious issues, I think they would have been reported by now. |
|
I would personally warrant both of them worthy of a bug fix release, if nothing else but to stem the long tail of users upgrading over the next 3 months (and longer as old versions of pip get released on various distribution channels) posting comments confirming they are also affected. Just from the existing comments it appears Python 3.9.0-3.9.6 and the legacy resolver are more wide spread than expected. But I'm not the one who has to do a release, and I don't feel super strongly on this, so happy to defer to you. |
|
I'd also (@notatallshaw ?) consider doing something with ResolutionTooDeep to at most go back to previous behaviour until we improve the new "shortcuts" - we've been discussing a scenario where Airflow in their default setup for Python 3.12 will get ResolutionTooDeep and our attempts to contain it - so far at least - failed. #13281 (comment) It's not a huge issue, but if we won't be able to find good set of limits for Airflow + 90 providers of ours that will work (the resolution works nicely with Ideally - we'd love to find out how we can limit it - but after 3 or 4 attempts I do not see many more options to try for now. Again - nothing "alarmy" so might not be worth it, we already recommend our users to use constraints for Airflow installation - and those constraints are resolved and generated by |
|
Since my original reply on #13359, my opinion has shifted to that it's probably better to cut a bugfix release as more people are encountering the crash than I was expecting when I wrote my reply. |
|
@potiuk the behavior isn't controlled by pip, it's controlled by resolvelib, so there's no chance of getting it changed for the pip 25.1 cycle, without reverting the vendorong of resolvelib, which would reintroduce the resolution impossible bugs. I am supportive of you pinning to <25.1 for now. I have a similar proposal on the resolvelib side: sarugaku/resolvelib#183, the approach is to use the old behavior until getting a Resolution Impossible and then trying with the new behavior. The reason I've been hesitant is I have a real world example where the old behavior produced a resolution too deep but the new behavior doesn't (and it happens to be an airflow user): #12754 (comment). Perhaps I could change the optimization to catch both resolution too deep and resolution impossible, but this would bring semantic questions about what max backtracking rounds means. Anyway, we should discuss on resolvelib side if you have opinions on this. Btw, once I have all your scenarios reproducible on my side I will see if there is some specific "knot" of requirements causing the problems, my experience is these are not always obvious, and if so I will raise a PR on airflow side to help out. |
|
Yeah, I'm not going to accept a new resolvelib release as part of a bugfix release of pip. And I don't consider tl;dr; I agree with @notatallshaw But there will be a 25.1.1, containing the (few) fixes that have been made this week. I'll cut it later in the week, maybe Friday (depending on my personal life commitments) from the main branch. Thanks all, for the feedback. And I'm taking a risk by saying this before cutting 25.1.1, but thanks to everyone who contributed to 25.1, there are a bunch of new features and remarkably few problems! |
|
Cool. I am fine with that - we'll pin
That would be cool, and I am happy to iterate and work on it as well - I just need a bit of guidance - maybe there is some debugging I can turn on to get a bit more intelligent guessing on what could be the problem ? Let's work on it offline - this is not related to this release discussion any more. |
|
I have no opinion on the serverity of this, but should at least be considered for 25.1.1: #13373 |
|
#13372 seems pretty bad to me, but reports haven't been pouring in. As the person who wrote the original broken thing, obviously I'm in favor of getting the fix in! If 25.1.1 is the version which will show up in CPython releases for the 3.14 cycle, I'd like people's first experience of |
|
I’m fine with adding it, I’ll merge it before I cut the bugfix, unless someone else does so first. |
|
Yeah, was going to mention #13363 also. |
|
Thanks, I had missed that one 🙁 I was looking at what was reported here, and not at the milestone... |
|
... and that's 25.1.1 released. One minor hiccup in the get-pip release, caused by a bug in the version of nox released yesterday. I just amended the workflows to avoid that specific nox release, as that seemed like the easiest fix. I've updated the CPython PR, and will merge it once the tests have passed. Short of some major disaster, I think that's the 25.1 release cycle complete. |
I'm mostly filing this now as I have some interest in being the next RM. Both to increase the maintainers available to do a release, and to give @sbidoul a break :)
There are several deprecations scheduled for removal in the 25.1 milestone which definitely makes it somewhat precarious, but I'd be happy to at least learn the mechanical parts and have someone else help me through the decision-making. pip 25.3 (October) is likely going to be a busy time for me, so 25.2 would be the best time for me to learn the release flow.
Any objections or thoughts? cc @pypa/pip-committers
The text was updated successfully, but these errors were encountered: