Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

argparse nargs=-1 fails on python 3.12.7 #126708

Closed
ikappaki opened this issue Nov 11, 2024 · 5 comments
Closed

argparse nargs=-1 fails on python 3.12.7 #126708

ikappaki opened this issue Nov 11, 2024 · 5 comments
Labels
pending The issue will be closed if no feedback is provided stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@ikappaki
Copy link

ikappaki commented Nov 11, 2024

Bug report

Bug description:

It seems there's a pattern in the wild with argparse where nargs=-1 is used to capture the remaining command line arguments. However, I couldn’t find anything in the argparse nargs documentation to indicate that this is a supported feature, though it appears to be widely used: https://grep.app/search?current=4&q=nargs%3D-1&filter[lang][0]=Python

This pattern has stopped working as of Python v3.12.7, likely due to #124631 which doesn't appear to have made it to v3.13.0, but it has been backported to the 3.13 branch.

To reproduce,

  1. Create an issue.py file
import argparse
import sys
print(f":version {sys.version}")
parser = argparse.ArgumentParser()
parser.add_argument("args", nargs=-1)
parsed_args, extra = parser.parse_known_args()
print(f":args {parsed_args} :extra {extra}")
  1. and run it with python 3.12.7, you should get a the following arguments are required error.
$ d:/local/Python312/python.exe issue.py 5 6
usage: issue.py [-h]
issue.py: error: the following arguments are required: args
:version 3.12.7 (tags/v3.12.7:0b05ead, Oct  1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]

The same works with any other version at the momenet, for example 3.11 or 3.13:

$ python.exe issue.py 5 6
:version 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
:args Namespace(args=[]) :extra ['5', '6']

$ d:/local/Python313/python.exe issue.py 5 6
:version 3.13.0 (tags/v3.13.0:60403a5, Oct  7 2024, 09:38:07) [MSC v.1941 64 bit (AMD64)]
:args Namespace(args=[]) :extra ['5', '6']

Could you please take a look and advise whether this is something you might want to continue supporting, or if there are alternative approaches you would recommend?

I don't personally use this pattern, but I came across it and thought it would be helpful to report, as it might assist developers in making informed decisions about it.

Thanks

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

Windows

@brianschubert
Copy link
Contributor

Just to confirm my understanding, is there any difference between nargs=-1 and the documented nargs="*"?

Looking at the "in the wild" usages in the link you shared, these seem to be predominantly examples of nargs=-1 being used with click, not with argparse (I at least couldn’t find a non-click example in the first 10 pages of results).
I tried refining the search to only include examples for argparse, which yielded only 3 cases, one of which is the repo from the linked issue: https://grep.app/search?q=add_argument%5C%28%5B%5E%29%5D%2Bnargs%3D-1&regexp=true&filter%5Blang%5D%5B0%5D=Python

@hugovk
Copy link
Member

hugovk commented Nov 12, 2024

@serhiy-storchaka
Copy link
Member

In never was intended to support negative nargs values. It only worked by accident, because 'A' * -1 == 'A' * 0 == ''. It worked almost like nargs=0, except that there were special guards against nargs=0. So this was a hack to circumvent argparse sanity checks. It is not surprising that it once stopped to work.

If you search non-hackish solution, use action='store_const', const=[] for options. For positional arguments, remove add_argument() and use set_defaults to set the default value in a namespace. You can also consider using nargs='...', although this feature is currently undocumented.

Thanks @brianschubert for pointing that such pattern is very rarely used with argparse.

@chrisrink10
Copy link

Just to confirm my understanding, is there any difference between nargs=-1 and the documented nargs="*"?

Looking at the "in the wild" usages in the link you shared, these seem to be predominantly examples of nargs=-1 being used with click, not with argparse (I at least couldn’t find a non-click example in the first 10 pages of results). I tried refining the search to only include examples for argparse, which yielded only 3 cases, one of which is the repo from the linked issue: https://grep.app/search?q=add_argument%5C%28%5B%5E%29%5D%2Bnargs%3D-1&regexp=true&filter%5Blang%5D%5B0%5D=Python

Hi all, I am the maintainer of the linked repo and the author of the original offending code. I hadn't had a chance to review the Github search results for the nargs=-1 previously but as @brianschubert suggested that seems to be a Click paradigm, not argparse. Doing a little digging I was able to confirm that it was indeed a Click-ism which I incorrectly translated over to argparse after removing Click.

Thanks to @serhiy-storchaka for suggestions for how to handle the intended use case in a more supported fashion and sorry to waste everyone's time. I think (if @ikappaki agrees) we can just close this ticket.

@picnixz picnixz added stdlib Python modules in the Lib dir pending The issue will be closed if no feedback is provided labels Nov 12, 2024
@ikappaki
Copy link
Author

Hi all, thank you for looking into this. I think this ticket has served its purpose and can be closed. It can still be used as a reference in case someone falls for the same.

Apologies for the shallow impact analysis with grep.app, which suggested the issue was more widespread than it actually is. I was misled by the similarity between click's arguments and those of argparse 😅

@hugovk hugovk closed this as not planned Won't fix, can't repro, duplicate, stale Nov 12, 2024
@github-project-automation github-project-automation bot moved this to Doc issues in Argparse issues Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pending The issue will be closed if no feedback is provided stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Status: Doc issues
Development

No branches or pull requests

6 participants