-
Notifications
You must be signed in to change notification settings - Fork 273
fix(markers): only parse versions on certain keys #939
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
fix(markers): only parse versions on certain keys #939
Conversation
According to the [environment markers](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers), most markers are strings, with only a small subset being use to handle versions. This commit ensures that only those keys which _are_ versions get compared as versions, and all other keys are compared as string literals. Signed-off-by: JP-Ellis <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replaces: #932 (this is a different approach)
I don't think this supersedes #932, because it's still possible to hit InvalidVersion with manual use of SpecifierSet (eg the example in #767). FWIW, the issue in this pull would actually be fixed by #932, though I think the explicit specification of markers which require a version here is good to have regardless.
| try: | ||
| spec = Specifier("".join([op.serialize(), rhs])) | ||
| spec = Specifier("".join([op_str, rhs])) | ||
| except InvalidSpecifier: | ||
| pass | ||
| else: | ||
| return spec.contains(lhs, prereleases=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm guessing the only reason this try/except is here is to attempt to address the issue you've discovered, but as a result has hidden several problems. I think the correct course is:
- Remove this try/except
- Keep the
spec = Specifier(...)andspec.containsotherwise, and let it error withInvalidSpecifier. This matches the spec:
Otherwise an error should be raised. e.g. the following will result in errors:
python_version ~= "surprise"
Currently packaging passes on Marker("python_full_version == 'surprise'").evaluate(), when my reading is that it should error. Removing this try/except would let this error correctly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, perhaps not; == is a valid python operator, and this example does error on ~=:
If there is no defined behaviour of this specification and the operator exists in Python, then the operator falls back to the Python behaviour for the types involved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While == is a valid Python operator, it also has special meaning when comparing versions. In particular, it allows for zero padding, so that 1, 1.0 and 1.0.0 are all treated as equal under the version-equality check.
There's also a mismatch in the behaviour when a glob is used, e.g. 1.1.*.
Under the hood, the Version class does overload the __eq__ operator, but only to check equality of the underlying _key tuple. As such, it does not support globbing or 0-padding.
According to the environment markers, most markers are strings, with only a small subset being use to handle versions. As such, this PR changes the behaviour to use version comparison only on those keys which are dealing with versions.
Note that the choice of only doing version comparison on keys which are defined as being versions results in one failure for a custom key:
From my reading of the specification, it is unclear whether the use of version comparison in this case is meant to be supported or not.
I can see a few ways of moving forward
If version comparisons are only meant to be used with the defined set of keys, then this PR is ready to go, requiring only that the one test be changed to expect failure.
If version comparison should be the default, and only some keys are exempted from the version comparison, then we can invert the logic in
_eval_opfrom anif key in MARKERS_REQUIRING_VERSION:to anif key in MARKERS_REQUIRING_STRING_COMPARISON. If this is the case, then we would need to define what those keys are.An intermediate approach would be to inspect the operator and, for example, using version comparison for
<,<=,>=,>,~=and===, and string comparison for==and!=. I don't like this approach as it muddies the logic, and I think it would be better to be explicit about which keys are to be treated as versions.Resolves: #938
Replaces: #932 (this is a different approach)