Skip to content

Conversation

@JP-Ellis
Copy link

@JP-Ellis JP-Ellis commented Oct 7, 2025

According to the environment markers, most markers are strings, with only a small subset being use to handle versions. As such, this PR changes the behaviour to use version comparison only on those keys which are dealing with versions.

Note that the choice of only doing version comparison on keys which are defined as being versions results in one failure for a custom key:

    def test_prefers_pep440(self):
>       assert Marker('"2.7.9" < "foo"').evaluate(dict(foo="2.7.10"))
E       assert False
E        +  where False = evaluate({'foo': '2.7.10'})
E        +    where evaluate = <Marker('"2.7.9" < "foo"')>.evaluate
E        +      where <Marker('"2.7.9" < "foo"')> = Marker('"2.7.9" < "foo"')
E        +    and   {'foo': '2.7.10'} = dict(foo='2.7.10')

From my reading of the specification, it is unclear whether the use of version comparison in this case is meant to be supported or not.

I can see a few ways of moving forward

  1. If version comparisons are only meant to be used with the defined set of keys, then this PR is ready to go, requiring only that the one test be changed to expect failure.

  2. If version comparison should be the default, and only some keys are exempted from the version comparison, then we can invert the logic in _eval_op from an if key in MARKERS_REQUIRING_VERSION: to an if key in MARKERS_REQUIRING_STRING_COMPARISON. If this is the case, then we would need to define what those keys are.

  3. An intermediate approach would be to inspect the operator and, for example, using version comparison for <, <=, >=, >, ~= and ===, and string comparison for == and !=. I don't like this approach as it muddies the logic, and I think it would be better to be explicit about which keys are to be treated as versions.

Resolves: #938
Replaces: #932 (this is a different approach)

According to the [environment
markers](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers),
most markers are strings, with only a small subset being use to handle
versions.

This commit ensures that only those keys which _are_ versions get
compared as versions, and all other keys are compared as string
literals.

Signed-off-by: JP-Ellis <[email protected]>
Copy link
Contributor

@Liam-DeVoe Liam-DeVoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaces: #932 (this is a different approach)

I don't think this supersedes #932, because it's still possible to hit InvalidVersion with manual use of SpecifierSet (eg the example in #767). FWIW, the issue in this pull would actually be fixed by #932, though I think the explicit specification of markers which require a version here is good to have regardless.

Comment on lines 188 to 193
try:
spec = Specifier("".join([op.serialize(), rhs]))
spec = Specifier("".join([op_str, rhs]))
except InvalidSpecifier:
pass
else:
return spec.contains(lhs, prereleases=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing the only reason this try/except is here is to attempt to address the issue you've discovered, but as a result has hidden several problems. I think the correct course is:

  • Remove this try/except
  • Keep the spec = Specifier(...) and spec.contains otherwise, and let it error with InvalidSpecifier. This matches the spec:

Otherwise an error should be raised. e.g. the following will result in errors:
python_version ~= "surprise"

Currently packaging passes on Marker("python_full_version == 'surprise'").evaluate(), when my reading is that it should error. Removing this try/except would let this error correctly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, perhaps not; == is a valid python operator, and this example does error on ~=:

If there is no defined behaviour of this specification and the operator exists in Python, then the operator falls back to the Python behaviour for the types involved.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While == is a valid Python operator, it also has special meaning when comparing versions. In particular, it allows for zero padding, so that 1, 1.0 and 1.0.0 are all treated as equal under the version-equality check.

There's also a mismatch in the behaviour when a glob is used, e.g. 1.1.*.

Under the hood, the Version class does overload the __eq__ operator, but only to check equality of the underlying _key tuple. As such, it does not support globbing or 0-padding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invalid version error with evaluating markers

2 participants