Skip to content

fix(markers): only parse versions on certain keys #939

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+31 −5
Open

fix(markers): only parse versions on certain keys #939

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions src/packaging/markers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@
Operator = Callable[[str, Union[str, AbstractSet[str]]], bool]
EvaluateContext = Literal["metadata", "lock_file", "requirement"]
MARKERS_ALLOWING_SET = {"extras", "dependency_groups"}
MARKERS_REQUIRING_VERSION = {
"python_version",
"python_full_version",
"implementation_version",
}


class InvalidMarker(ValueError):
Expand Down Expand Up @@ -177,16 +182,17 @@ def _format_marker(
}


def _eval_op(lhs: str, op: Op, rhs: str | AbstractSet[str]) -> bool:
if isinstance(rhs, str):
def _eval_op(lhs: str, op: Op, rhs: str | AbstractSet[str], *, key: str) -> bool:
op_str = op.serialize()
if key in MARKERS_REQUIRING_VERSION:
try:
spec = Specifier("".join([op.serialize(), rhs]))
spec = Specifier("".join([op_str, rhs]))
except InvalidSpecifier:
pass
else:
return spec.contains(lhs, prereleases=True)
Comment on lines 188 to 193
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing the only reason this try/except is here is to attempt to address the issue you've discovered, but as a result has hidden several problems. I think the correct course is:

  • Remove this try/except
  • Keep the spec = Specifier(...) and spec.contains otherwise, and let it error with InvalidSpecifier. This matches the spec:

Otherwise an error should be raised. e.g. the following will result in errors:
python_version ~= "surprise"

Currently packaging passes on Marker("python_full_version == 'surprise'").evaluate(), when my reading is that it should error. Removing this try/except would let this error correctly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, perhaps not; == is a valid python operator, and this example does error on ~=:

If there is no defined behaviour of this specification and the operator exists in Python, then the operator falls back to the Python behaviour for the types involved.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While == is a valid Python operator, it also has special meaning when comparing versions. In particular, it allows for zero padding, so that 1, 1.0 and 1.0.0 are all treated as equal under the version-equality check.

There's also a mismatch in the behaviour when a glob is used, e.g. 1.1.*.

Under the hood, the Version class does overload the __eq__ operator, but only to check equality of the underlying _key tuple. As such, it does not support globbing or 0-padding.


oper: Operator | None = _operators.get(op.serialize())
oper: Operator | None = _operators.get(op_str)
if oper is None:
raise UndefinedComparison(f"Undefined {op!r} on {lhs!r} and {rhs!r}.")

Expand Down Expand Up @@ -234,9 +240,10 @@ def _evaluate_markers(
lhs_value = lhs.value
environment_key = rhs.value
rhs_value = environment[environment_key]

assert isinstance(lhs_value, str), "lhs must be a string"
lhs_value, rhs_value = _normalize(lhs_value, rhs_value, key=environment_key)
groups[-1].append(_eval_op(lhs_value, op, rhs_value))
groups[-1].append(_eval_op(lhs_value, op, rhs_value, key=environment_key))
else:
assert marker in ["and", "or"]
if marker == "or":
Expand Down
19 changes: 19 additions & 0 deletions tests/test_markers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -426,3 +426,22 @@ def test_extras_and_dependency_groups_disallowed(self, variable):

with pytest.raises(KeyError):
marker.evaluate(context="requirement")

@pytest.mark.parametrize(
("marker_string", "environment", "expected"),
[
('extra == "v2"', None, False),
('extra == "v2"', {"extra": ""}, False),
('extra == "v2"', {"extra": "v2"}, True),
('extra == "v2"', {"extra": "v2a3"}, False),
('extra == "v2a3"', {"extra": "v2"}, False),
('extra == "v2a3"', {"extra": "v2a3"}, True),
],
)
def test_version_like_equality(self, marker_string, environment, expected):
"""
Test for issue #938: Extras are meant to be literal strings, even if
they look like versions, and therefore should not be parsed as version.
"""
marker = Marker(marker_string)
assert marker.evaluate(environment) is expected