[ty] Infer typevar specializations for Callable types
#21551
+212
−41
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-11-26 23:39:34.860924727 +0000
+++ new-output.txt 2025-11-26 23:39:38.241947151 +0000
@@ -1,4 +1,5 @@
_directives_deprecated_library.py:15:31: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
+_directives_deprecated_library.py:30:5: error[invalid-overload] Overloaded function `foo` requires at least two overloads
_directives_deprecated_library.py:30:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
_directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__add__`
_directives_deprecated_library.py:41:25: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int | float`
@@ -166,7 +167,7 @@
callables_protocol.py:169:7: error[invalid-assignment] Object of type `def cb8_bad1(x: int) -> Any` is not assignable to `Proto8`
callables_protocol.py:186:5: error[invalid-assignment] Object of type `Literal["str"]` is not assignable to attribute `other_attribute` of type `int`
callables_protocol.py:187:5: error[unresolved-attribute] Unresolved attribute `xxx` on type `Proto9[P@decorator1, R@decorator1]`.
-callables_protocol.py:197:7: error[unresolved-attribute] Object of type `Proto9[Unknown, Unknown]` has no attribute `other_attribute2`
+callables_protocol.py:197:7: error[unresolved-attribute] Object of type `Proto9[Unknown, str] | Proto9[Unknown, Unknown]` has no attribute `other_attribute2`
callables_protocol.py:238:8: error[invalid-assignment] Object of type `def cb11_bad1(x: int, y: str, /) -> Any` is not assignable to `Proto11`
callables_protocol.py:260:8: error[invalid-assignment] Object of type `def cb12_bad1(*args: Any, *, kwarg0: Any) -> None` is not assignable to `Proto12`
callables_protocol.py:284:27: error[invalid-assignment] Object of type `def cb13_no_default(path: str) -> str` is not assignable to `Proto13_Default`
@@ -248,29 +249,22 @@
constructors_call_type.py:40:5: error[missing-argument] No arguments provided for required parameters `x`, `y` of function `__new__`
constructors_call_type.py:50:5: error[missing-argument] No arguments provided for required parameters `x`, `y` of bound method `__init__`
constructors_call_type.py:59:9: error[too-many-positional-arguments] Too many positional arguments to bound method `__init__`: expected 1, got 2
-constructors_callable.py:36:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:37:1: error[type-assertion-failure] Type `Class1` does not match asserted type `Unknown`
-constructors_callable.py:49:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:50:1: error[type-assertion-failure] Type `Class2` does not match asserted type `Unknown`
+constructors_callable.py:36:13: info[revealed-type] Revealed type: `(...) -> Class1`
+constructors_callable.py:49:13: info[revealed-type] Revealed type: `(...) -> Class2`
constructors_callable.py:57:42: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__new__`
constructors_callable.py:63:13: info[revealed-type] Revealed type: `(...) -> Unknown`
constructors_callable.py:64:1: error[type-assertion-failure] Type `Class3` does not match asserted type `Unknown`
constructors_callable.py:73:33: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
-constructors_callable.py:77:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:78:1: error[type-assertion-failure] Type `int` does not match asserted type `Unknown`
-constructors_callable.py:97:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:100:5: error[type-assertion-failure] Type `Never` does not match asserted type `Unknown`
-constructors_callable.py:105:5: error[type-assertion-failure] Type `Never` does not match asserted type `Unknown`
-constructors_callable.py:125:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:126:1: error[type-assertion-failure] Type `Class6Proxy` does not match asserted type `Unknown`
+constructors_callable.py:77:13: info[revealed-type] Revealed type: `(...) -> int`
+constructors_callable.py:97:13: info[revealed-type] Revealed type: `(...) -> Never`
+constructors_callable.py:125:13: info[revealed-type] Revealed type: `(...) -> Class6Proxy`
constructors_callable.py:142:13: info[revealed-type] Revealed type: `(...) -> Unknown`
constructors_callable.py:162:5: info[revealed-type] Revealed type: `(...) -> Unknown`
constructors_callable.py:164:1: error[type-assertion-failure] Type `Class7[int]` does not match asserted type `Unknown`
constructors_callable.py:165:1: error[type-assertion-failure] Type `Class7[str]` does not match asserted type `Unknown`
-constructors_callable.py:182:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:183:1: error[type-assertion-failure] Type `Class8[str]` does not match asserted type `Unknown`
-constructors_callable.py:193:13: info[revealed-type] Revealed type: `(...) -> Unknown`
-constructors_callable.py:194:1: error[type-assertion-failure] Type `Class9` does not match asserted type `Unknown`
+constructors_callable.py:182:13: info[revealed-type] Revealed type: `(...) -> Top[Class8[Unknown]]`
+constructors_callable.py:183:1: error[type-assertion-failure] Type `Class8[str]` does not match asserted type `Top[Class8[Unknown]]`
+constructors_callable.py:193:13: info[revealed-type] Revealed type: `(...) -> Class9`
dataclasses_descriptors.py:23:62: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int | Desc1`
dataclasses_descriptors.py:50:63: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `list[T@Desc2] | T@Desc2`
dataclasses_descriptors.py:66:1: error[type-assertion-failure] Type `int` does not match asserted type `int | Desc2[int]`
@@ -371,11 +365,8 @@
directives_cast.py:16:13: error[invalid-type-form] Int literals are not allowed in this context in a type expression
directives_cast.py:17:22: error[too-many-positional-arguments] Too many positional arguments to function `cast`: expected 2, got 3
directives_deprecated.py:18:44: warning[deprecated] The class `Ham` is deprecated: Use Spam instead
-directives_deprecated.py:24:9: warning[deprecated] The function `norwegian_blue` is deprecated: It is pining for the fjords
-directives_deprecated.py:25:13: warning[deprecated] The function `norwegian_blue` is deprecated: It is pining for the fjords
+directives_deprecated.py:30:13: error[invalid-argument-type] Argument to function `foo` is incorrect: Expected `str`, found `Literal[1]`
directives_deprecated.py:34:7: warning[deprecated] The class `Ham` is deprecated: Use Spam instead
-directives_deprecated.py:69:1: warning[deprecated] The function `lorem` is deprecated: Deprecated
-directives_deprecated.py:98:7: warning[deprecated] The function `foo` is deprecated: Deprecated
directives_no_type_check.py:15:14: error[invalid-assignment] Object of type `Literal[""]` is not assignable to `int`
directives_no_type_check.py:29:7: error[invalid-argument-type] Argument to function `func1` is incorrect: Expected `int`, found `Literal[b"invalid"]`
directives_no_type_check.py:29:19: error[invalid-argument-type] Argument to function `func1` is incorrect: Expected `str`, found `Literal[b"arguments"]`
@@ -408,7 +399,7 @@
enums_member_values.py:96:1: error[type-assertion-failure] Type `int` does not match asserted type `Unknown`
enums_members.py:82:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `Unknown | ((x) -> Unknown)`
enums_members.py:82:37: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
-enums_members.py:83:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `Unknown | ((...) -> Unknown)`
+enums_members.py:83:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `Unknown | ((...) -> int)`
enums_members.py:83:37: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:84:1: error[type-assertion-failure] Type `Unknown` does not match asserted type `property`
enums_members.py:84:35: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
@@ -1031,4 +1022,4 @@
typeddicts_usage.py:28:17: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_usage.py:28:18: error[invalid-key] Unknown key "title" for TypedDict `Movie`: Unknown key "title"
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions
-Found 1033 diagnostics
+Found 1024 diagnostics
|
|
dcreager
force-pushed
the
dcreager/callable-return
branch
from
965a9f8 to
dbecc68
Compare
dcreager
changed the title
Callable return typesCallable types
dcreager
force-pushed
the
dcreager/callable-return
branch
from
7995e43 to
f89ec1a
Compare
dcreager
commented
Nov 26, 2025
| return fn(value) | ||
|
|
||
| def identity(x: T) -> T: | ||
| def identity(x: T, /) -> T: |
Member
Author
There was a problem hiding this comment.
This is an interesting nuance: Callable[[A], B] creates a signature containing positional-only parameters, so we have to make the signature here positional-only as well to make identity pass the assignability check for it to be a valid argument to the fn parameter.
(That said, I'm not convinced that's...correct? Are we mixing up the ordering of the assignability check operands somewhere?)
Comment on lines
390
to
391
| # TODO: this should be `Unknown | int` | ||
| reveal_type(invoke(head, [1, 2, 3])) # revealed: Unknown |
Member
Author
There was a problem hiding this comment.
This TODO is not also removed because we end up inferring this constraint set when comparing head to Callable[[A], B]:
(B@invoke ≤ T@head) ∧ (list[T@head] ≤ A@invoke)
We then try to remove T@head from the constraint set by calculating
∃T@head ⋅ (B@invoke ≤ T@head) ∧ (list[T@head] ≤ A@invoke)
We should be able to pick T@head = B@invoke and simplify that to
(B@invoke = *) ∧ (list[B@invoke] ≤ A@invoke)
which I think would then be enough to propagate through the return type to discharge this TODO. I think this would require adding more derived facts to the sequent map.
|
|
||
| x12: Y[Y[Literal[1]]] = [[1]] | ||
| reveal_type(x12) # revealed: list[Y[Literal[1]]] | ||
| reveal_type(x12) # revealed: list[list[Literal[1]]] |
Member
Author
There was a problem hiding this comment.
This is because we're now using specialize_recursive instead of specialize_partial.
Comment on lines
+1680
to
+1718
| let when = formal_signature.when_constraint_set_assignable_to( | ||
| self.db, | ||
| actual_signature, | ||
| self.inferable, | ||
| ); | ||
| when.for_each_path(self.db, |path| { | ||
| for constraint in path.positive_constraints() { | ||
| let typevar = constraint.typevar(self.db); | ||
| let lower = constraint.lower(self.db); | ||
| let upper = constraint.upper(self.db); | ||
| if !upper.is_object() { | ||
| self.add_type_mapping(typevar, upper, polarity, &mut f); | ||
| } | ||
| if let Type::TypeVar(lower_bound_typevar) = lower { | ||
| self.add_type_mapping( | ||
| lower_bound_typevar, | ||
| Type::TypeVar(typevar), | ||
| polarity, | ||
| &mut f, | ||
| ); | ||
| } | ||
| } | ||
| }); |
Member
Author
There was a problem hiding this comment.
This is the meat of the change. We use the new ConstraintAssignability type relation here to calculate a constraint set describing when the two callables are assignable. That will recurse into any typevars in the callables, and find whatever constraint set allows them to unify.
We then use this new for_each_path method to find each way that constraint set can be satisfied, and add new typevar bindings to the old solver for whatever we find.
Comment on lines
+968
to
+1020
| result.intersect( | ||
| db, | ||
| ConstraintSet::from( | ||
| relation.is_assignability() || relation.is_constraint_set_assignability(), | ||
| ), | ||
| ); | ||
| return result; |
Member
Author
There was a problem hiding this comment.
We need to combine this with result to hang on to any typevar mapping our constraint set has recorded for the return type comparison above. That's need to support something like
Callable[[int], int] ≤ Callable[..., T]
and have it return int ≤ T.
Comment on lines
+1751
to
+1973
| // A typevar satisfies a relation when...it satisfies the relation. Yes that's a | ||
| // tautology! We're moving the caller's subtyping/assignability requirement into a | ||
| // constraint set. If the typevar has an upper bound or constraints, then the relation | ||
| // only has to hold when the typevar has a valid specialization (i.e., one that | ||
| // satisfies the upper bound/constraints). |
Member
Author
There was a problem hiding this comment.
This is what #20093 is trying to add to replace assignability across the board. In the meantime, I've added a new TypeRelation that lets us opt into the new behavior only in certain places.
dcreager
force-pushed
the
dcreager/callable-return
branch
from
3868294 to
2c62674
Compare
dcreager
requested review from
AlexWaygood,
carljm and
sharkdp
as code owners
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-argument-type |
1,565 | 13 | 96 |
non-subscriptable |
337 | 0 | 0 |
deprecated |
0 | 255 | 0 |
unresolved-attribute |
64 | 125 | 3 |
possibly-missing-attribute |
141 | 3 | 24 |
invalid-return-type |
80 | 0 | 5 |
no-matching-overload |
70 | 15 | 0 |
type-assertion-failure |
3 | 19 | 36 |
invalid-assignment |
43 | 0 | 11 |
unsupported-operator |
8 | 0 | 26 |
unused-ignore-comment |
16 | 17 | 0 |
call-non-callable |
30 | 0 | 0 |
possibly-unresolved-reference |
26 | 4 | 0 |
missing-argument |
27 | 0 | 0 |
unknown-argument |
22 | 0 | 0 |
invalid-overload |
9 | 1 | 0 |
invalid-type-form |
10 | 0 | 0 |
not-iterable |
7 | 0 | 0 |
invalid-method-override |
0 | 5 | 0 |
unsupported-base |
2 | 0 | 0 |
invalid-await |
1 | 0 | 0 |
invalid-context-manager |
1 | 0 | 0 |
invalid-raise |
1 | 0 | 0 |
redundant-cast |
1 | 0 | 0 |
| Total | 2,464 | 457 | 201 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a first stab at solving astral-sh/ty#500, at least in part, with the old solver. We add a new
TypeRelationthat lets us opt into using constraint sets to describe when a typevar is assignability to some type, and then use that to calculate a constraint set that describes when two callable types are assignable. If the callable types contain typevars, that constraint set will describe their valid specializations. We can then walk through all of the ways the constraint set can be satisfied, and record a type mapping in the old solver for each one.