Skip to content

Commit

Permalink
Replace nb::typed with a more intuitive interface
Browse files Browse the repository at this point in the history
  • Loading branch information
wjakob committed Feb 23, 2024
1 parent 624b63c commit 5ab4b9c
Show file tree
Hide file tree
Showing 13 changed files with 191 additions and 142 deletions.
52 changes: 11 additions & 41 deletions docs/api_core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1064,13 +1064,6 @@ Wrapper classes

Iterator inequality comparison operator.

.. cpp:class:: template <typename T> iterator_t : public iterator

Type-parameterized form of the iterator class. Note that the use of this
class does not enable any kind of type checking within nanobind. It is
mainly useful to annotate arguments and return values so that they are
rendered as ``collections.abc.Iterator[T]`` in typing :ref:`stubs <stubs>`.

.. cpp:class:: iterable : public object

Wrapper class representing an object that can be iterated upon (in the sense
Expand Down Expand Up @@ -2572,14 +2565,11 @@ Miscellaneous
is not available (e.g., for custom binding-specific constructors that don't
exist in `Target` type).

.. cpp:struct:: template <typename T, typename D> typed
.. cpp:class:: template <typename T, typename... Ts> typed

This helper class provides an an API for overriding the type
annotation of a function argument or return value in generated
docstrings. It is particularly helpful when the type signature is
not obvious and must be computed at compile time. Otherwise, the
:cpp:class:`signature` attribute provides a simpler alternative for
taking full control function type annotations.
This helper class provides an an interface to parameterize generic types to
improve generated Python function signatures (e.g., to turn ``list`` into
``list[MyType]``).

Consider the following binding that iterates over a Python list.

Expand All @@ -2591,36 +2581,16 @@ Miscellaneous
}
});
Suppose that ``f`` expects a list of ``Foo`` objects, which is not clear
from the signature. To improve the function signature, use the
``nb::typed<T, D>`` wrapper class to pass the argument.

The template argument `T` should be set to the original argument type, and
`D` points to a helper class that will be used to compute the type name at
compile time. Any access to the list ``l`` must be replaced by ``l.value``:
Suppose that ``f`` expects a list of ``MyType`` objects, which is not clear
from the signature. To make this explicit, use the ``nb::typed<T, Ts...>``
wrapper to pass additional type parameters. This has no effect besides
clarifying the signature---in particular, nanobind does *not* insert
additional runtime checks!

.. code-block:: cpp
m.def("f", [](nb::typed<nb::list, FooListName> l) {
for (nb::handle h : l.value) {
m.def("f", [](nb::typed<nb::list, MyType> l) {
for (nb::handle h : l) {
// ...
}
});
In this simple example, the ``FooListName`` type can be defined as follows:

.. code-block:: cpp
struct FooListName {
static constexpr auto Name =
nb::detail::const_name("list[") +
nb::detail::const_name<Foo>() +
nb::detail::const_name("]");
};
More generally, `D` can be a templated class with partial overloads,
which allows for advanced constructions.

.. cpp:member:: T value

Wrapped value of the `typed` parameter.
15 changes: 9 additions & 6 deletions docs/api_extra.rst
Original file line number Diff line number Diff line change
Expand Up @@ -373,8 +373,9 @@ include directive:
:cpp:class:`keep_alive` annotation is needed to tie the lifetime of the
parent container to that of the iterator.

The return value is a typed iterator (:cpp:class:`iterator_t`), whose
template parameter is given by the type of ``*first``.
The return value is a typed iterator (:cpp:class:`iterator` wrapped using
:cpp:class:`typed`), whose template parameter is given by the type of
``*first``.

Here is an example of what this might look like for a STL vector:

Expand Down Expand Up @@ -402,8 +403,9 @@ include directive:
key-value pairs. `make_key_iterator` returns the first pair element to
iterate over keys.

The return value is a typed iterator (:cpp:class:`iterator_t`), whose
template parameter is given by the type of ``(*first).first``.
The return value is a typed iterator (:cpp:class:`iterator` wrapped using
:cpp:class:`typed`), whose template parameter is given by the type of
``(*first).first``.


.. cpp:function:: template <rv_policy Policy = rv_policy::reference_internal, typename Iterator, typename... Extra> iterator make_value_iterator(handle scope, const char * name, Iterator &&first, Iterator &&last, Extra &&...extra)
Expand All @@ -412,8 +414,9 @@ include directive:
key-value pairs. `make_value_iterator` returns the second pair element to
iterate over values.

The return value is a typed iterator (:cpp:class:`iterator_t`), whose
template parameter is given by the type of ``(*first).second``.
The return value is a typed iterator (:cpp:class:`iterator` wrapped using
:cpp:class:`typed`), whose template parameter is given by the type of
``(*first).second``.

N-dimensional array type
------------------------
Expand Down
19 changes: 16 additions & 3 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,25 @@ API and ABI level, requiring new major version according to `SemVer

- The ability to override the combined docstring and overload signature listing
with a raw string (formerly ``nb::raw_doc``) was replaced with a more
fine-grained annotation named :cpp:class:`nb::signature`.
fine-grained annotation named :cpp:class:`nb::signature <signature>`.

The new interface enables changing the signature of individual overloads
without touching the docstring part. This change was needed by the new stub
generator. Existing use of ``nb::raw_doc`` must be reworked into this format,
see :ref:`here <fsig_override>` for an example.
see :ref:`here <fsig_override>` for an example. This is an API-breaking change.

- The behavior of the :cpp:class:`nb::typed\<T, Ts...\> <typed>` wrapper was
changed to make this feature equivalent to parameterization of generic types
in in Python -- for example

.. code-block:: cpp
m.def("f", [](nb::typed<nb::mapping, int, nb::str> list) { ... });
produces ``collections.abc.Mapping[int, str]`` in generated stubs.
Previously, this feature required that the user provide a custom type
formatting implementation, which was somewhat awkward to use. This is an
API-breaking change.

- The release improves many type caster so that they produce more accurate type
signatures. For example, the STL ``std::vector<T>`` type caster now renders
Expand Down Expand Up @@ -71,7 +84,7 @@ Version 1.9.2 (Feb 23, 2024)

* :cpp:func:`nb::try_cast() <try_cast>` no longer crashes the interpreter when
attempting to cast a Python ``None`` to a C++ type that was bound using
:cpp:class:`nb::class_<...> <class_>`. Previously this would raise an
:cpp:class:`nb::class_\<...\> <class_>`. Previously this would raise an
exception from the cast operator, which would result in a call to
``std::terminate()`` because :cpp:func:`try_cast() <try_cast>` is declared
``noexcept``. (PR `#386 <https://github.com/wjakob/nanobind/pull/386>`__).
Expand Down
4 changes: 1 addition & 3 deletions docs/exchanging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -390,9 +390,7 @@ directives:
:cpp:class:`dict`, :cpp:class:`ellipsis`, :cpp:class:`handle`,
:cpp:class:`handle_t\<T\> <handle_t>`,
:cpp:class:`bool_`, :cpp:class:`int_`, :cpp:class:`float_`,
:cpp:class:`iterable`,
:cpp:class:`iterator`,
:cpp:class:`iterator_t`,
:cpp:class:`iterable`, :cpp:class:`iterator`,
:cpp:class:`list`, :cpp:class:`mapping`,
:cpp:class:`module_`, :cpp:class:`object`, :cpp:class:`set`, :cpp:class:`sequence`,
:cpp:class:`slice`, :cpp:class:`str`, :cpp:class:`tuple`,
Expand Down
97 changes: 97 additions & 0 deletions docs/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -475,3 +475,100 @@ The following interactive session shows how to call them from Python.
This functionality is very useful when generating bindings for callbacks in
C++ libraries (e.g. GUI libraries, asynchronous networking libraries,
etc.).

.. _typing_generics:

Parameterizing generic types
----------------------------

Various standard Python types are `generic
<https://typing.readthedocs.io/en/latest/spec/generics.html>`__ can be
parameterized to improve the effectiveness of static type checkers such as
`MyPy <https://github.com/python/mypy>`__. In the presence of such a
specialization, a type checker can, e.g., infer that the variable ``a`` below
is of type ``int``.

.. code-block:: python
def f() -> list[int]: ...
a = f()[0]
This is even supported for *abstract types*---for example,
``collections.abc.Mapping[str, int]`` indicates an abstract mapping from
strings to integers.

nanobind provides the template class :cpp:class:`nb::typed\<T, Ts...\> <typed>`
to generate parameterized type annotations in C++ bindings. For example, the
argument and return value of the following function binding reproduces the
exact list and mapping types mentioned above.

.. code-block:: cpp
m.def("f", [](nb::typed<nb::mapping, nb::str, int> arg)
-> nb::typed<nb::list, int> { ... });
(Usually, :cpp:class:`nb::typed\<T, Ts...\> <typed>` would be applied to
:ref:`wrapper <wrappers>` types, though this is not a strict limitation.)

An important limitation of this feature is that it *only* affects function
signatures. Nanobind will (as always) ensure that ``f`` can only be called with
a ``nb::mapping``, but it will *not* insert additional runtime to verify that
``arg`` indeed maps strings to integers. It is the responsibility of the
function to perform such checks at runtime and, if needed, raise a
:cpp:func:`nb::type_error <type_error>`.

The parameterized C++ type :cpp:class:`nb::typed\<T, Ts...\> <typed>`
subclasses the type ``T`` and can be used interchangeably with ``T``. The other
arguments (``Ts...``) are used to generate a Python type signature but have no
other effect (for example, a parameterizing by ``str`` on the Python end can
alternatively be achieved by passing ``nb::str``, ``std::string``, or ``const
char*`` as part of the ``Ts..`` parameter pack).

.. _typing_signatures:

Signature customization
-----------------------

In larger binding projects, some customization of function signatures is often
needed so that static type checkers accept the generated stubs. For example,
the following function binding

.. code-block:: cpp
nb::class_<Int>(m, "Int")
.def(nb::self == nb::self);
is likely to be rejected because the nanobind-derived ``__eq__`` function
signature

.. code-block:: text
__eq__(self, arg: Int, /) -> bool
is more specific than that of the parent class ``object``:

.. code-block:: text
__eq__(self, arg: object, /) -> bool
In this case, `MyPy <https://github.com/python/mypy>`__, e.g., reports

.. code-block:: text
error: Argument 1 of "__eq__" is incompatible with supertype "object"; supertype defines the argument type as "object" [override]
To handle such cases, you can use the :cpp:class:`nb::signature <signature>`
function binding attribute, which overrides the complete function signature
with a custom string.

.. code-block:: cpp
nb::class_<Int>(m, "Int")
.def(nb::self == nb::self,
nb::signature("__eq__(self, arg: object, /) -> bool"));
Note that this *must* be a valid Python function signature of the form
``name(...) -> ...``, where ``name`` must furthermore match the name given to
the binding declaration (this comment applies to ``.def("name", ...)``-style
bindings with explicit name).
46 changes: 6 additions & 40 deletions docs/stubs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -218,47 +218,13 @@ Note that for now, the ``nanobind.stubgen.StubGen`` API is considered
experimental and not subject to the semantic versioning policy used by the
nanobind project.

Signature customization
-----------------------
Customizing type and function signatures
----------------------------------------

In larger binding projects, some customization is often needed so that static
type checkers accept the generated stubs. The stub produced by the following
binding
type checkers accept the generated stubs.

.. code-block:: cpp
Please see the section on :ref:`overriding function signatures
<typing_signatures>` and on :ref:`parameterizing generic types
<typing_generics>` for ideas on this topic.

nb::class_<Int>(m, "Int")
.def(nb::self == nb::self);
is likely rejected because the nanobind-derived ``__eq__`` function signature

.. code-block:: text
__eq__(self, arg: Int, /) -> bool
is more specific than that of the parent class ``object``:

.. code-block:: text
__eq__(self, arg: object, /) -> bool
In this case, MyPy, e.g., reports

.. code-block:: text
error: Argument 1 of "__eq__" is incompatible with supertype "object"; supertype defines the argument type as "object" [override]
To handle such cases, you can use the :cpp:class:`nb::signature <signature>`
function binding attribute, which overrides the complete function signature
with a custom string.

.. code-block:: cpp
nb::class_<Int>(m, "Int")
.def(nb::self == nb::self,
nb::signature("__eq__(self, arg: object, /) -> bool"));
Note that this *must* be a valid Python function signature of the form
``name(...) -> ...``, where ``name`` must furthermore match the name given to
the binding declaration (this comment applies to ``.def("name", ...)``-style
bindings with explicit name).
15 changes: 13 additions & 2 deletions docs/why.rst
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,13 @@ The following lists minor-but-useful additions relative to pybind11.
overload when the underlying Python type object is a subtype of the C++ type
``T``.

Finally, the :cpp:class:`nb::typed\<T, Ts...\> <typed>` annotation can be
used to parameterize any other type. The feature exists to improve the
expressiveness of type signatures (e.g., to turn ``list`` into
``list[int]``). Note, however, that nanobind does not perform additional
runtime checks in this case. Please see the section on :ref:`parameterizing
generics <typing_generics>` for further details.

.. _fsig_override:

- **Function signature overrides**: it may sometimes be necessary to tweak the
Expand All @@ -204,8 +211,8 @@ The following lists minor-but-useful additions relative to pybind11.

For example, the following signature annotation creates an overload that
should only be called with an ``1``-valued integer literal. While the
function also includes a runtime-check, a type checker can now statically
enforce this constraint.
function also includes a runtime-check, a type checker can now already
statically enforce this constraint.

.. code-block:: cpp
Expand All @@ -216,3 +223,7 @@ The following lists minor-but-useful additions relative to pybind11.
return arg;
},
nb::signature("f(arg: typing.Literal[1], /) -> int"));
Please see the section on :ref:`customizing function signatures
<typing_signatures>` for further details.

10 changes: 5 additions & 5 deletions include/nanobind/make_iterator.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,9 @@ template <typename Iterator> struct iterator_value_access {

template <typename Access, rv_policy Policy, typename Iterator,
typename Sentinel, typename ValueType, typename... Extra>
iterator_t<ValueType> make_iterator_impl(handle scope, const char *name,
Iterator &&first, Sentinel &&last,
Extra &&...extra) {
typed<iterator, ValueType> make_iterator_impl(handle scope, const char *name,
Iterator &&first, Sentinel &&last,
Extra &&...extra) {
using State = iterator_state<Access, Policy, Iterator, Sentinel, ValueType, Extra...>;

if (!type<State>().is_valid()) {
Expand All @@ -70,8 +70,8 @@ iterator_t<ValueType> make_iterator_impl(handle scope, const char *name,
Policy);
}

return borrow<iterator_t<ValueType>>(cast(State{ std::forward<Iterator>(first),
std::forward<Sentinel>(last), true }));
return borrow<typed<iterator, ValueType>>(cast(State{
std::forward<Iterator>(first), std::forward<Sentinel>(last), true }));
}

NAMESPACE_END(detail)
Expand Down
Loading

0 comments on commit 5ab4b9c

Please sign in to comment.