Merge branch 'main' into int_int_inplace

Author: eendebakpt

.github/workflows/reusable-emscripten.yml

@@ -10,7 +10,7 @@ jobs:
   build-emscripten-reusable:
     name: 'build and test'
     runs-on: ubuntu-24.04
-    timeout-minutes: 60
+    timeout-minutes: 40
     steps:
     - uses: actions/checkout@v6
       with:

Doc/data/threadsafety.dat

@@ -123,4 +123,33 @@ PyByteArray_GET_SIZE:atomic:
 
 # Raw data - no locking; mutating it is unsafe if the bytearray object is shared between threads
 PyByteArray_AsString:compatible:
-PyByteArray_AS_STRING:compatible:
+PyByteArray_AS_STRING:compatible:
+
+# Capsule objects (Doc/c-api/capsule.rst)
+
+# Type check - read ob_type pointer, always safe
+PyCapsule_CheckExact:atomic:
+
+# Creation - pure allocation, no shared state
+PyCapsule_New:atomic:
+
+# Validation - reads pointer and name fields; safe on distinct objects
+PyCapsule_IsValid:distinct:
+
+# Getters - read struct fields; safe on distinct objects but
+# concurrent access to the same capsule requires external synchronization
+PyCapsule_GetPointer:distinct:
+PyCapsule_GetName:distinct:
+PyCapsule_GetDestructor:distinct:
+PyCapsule_GetContext:distinct:
+
+# Setters - write struct fields; safe on distinct objects but
+# concurrent access to the same capsule requires external synchronization
+PyCapsule_SetPointer:distinct:
+PyCapsule_SetName:distinct:
+PyCapsule_SetDestructor:distinct:
+PyCapsule_SetContext:distinct:
+
+# Import - looks up a capsule from a module attribute and
+# calls PyCapsule_GetPointer; may call arbitrary code
+PyCapsule_Import:compatible:

Doc/library/array.rst

@@ -42,13 +42,15 @@ defined:
 +-----------+--------------------+-------------------+-----------------------+-------+
 | ``'Q'``   | unsigned long long | int               | 8                     |       |
 +-----------+--------------------+-------------------+-----------------------+-------+
+| ``'e'``   | _Float16           | float             | 2                     | \(3)  |
++-----------+--------------------+-------------------+-----------------------+-------+
 | ``'f'``   | float              | float             | 4                     |       |
 +-----------+--------------------+-------------------+-----------------------+-------+
 | ``'d'``   | double             | float             | 8                     |       |
 +-----------+--------------------+-------------------+-----------------------+-------+
-| ``'F'``   | float complex      | complex           | 8                     | \(3)  |
+| ``'F'``   | float complex      | complex           | 8                     | \(4)  |
 +-----------+--------------------+-------------------+-----------------------+-------+
-| ``'D'``   | double complex     | complex           | 16                    | \(3)  |
+| ``'D'``   | double complex     | complex           | 16                    | \(4)  |
 +-----------+--------------------+-------------------+-----------------------+-------+
 
 
@@ -69,6 +71,15 @@ Notes:
    .. versionadded:: 3.13
 
 (3)
+   The IEEE 754 binary16 "half precision" type was introduced in the 2008
+   revision of the `IEEE 754 standard <ieee 754 standard_>`_.
+   This type is not widely supported by C compilers.  It's available
+   as :c:expr:`_Float16` type, if the compiler supports the Annex H
+   of the C23 standard.
+
+   .. versionadded:: next
+
+(4)
    Complex types (``F`` and ``D``) are available unconditionally,
    regardless on support for complex types (the Annex G of the C11 standard)
    by the C compiler.
@@ -304,3 +315,5 @@ Examples::
 
    `NumPy <https://numpy.org/>`_
       The NumPy package defines another array type.
+
+.. _ieee 754 standard: https://en.wikipedia.org/wiki/IEEE_754-2008_revision

Doc/library/plistlib.rst

@@ -18,7 +18,7 @@ and XML plist files.
 
 The property list (``.plist``) file format is a simple serialization supporting
 basic object types, like dictionaries, lists, numbers and strings.  Usually the
-top level object is a dictionary.
+top level object is a dictionary or a frozen dictionary.
 
 To write out and to parse a plist file, use the :func:`dump` and
 :func:`load` functions.

Doc/library/stdtypes.rst

@@ -3735,12 +3735,13 @@ arbitrary binary data.
    The separator to search for may be any :term:`bytes-like object`.
 
 
-.. method:: bytes.replace(old, new, count=-1, /)
-            bytearray.replace(old, new, count=-1, /)
+.. method:: bytes.replace(old, new, /, count=-1)
+            bytearray.replace(old, new, /, count=-1)
 
    Return a copy of the sequence with all occurrences of subsequence *old*
-   replaced by *new*.  If the optional argument *count* is given, only the
-   first *count* occurrences are replaced.
+   replaced by *new*.  If *count* is given, only the first *count* occurrences
+   are replaced.  If *count* is not specified or ``-1``, then all occurrences
+   are replaced.
 
    The subsequence to search for and its replacement may be any
    :term:`bytes-like object`.
@@ -3750,6 +3751,9 @@ arbitrary binary data.
       The bytearray version of this method does *not* operate in place - it
       always produces a new object, even if no changes were made.
 
+   .. versionchanged:: next
+      *count* is now supported as a keyword argument.
+
 
 .. method:: bytes.rfind(sub[, start[, end]])
             bytearray.rfind(sub[, start[, end]])

Doc/library/xml.etree.elementtree.rst

@@ -691,7 +691,7 @@ Functions
    .. versionadded:: 3.2
 
 
-.. function:: SubElement(parent, tag, attrib={}, **extra)
+.. function:: SubElement(parent, tag, /, attrib={}, **extra)
 
    Subelement factory.  This function creates an element instance, and appends
    it to an existing element.
@@ -705,6 +705,9 @@ Functions
    .. versionchanged:: 3.15
       *attrib* can now be a :class:`frozendict`.
 
+   .. versionchanged:: next
+      *parent* and *tag* are now positional-only parameters.
+
 
 .. function:: tostring(element, encoding="us-ascii", method="xml", *, \
                        xml_declaration=None, default_namespace=None, \
@@ -880,7 +883,7 @@ Element Objects
    :noindex:
    :no-index:
 
-.. class:: Element(tag, attrib={}, **extra)
+.. class:: Element(tag, /, attrib={}, **extra)
 
    Element class.  This class defines the Element interface, and provides a
    reference implementation of this interface.
@@ -893,6 +896,9 @@ Element Objects
    .. versionchanged:: 3.15
       *attrib* can now be a :class:`frozendict`.
 
+   .. versionchanged:: next
+      *tag* is now a positional-only parameter.
+
 
    .. attribute:: tag
 

Doc/whatsnew/3.15.rst

@@ -217,7 +217,8 @@ For example::
 
 The following standard library modules have been updated to accept
 :class:`!frozendict`: :mod:`copy`, :mod:`decimal`, :mod:`json`, :mod:`marshal`,
-:mod:`pickle`, :mod:`pprint` and :mod:`xml.etree.ElementTree`.
+:mod:`plistlib` (only for serialization), :mod:`pickle`, :mod:`pprint` and
+:mod:`xml.etree.ElementTree`.
 
 :func:`eval` and :func:`exec` accept :class:`!frozendict` for *globals*, and
 :func:`type` and :meth:`str.maketrans` accept :class:`!frozendict` for *dict*.
@@ -604,6 +605,9 @@ Other language changes
   respectively.
   (Contributed by Sergey B Kirpichev in :gh:`146151`.)
 
+* Allow the *count* argument of :meth:`bytes.replace` to be a keyword.
+  (Contributed by Stan Ulbrych in :gh:`147856`.)
+
 
 New modules
 ===========
@@ -642,6 +646,10 @@ array
   formatting characters ``'F'`` and ``'D'`` respectively.
   (Contributed by Sergey B Kirpichev in :gh:`146151`.)
 
+* Support half-floats (16-bit IEEE 754 binary interchange format): formatting
+  character ``'e'``.
+  (Contributed by Sergey B Kirpichev in :gh:`146238`.)
+
 
 base64
 ------

Include/internal/pycore_long.h

@@ -240,6 +240,20 @@ _PyLong_IsPositive(const PyLongObject *op)
     return (op->long_value.lv_tag & SIGN_MASK) == 0;
 }
 
+/* Return true if the argument is a small int */
+static inline bool
+_PyLong_IsSmallInt(const PyLongObject *op)
+{
+    assert(PyLong_Check(op));
+    bool is_small_int = (op->long_value.lv_tag & IMMORTALITY_BIT_MASK) != 0;
+    assert(PyLong_CheckExact(op) || (!is_small_int));
+    assert(_Py_IsImmortal(op) || (!is_small_int));
+    assert((_PyLong_IsCompact(op)
+            && _PY_IS_SMALL_INT(_PyLong_CompactValue(op)))
+           || (!is_small_int));
+    return is_small_int;
+}
+
 static inline Py_ssize_t
 _PyLong_DigitCount(const PyLongObject *op)
 {
@@ -301,7 +315,9 @@ _PyLong_SetDigitCount(PyLongObject *op, Py_ssize_t size)
 #define NON_SIZE_MASK ~(uintptr_t)((1 << NON_SIZE_BITS) - 1)
 
 static inline void
-_PyLong_FlipSign(PyLongObject *op) {
+_PyLong_FlipSign(PyLongObject *op)
+{
+    assert(!_PyLong_IsSmallInt(op));
     unsigned int flipped_sign = 2 - (op->long_value.lv_tag & SIGN_MASK);
     op->long_value.lv_tag &= NON_SIZE_MASK;
     op->long_value.lv_tag |= flipped_sign;

InternalDocs/jit.md

@@ -11,24 +11,54 @@ and this enables optimizations that span multiple instructions.
 Historically, the adaptive interpreter was referred to as `tier 1` and
 the JIT as `tier 2`. You will see remnants of this in the code.
 
-## The Optimizer and Executors
+## The Trace Recorder and Executors
 
-The program begins running on the adaptive interpreter, until a `JUMP_BACKWARD`
-instruction determines that it is "hot" because the counter in its
+There are two interpreters in this section:
+  1. Adaptive interpreter (the default behavior)
+  2. Trace recording interpreter (enabled on JIT builds)
+
+The program begins running on the adaptive interpreter, until a `JUMP_BACKWARD` or
+`RESUME` instruction determines that it is "hot" because the counter in its
 [inline cache](interpreter.md#inline-cache-entries) indicates that it
 executed more than some threshold number of times (see
 [`backoff_counter_triggers`](../Include/internal/pycore_backoff.h)).
-It then calls the function `_PyOptimizer_Optimize()` in
+It then calls the function `_PyJit_TryInitializeTracing` in
 [`Python/optimizer.c`](../Python/optimizer.c), passing it the current
-[frame](frames.md) and instruction pointer. `_PyOptimizer_Optimize()`
-constructs an object of type
-[`_PyExecutorObject`](../Include/internal/pycore_optimizer.h) which implements
-an optimized version of the instruction trace beginning at this jump.
-
-The optimizer determines where the trace ends, and the executor is set up
+[frame](frames.md), instruction pointer and state.
+The interpreter then switches into "tracing mode" via the macro
+`ENTER_TRACING()`. On platforms that support computed goto and tail-calling
+interpreters, the dispatch table is swapped out, while other platforms that do
+not support either use a single flag in the opcode.
+Execution between the normal interpreter and tracing interpreter are
+interleaved via this dispatch mechanism. This means that while logically
+there are two interpreters, the implementation appears to be a single
+interpreter. 
+
+During tracing mode, after each interpreter instruction's `DISPATCH()`,
+the interpreter jumps to the `TRACE_RECORD` instruction. This instruction
+records the previous instruction executed and also any live values of the next
+operation it may require. It then translates the previous instruction to
+a sequence of micro-ops using `_PyJit_translate_single_bytecode_to_trace`.
+To ensure that the adaptive interpreter instructions
+and cache entries are up-to-date, the trace recording interpreter always resets
+the adaptive counters of adaptive instructions it sees.
+This forces a re-specialization of any new instruction should an instruction
+deoptimize. Thus, feeding the trace recorder up-to-date information.
+Finally, the `TRACE_RECORD` instruction decides when to stop tracing 
+using various heuristics.
+
+Once trace recording concludes, `LEAVE_TRACING()` swaps out the dispatch
+table/the opcode flag set earlier by `ENTER_TRACING()` is unset.
+`stop_tracing_and_jit()` then calls `_PyOptimizer_Optimize()` which optimizes
+the trace and constructs an
+[`_PyExecutorObject`](../Include/internal/pycore_optimizer.h).
+
+JIT execution is set up
 to either return to the adaptive interpreter and resume execution, or
 transfer control to another executor (see `_PyExitData` in
-Include/internal/pycore_optimizer.h).
+Include/internal/pycore_optimizer.h). When resuming to the adaptive interpreter,
+a "side exit", generated by an `EXIT_IF` may trigger recording of another trace.
+While a "deopt", generated by a `DEOPT_IF`, does not trigger recording.
 
 The executor is stored on the [`code object`](code_objects.md) of the frame,
 in the `co_executors` field which is an array of executors. The start
@@ -40,12 +70,7 @@ executor in `co_executors`.
 
 The micro-op (abbreviated `uop` to approximate `μop`) optimizer is defined in
 [`Python/optimizer.c`](../Python/optimizer.c) as `_PyOptimizer_Optimize`.
-It translates an instruction trace into a sequence of micro-ops by replacing
-each bytecode by an equivalent sequence of micro-ops (see
-`_PyOpcode_macro_expansion` in
-[pycore_opcode_metadata.h](../Include/internal/pycore_opcode_metadata.h)
-which is generated from [`Python/bytecodes.c`](../Python/bytecodes.c)).
-The micro-op sequence is then optimized by
+It takes a micro-op sequence from the trace recorder and optimizes with
 `_Py_uop_analyze_and_optimize` in
 [`Python/optimizer_analysis.c`](../Python/optimizer_analysis.c)
 and an instance of `_PyUOpExecutor_Type` is created to contain it.

Lib/collections/__init__.py

@@ -328,14 +328,14 @@ def __ior__(self, other):
         return self
 
     def __or__(self, other):
-        if not isinstance(other, dict):
+        if not isinstance(other, (dict, frozendict)):
             return NotImplemented
         new = self.__class__(self)
         new.update(other)
         return new
 
     def __ror__(self, other):
-        if not isinstance(other, dict):
+        if not isinstance(other, (dict, frozendict)):
             return NotImplemented
         new = self.__class__(other)
         new.update(self)
@@ -1217,14 +1217,14 @@ def __repr__(self):
     def __or__(self, other):
         if isinstance(other, UserDict):
             return self.__class__(self.data | other.data)
-        if isinstance(other, dict):
+        if isinstance(other, (dict, frozendict)):
             return self.__class__(self.data | other)
         return NotImplemented
 
     def __ror__(self, other):
         if isinstance(other, UserDict):
             return self.__class__(other.data | self.data)
-        if isinstance(other, dict):
+        if isinstance(other, (dict, frozendict)):
             return self.__class__(other | self.data)
         return NotImplemented