Skip to content

[INTERPRETER] Make int1 add/sub wrap around like the GPU - #10923

Merged
Jokeren merged 1 commit into
triton-lang:mainfrom
ArsalanShakil:fix/interp-int1-add-wraparound
Jul 18, 2026
Merged

[INTERPRETER] Make int1 add/sub wrap around like the GPU#10923
Jokeren merged 1 commit into
triton-lang:mainfrom
ArsalanShakil:fix/interp-int1-add-wraparound

Conversation

@ArsalanShakil

Copy link
Copy Markdown
Contributor

Summary

Fixes #10919.

int1 + int1 produced different results in the interpreter and on the GPU:

  • GPU: int1 is a 1-bit integer, so arith.addi wraps mod 2 and True + True == 0.
  • Interpreter: binary_op ran numpy bool arithmetic, which is logical/saturating (True + True == True == 1), so it disagreed with the GPU.

As noted by @peterbell10 in the issue, 0 is the intended result (int1 should wrap like any other integer type), so the interpreter is the side to change; the GPU path is unchanged.

This also fixes a related interpreter crash: int1 - int1 previously raised TypeError: numpy boolean subtract ... is not supported, since numpy rejects bool subtraction. It now wraps correctly (matching the GPU).

Change

In InterpreterBuilder.binary_op, when the operand type is int1, compute the op in a wider integer domain (int8) and truncate back to one bit (& 1). This matches the GPU's 1-bit wraparound. Ops whose int1 results are already in {0, 1} (mul, and, or, xor, min, max, comparisons) are unaffected.

Test

test_int1_bin_op_wraparound in test_core.py runs int1 +/- int1 and asserts mod-2 wraparound. Marked @pytest.mark.interpreter so it runs on both the GPU and interpreter paths, pinning them together.

@ArsalanShakil
ArsalanShakil requested a review from ptillet as a code owner July 17, 2026 12:21
@peterbell10
peterbell10 enabled auto-merge (squash) July 17, 2026 12:42
auto-merge was automatically disabled July 17, 2026 13:11

Head branch was pushed to by a user without write access

@ArsalanShakil
ArsalanShakil force-pushed the fix/interp-int1-add-wraparound branch from 8eee3e8 to ee62978 Compare July 17, 2026 13:11
@ArsalanShakil

Copy link
Copy Markdown
Contributor Author

Update: fixed the failing interpreter test

The first push failed the integration-tests-nvidia interpret step on both parametrizations of the new test:

  • test_int1_bin_op_wraparound[+]: interpreter returned [0, 1, 1, 1] instead of the expected [0, 1, 1, 0]
  • test_int1_bin_op_wraparound[-]: interpreter raised TypeError: numpy boolean subtract ... is not supported

Why it was failing. My first version of the fix gated the new behavior on the Triton dtype: if lhs.dtype.scalar == tl.int1. In the interpreter that condition never held for these operands, so the branch was skipped and numpy ran its native bool arithmetic (logical or for +, and an outright error for -). The GPU test passed the whole time, which is what pointed at an interpreter-specific dtype mismatch rather than a problem with the fix itself.

The fix. Key the branch off the actual numpy data instead, which is the ground truth for what the op executes on:

if lhs.data.dtype == np.bool_ and rhs.data.dtype == np.bool_:
    output = (op(lhs.data.astype(np.int8), rhs.data.astype(np.int8)) & 1).astype(np.bool_)
    return TensorHandle(output, tl_dtype)

Computing in int8 and masking back to one bit reproduces the GPU's 1-bit wraparound. Every other op that routes through binary_op with bool operands (==, <, &, |, ^, *, min, max) already produces values in {0, 1}, so the & 1 is a no-op for them and their results are unchanged. The sanitize_overflow path is unaffected because it casts operands to int64 before reaching binary_op, so their data is never bool.

The test is marked @pytest.mark.interpreter, so it now pins the interpreter and GPU paths together for int1 add/sub.

Comment thread python/triton/runtime/interpreter.py
`bool + bool` in the interpreter used numpy's logical/saturating bool
arithmetic (`True + True == True`), and `bool - bool` raised a TypeError
outright. The GPU backend treats int1 as a 1-bit integer that wraps
around, so `True + True == 0`. Compute int1 binary ops in a wider integer
domain and truncate back to one bit so the interpreter matches the GPU.

Fixes triton-lang#10919
@ArsalanShakil
ArsalanShakil force-pushed the fix/interp-int1-add-wraparound branch from 70892ab to ae54685 Compare July 17, 2026 13:33
@Jokeren
Jokeren merged commit e331a7c into triton-lang:main Jul 18, 2026
10 checks passed
Hughshine added a commit to Hughshine/triton that referenced this pull request Jul 18, 2026
The interpreter stores bf16 as a uint16 bit pattern, but binary_op, unary_op,
fma and dot ran the numpy op directly on that storage, so bf16 arithmetic and
comparisons computed on the bit pattern instead of the value, diverging from
the GPU. Convert to float32 before the op (as the GPU does) and round back to
bf16 after. Adds interpreter tests for bf16 binary_op, comparison, neg and fma.

(int1, the other half of triton-lang#10919, was fixed separately in triton-lang#10923.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bool + bool on a tensor gives an inconsistent result across the GPU backend, the interpreter, and PyTorch

3 participants