Skip to content

gh-153569: return token spans and unify decoded source storage - #156482

Merged
pablogsal merged 6 commits into
python:mainfrom
pablogsal:gh-153569-tokenizer-offset-state
Sep 6, 2026
Merged

gh-153569: return token spans and unify decoded source storage#156482
pablogsal merged 6 commits into
python:mainfrom
pablogsal:gh-153569-tokenizer-offset-state

Remove stale tokenizer buffer ownership comment

6117ae3
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
DO-NOT-MERGE
succeeded Sep 5, 2026 in 5s