Commit ade8dbe
perf(array): append repeated and viewed lists in bulk, not a value at a time
Four callers walked a builder one list at a time where a whole run of them was
available up front.
`Sparse` canonicalization filled the gaps between patches by appending the fill
value once per row, and appended the patches themselves one list at a time. A
gap is a run of one value, so it goes in as a single `ConstantArray`:
canonicalizing a constant list array points every view at one copy of the
value, so a gap now costs the fill value's elements once however many rows it
covers. Appending a 10,000-row gap of a three-element fill produced 30,000
elements; it now produces 3. Patches landing on consecutive rows go in as one
slice of the patch array, so the builder sees an append per gap rather than one
per patch.
The patch values are flattened once up front instead. That is what lets a run
be sliced out of them, and it also means a null patch carries a zero-size view
rather than the elements the old code skipped, so the appended run holds
exactly the elements the patches reference.
`ListBuilder::append_listview_array` sliced the elements array and appended the
slice once per list, which is what its `ListViewBuilder` twin stopped doing.
`ListArray` offsets can only describe contiguous, in-order lists, so flatten
the incoming views to that layout - a no-op when they are laid out that way
already - and then append the referenced elements in one go, walking only the
metadata to rebase the offsets.
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 08c336f commit ade8dbe
2 files changed
Lines changed: 290 additions & 142 deletions
0 commit comments