Skip to content

Commit ade8dbe

Browse files
robert3005claude
andcommitted
perf(array): append repeated and viewed lists in bulk, not a value at a time
Four callers walked a builder one list at a time where a whole run of them was available up front. `Sparse` canonicalization filled the gaps between patches by appending the fill value once per row, and appended the patches themselves one list at a time. A gap is a run of one value, so it goes in as a single `ConstantArray`: canonicalizing a constant list array points every view at one copy of the value, so a gap now costs the fill value's elements once however many rows it covers. Appending a 10,000-row gap of a three-element fill produced 30,000 elements; it now produces 3. Patches landing on consecutive rows go in as one slice of the patch array, so the builder sees an append per gap rather than one per patch. The patch values are flattened once up front instead. That is what lets a run be sliced out of them, and it also means a null patch carries a zero-size view rather than the elements the old code skipped, so the appended run holds exactly the elements the patches reference. `ListBuilder::append_listview_array` sliced the elements array and appended the slice once per list, which is what its `ListViewBuilder` twin stopped doing. `ListArray` offsets can only describe contiguous, in-order lists, so flatten the incoming views to that layout - a no-op when they are laid out that way already - and then append the referenced elements in one go, walking only the metadata to rebase the offsets. Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: Robert Kruszewski <robert@spiraldb.com> Signed-off-by: Robert Kruszewski <github@robertk.io> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 08c336f commit ade8dbe

2 files changed

Lines changed: 290 additions & 142 deletions

File tree

0 commit comments

Comments
 (0)