Skip to content

perf: column-native recall output (AddItemColumns) — bypass map-to-column transposition #159

Description

@Liam0205

Context

Additions remain row-store's inherent win even after PR #155's column-major two-pass rewrite: row store takes ownership of the caller's item maps zero-copy (123μs), while column store must transpose every map into columns (340μs, down from 600μs). The remaining gap is the unavoidable map→column transposition.

Proposal

Let recall-type operators emit columns directly:

  • out.AddItemColumns(cols map[string][]any, n int) (names TBD per engine) — column store adopts the slices zero-copy; row store transposes (inverting today's asymmetry, which is fair: whoever mismatches the storage pays)
  • Existing AddItem(map) stays; the column form is opt-in for sources that naturally produce columnar data

Why low priority

  • Upstream sources (JSON requests, Redis, remote pineapple) are naturally row-shaped; only in-memory sources (recall_static, resource-backed recalls, bench stubs) can produce columns without paying the transposition themselves
  • Changes the Recall operator output API surface across three engines
  • Real benefit depends on workload: only matters when additions dominate and storage_mode=column, which contradicts the current usage guidance (recall-heavy → row)

Park until profiling shows recall transposition as a production hotspot on a column-mode deployment.

References

  • llmdoc/memory/reflections/column-vs-row-parity-investigation.md (root cause 3, additions)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P4Priority 4 — parked / roadmap end; do not start without explicit triggerenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions