Skip to content

Add bulk off-heap scoring for vector formats #15155

@benwtrent

Description

@benwtrent

Description

We found some pretty awesome latency improvements with float32 vectors bulk scoring off-heap. It makes me think we can get nice improvements for byte vectors and all the quantized formats.

See: #14980

&: #14978

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions