Description

Currently at a segment level when vector search is happening there are multiple cases when we move from ANN search to exact search. Cases like when filter threshold hit or if there are no graphs present on the segment due to graph creation threshold is not met for a segment. In all these cases we do exact search but while doing exact search we serially iterate over valid docIds, read the vectors and then compute the score.

Possible Optimizations

One possible optimization we can do is to actually parallelize this, which can give a reduce the latency.

Some ideas can be:

1. We can actually gather all the docIds in an array and then let multiple threads consume those docIds with their VectorValues to read the vector and compute the scores in parallel.
2. We can also first read the vectors and docsIds in heap and then compute scores in parallel.
3. We can have a VectorValuesIterator that is thread safe and each thread can read from single iterator till the iterator reaches NO_MORE_DOCS. Thanks to @heemin32 for suggesting this.

I think 1 and 3, should be better over 2 as in that case we don't need to read all vectors in heap and put more stress on heap. Looking for feedbacks here.

But during implementation we can see which one we should choose.

Similar thing can be done for the script score based exact search but that will need some more thinking. Will try to update add that too.