Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Performance Optimization] Shard level search - optimize for special cases #1236

Open
martin-gaievski opened this issue Mar 19, 2025 · 0 comments

Comments

@martin-gaievski
Copy link
Member

For hybrid query execution there are some special cases where we can improve performance. While it's not impacting every single query, in niche scenario impact can be big.

Some scenarios I can think of:

  • multiple sub-queries that after being rewritten are same. We can skip execution of the second query and just copy results from the first execution. This is equivalent of retrieving results from cache instead of executing same query again.
  • keeping track of the min score threshold, if we have size docs don't take any docs with score lower then the threshold. This is an optimization that will allow to skip adding score and kicking out the element from the min heap of collected scores, which is case of broad queries can be huge (e.g. query with 10M potentially matching docs executing in index with 100 shards -> saving min heap 100.000 operations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants