Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Timestamp sort optimizations don't work on searchable snapshot or segrep replicas #17579

Open
msfroh opened this issue Mar 12, 2025 · 1 comment
Labels
bug Something isn't working Search:Performance

Comments

@msfroh
Copy link
Collaborator

msfroh commented Mar 12, 2025

Describe the bug

Diving through some code to understand performance regressions in descending sort on timestamp ahead of OpenSearch 3.0, I finally understood the change added in #7967.

Essentially, we apply a sort to the IndexWriterConfig to say that any IndexReader opened from the IndexWriter should have its segments sorted by descending max timestamp. Unfortunately, that only happens in InternalEngine.

We have other engine types, ReadOnlyEngine and NRTReplicationEngine, which don't even have an IndexWriter (since neither is capable of writing Lucene indexes). The sort ordering optimization is not applied to them.

We should push the leafSorter (via the EngineConfig) into these other engines, so that they can pass them to the their DirectoryReader.open calls.

Related component

Search:Performance

To Reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

See above.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@msfroh
Copy link
Collaborator Author

msfroh commented Mar 12, 2025

Relevant lines where the IndexReaders get opened for the other engine types:

DirectoryReader innerReader = StandardDirectoryReader.open(referenceToRefresh.directory(), currentInfos, subs, null);

reader = DirectoryReader.open(commit, this.minimumSupportedVersion.luceneVersion.major, null);

In the first two cases, the final null parameter should be the leafSorter. In the third case, we need to either use StandardDirectoryReader or we should change it to:

reader = DirectoryReader.open(commit, Version.CURRENT.minimumCompatibilityVersion().luceneVersion.major, config().getLeafSorter());

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Search:Performance
Projects
Status: 🆕 New
Development

No branches or pull requests

2 participants