You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Diving through some code to understand performance regressions in descending sort on timestamp ahead of OpenSearch 3.0, I finally understood the change added in #7967.
Essentially, we apply a sort to the IndexWriterConfig to say that any IndexReader opened from the IndexWriter should have its segments sorted by descending max timestamp. Unfortunately, that only happens in InternalEngine.
We have other engine types, ReadOnlyEngine and NRTReplicationEngine, which don't even have an IndexWriter (since neither is capable of writing Lucene indexes). The sort ordering optimization is not applied to them.
We should push the leafSorter (via the EngineConfig) into these other engines, so that they can pass them to the their DirectoryReader.open calls.
Related component
Search:Performance
To Reproduce
Go to '...'
Click on '....'
Scroll down to '....'
See error
Expected behavior
See above.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
OS: [e.g. iOS]
Version [e.g. 22]
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
In the first two cases, the final null parameter should be the leafSorter. In the third case, we need to either use StandardDirectoryReader or we should change it to:
Describe the bug
Diving through some code to understand performance regressions in descending sort on timestamp ahead of OpenSearch 3.0, I finally understood the change added in #7967.
Essentially, we apply a sort to the
IndexWriterConfig
to say that anyIndexReader
opened from theIndexWriter
should have its segments sorted by descending max timestamp. Unfortunately, that only happens inInternalEngine
.We have other engine types,
ReadOnlyEngine
andNRTReplicationEngine
, which don't even have anIndexWriter
(since neither is capable of writing Lucene indexes). The sort ordering optimization is not applied to them.We should push the
leafSorter
(via theEngineConfig
) into these other engines, so that they can pass them to the theirDirectoryReader.open
calls.Related component
Search:Performance
To Reproduce
Expected behavior
See above.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: