Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use 'Lucene912PostingsFormat' when storage efficiency isn't critical #119051

Closed
Tracked by #115021
martijnvg opened this issue Dec 19, 2024 · 1 comment · Fixed by #126080
Closed
Tracked by #115021

Use 'Lucene912PostingsFormat' when storage efficiency isn't critical #119051

martijnvg opened this issue Dec 19, 2024 · 1 comment · Fixed by #126080

Comments

@martijnvg
Copy link
Member

martijnvg commented Dec 19, 2024

Use Lucene912PostingsFormat on indexes whose storage efficiency is not critical. Likely index.codec is a good heuristic. Use cases that require batter space efficiency will have it set to best_compression. So if it is set to default is a good heuristic to use the stack Lucene912PostingsFormat.

This will ensure search use cases will benefit from the query time performance improvements that have been made overtime in Lucene's default postings format. For uses cases that rely on better storage efficiency we should keep using the ES812PostingsFormat and in a follow up develop new posting format that uses PFOR and take advantage of query time benefits (like #115021 describes). The latter a higher hanging fruit, then just using the stock Lucene912PostingsFormat that comes with Lucene 10.

@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Dec 19, 2024
@martijnvg martijnvg changed the title Use Lucene912PostingsFormat on indexes whose storage efficiency is not critical (heuristic to be defined, e.g. when index.codec is default and source.mode is not synthetic?). Use Lucene912PostingsFormat on indexes whose storage efficiency is not critical Dec 19, 2024
@martijnvg martijnvg changed the title Use Lucene912PostingsFormat on indexes whose storage efficiency is not critical Use 'Lucene912PostingsFormat' when storage efficiency is not critical Dec 19, 2024
@martijnvg martijnvg changed the title Use 'Lucene912PostingsFormat' when storage efficiency is not critical Use 'Lucene912PostingsFormat' when storage efficiency isn't critical Dec 19, 2024
@martijnvg martijnvg added :StorageEngine/Codec >enhancement and removed needs:triage Requires assignment of a team area label labels Dec 19, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants