|
1 | 1 | ---
|
2 | 2 | layout: post
|
3 |
| -title: OpenSearch Project Roadmap 2024&endash;2025 |
| 3 | +title: OpenSearch Project Roadmap 2024–2025 |
4 | 4 | authors:
|
5 | 5 | - pallp
|
6 | 6 | - bburkholder
|
|
9 | 9 | date: 2024-09-12
|
10 | 10 | categories:
|
11 | 11 | - community-updates
|
12 |
| -excerpt: OpenSearch is a rapidly growing open-source product suite comprising a search engine, an ingestion system, language clients, and a user interface for analytics. OpenSearch contributors and maintainers are innovating in all these areas at a fast pace. To steer the project's development effectively, we have revamped the project roadmap to provide better transparency into both short- and long-term enhancements. In this blog post, we are excited to share the new theme-based, community-driven OpenSearch Project Roadmap for 2024–2025. |
| 12 | +excerpt: OpenSearch is a rapidly growing open-source product suite comprising a search engine, an ingestion system, language clients, and a user interface for analytics. OpenSearch contributors and maintainers are innovating in all these areas at a fast pace. To steer the project's development effectively, we have revamped the project roadmap to provide better transparency into both short- and long-term enhancements. In this blog post, we are excited to share the new theme-based, community-driven OpenSearch Project Roadmap for 2024–2025. |
| 13 | +featured_blog_post: true |
13 | 14 | additional_author_info: We sincerely appreciate the contributions to this blog from Yupeng Fu (Uber), Karthikeyan Ramasamy (Uber), Shuyi Z. (Uber), Shubham Gupta (Uber), George Luong (Slack/Salesforce), Austin Lee (Aryn), Andriy Redko (Aiven), Yuncheng Lu (ByteDance), Tianjin Li (Cohere), and AWS contributors (Anandhi Bumstead, Carl Meadows, Michael Froh, Shweta Thareja, Saurabh Singh, Kunal Khatua, Andrew Ross, Sean Zheng, Anirudha Jadhav, Gopala Krishna Ambareesh, Yan Zeng, Huan Jiang, Vamshi Vijay Nakkirtha, Craig Perkins, Peter Zhu, Joshua Bright, David Venable, Bukhtawar Khan, Prabhakar Sithanandam, Ranjith Ramachandra, Sorabh Hamirwasia, Charlie Yang, Rohit Wali, Peng Huo, and Fanit Kolchina).
|
14 | 15 | meta_keywords: OpenSearch roadmap, open source search, vector database, generative AI, observability, log analytics, security analytics, cloud native architecture, OpenSearch performance optimization
|
15 | 16 | meta_description: Explore the OpenSearch Project's roadmap for 2024-2025 and learn how the community is driving the project forward to make it the preferred open-source solution for search, analytics, and generative AI applications.
|
@@ -130,7 +131,7 @@ The OpenSearch Project continues to enhance its observability and security analy
|
130 | 131 |
|
131 | 132 | **Next-generation snapshots for remote-backed clusters**: [Snapshots v2](https://github.com/opensearch-project/OpenSearch/issues/15057) aims to enhance the scalability of snapshots for remote-backed clusters and reduce dependence on per-shard state updates in the cluster manager. The new snapshots rely on a timestamp-based pinning strategy, where instead of resolving shard-level files at snapshot time, the timestamp for the snapshot is pinned and the resolution is deferred until restore time. This approach makes the snapshot process much faster, allowing snapshot operations to finish within a couple of minutes, even for larger clusters, while significantly reducing the computational load associated with data backup. Timestamp pinning serves as the fundamental building block for future features, such as [Point-In-Time-Restore (PITR)](https://github.com/opensearch-project/OpenSearch/issues/1147).
|
132 | 133 |
|
133 |
| -**Scaling admin APIs**: For large cluster configurations, cluster manager nodes become scaling bottlenecks as multiple admin APIs obtain the cluster state from the active cluster manager node, even if the latest state is present locally or present in a remote store. With the ongoing optimizations, the coordinator node can [serve the admin APIs without relaying the request](https://github.com/opensearch-project/OpenSearch/pull/12252/) to the cluster manager node in most cases. Also, for APIs like CAT Shards and CAT Snapshotos, the response size increases as the cluster expands to 100K shards or more. We plan to introduce [pagination](https://github.com/opensearch-project/OpenSearch/issues/14258) and [cancellation](https://github.com/opensearch-project/OpenSearch/issues/13908) for these APIs to ensure that they continue to operate efficiently regardless of the metadata size. We are implementing multiple optimizations to the Stats and Cluster APIs that will eliminate redundant processing and perform [pre-aggregation](https://github.com/opensearch-project/OpenSearch/pull/14426) on the data node before responding to the coordinator node receiving the user request. |
| 134 | +**Scaling admin APIs**: For large cluster configurations, cluster manager nodes become scaling bottlenecks as multiple admin APIs obtain the cluster state from the active cluster manager node, even if the latest state is present locally or present in a remote store. With the ongoing optimizations, the coordinator node can [serve the admin APIs without relaying the request](https://github.com/opensearch-project/OpenSearch/pull/12252/) to the cluster manager node in most cases. Also, for APIs like CAT Shards and CAT Snapshots, the response size increases as the cluster expands to 100K shards or more. We plan to introduce [pagination](https://github.com/opensearch-project/OpenSearch/issues/14258) and [cancellation](https://github.com/opensearch-project/OpenSearch/issues/13908) for these APIs to ensure that they continue to operate efficiently regardless of the metadata size. We are implementing multiple optimizations to the Stats and Cluster APIs that will eliminate redundant processing and perform [pre-aggregation](https://github.com/opensearch-project/OpenSearch/pull/14426) on the data node before responding to the coordinator node receiving the user request. |
134 | 135 |
|
135 | 136 | ### Roadmap Theme 6: Stability, Availability, and Resiliency
|
136 | 137 |
|
|
0 commit comments