[RFC] Prometheus as a first class datasource

[d-buckner]

Summary

This RFC proposes the integration of Prometheus as a first-class datasource in OpenSearch Dashboards. This integration would enable users to explore and create dashboards using Prometheus metrics directly within the OpenSearch Dashboards interface, expanding its capabilities beyond log analytics.

Background

The current core OpenSearch Dashboards experience is primarily focused on log analytics. While this is valuable, many organizations use Prometheus for metrics and monitoring. By integrating Prometheus as a first-class datasource, we can provide a more comprehensive observability solution within OpenSearch Dashboards.

Prometheus is a popular open-source monitoring and alerting toolkit, widely used for collecting and querying metrics from various systems and applications. It has become a standard in cloud-native environments and is a graduated project of the Cloud Native Computing Foundation (CNCF).

Motivation

1. Unified Observability Platform: Create a single platform that addresses the challenge of tool sprawl in modern monitoring and observability stacks.

2. Comprehensive System View: Provide a holistic view of system health and performance by integrating Prometheus metrics alongside existing log analytics capabilities.

3. Reduced Context Switching: Minimize the need for users to switch between multiple tools for observability tasks, improving workflow efficiency.

4. Seamless User Experience: Offer an integrated experience for both metrics and log analysis within the familiar OpenSearch Dashboards interface.

5. Enhanced Analysis Capabilities: Enable users to correlate metrics and logs more effectively, leading to improved problem identification and resolution.

Proposal

1. Data Source Management

   • Provide a user interface for configuring Prometheus data sources

2. Discover

   • Implement PromQL support in the query bar
   • Provide autocomplete suggestions for PromQL queries
   • Visualize time series in a line chart
   • Support discovery of metrics and labels in the left data pane

3. Visualizations

   • Support line and bar charts for time series metric data

4. Dashboards

   • Allow creation of dashboards with Prometheus data visualizations
   • Enable mixed dashboards with both Prometheus and other data sources
   • Support time range controls for Prometheus-based panels

Feedback Requested

I'm seeking input from the community on the following aspects of the Prometheus integration:

• Priority Features: Which Prometheus features are most critical for your use cases?
• Use Cases: What specific use cases or workflows would you like to see supported in this integration?
• Visualization Preferences: What types of visualizations are most important for your Prometheus data? Are there any specific chart types or dashboarding features you'd like to see?
• Query Experience: What enhancements to the PromQL query experience would be most valuable? (e.g., query builders, advanced autocomplete features)
• Ecosystem Integrations: Are there specific tools or platforms in the Prometheus ecosystem that you'd like to see integrated or supported?
• User Experience: How do you envision the ideal user experience for working with Prometheus data alongside other data sources in OpenSearch Dashboards?