Analytics Engine & Data Pipeline Integration
This release introduces a comprehensive analytics reporting engine, shifting the application from on-demand summaries to a high-performance, scheduled data pipeline. It enables the aggregation, transformation, and storage of critical metrics from both internal databases and external providers like Google Analytics 4 and Mixpanel, ensuring immediate data availability for dashboards.
📊 Core Analytics Infrastructure
A new dedicated worker and storage strategy have been implemented to handle heavy data processing asynchronously. This ensures the primary API remains responsive while complex metrics are calculated in the background.
- ⚙️ Standalone Sync Worker: Introduced
bin/analytics_sync_worker.dart, a new executable that runs periodic ETL (Extract, Transform, Load) tasks to process raw data and keep dashboard metrics current. #109 - 💾 Optimized Storage Strategy: Implemented dedicated MongoDB collections (e.g.,
kpi_card_data,chart_card_data) to store pre-aggregated data, significantly reducing load times for analytical queries. #109 - 🔎 Advanced Query Builder: Developed a specialized
AnalyticsQueryBuilderto generate optimized database aggregation pipelines, supported by new analytics-specific database indexes. #109
🔌 Provider Integration & Security
The architecture is designed to be provider-agnostic, allowing seamless switching or combination of analytics sources while ensuring access to sensitive data is strictly controlled.
- 🌐 Provider-Agnostic Design: Established a flexible
AnalyticsReportingClientinterface that supports configuration-based integration with external providers like Google Analytics 4 and Mixpanel. #109 - 🛡️ Granular Access Control: Introduced a new
analytics.readpermission, allowing administrators to precisely control which users have access to sensitive dashboard metrics. #109