|
| 1 | +.. _arch-center-is-rocket-launch: |
| 2 | + |
| 3 | +===================================== |
| 4 | +App-Driven Intelligence with IoT data |
| 5 | +===================================== |
| 6 | + |
| 7 | +.. facet:: |
| 8 | + :name: genre |
| 9 | + :values: tutorial |
| 10 | + |
| 11 | +.. meta:: |
| 12 | + :keywords: Internet of Things, IoT, Manufacturing, Analytics |
| 13 | + :description: Create real-time analysis on IoT data on the MongoDB developer data platform |
| 14 | + |
| 15 | +.. contents:: On this page |
| 16 | + :local: |
| 17 | + :backlinks: none |
| 18 | + :depth: 1 |
| 19 | + :class: singlecol |
| 20 | + |
| 21 | +Monitoring a rocket launch using MongoDB Atlas and real-time IoT data. |
| 22 | + |
| 23 | +**Use cases:** `Analytics <https://www.mongodb.com/solutions/use-cases/analytics>`__, |
| 24 | +`IoT <https://www.mongodb.com/solutions/use-cases/internet-of-things>`__ |
| 25 | + |
| 26 | +**Industries:** `Manufacturing and Mobility <https://www.mongodb.com/solutions/industries/manufacturing>`__, |
| 27 | +`Retail <https://www.mongodb.com/solutions/industries/retail>`__ |
| 28 | + |
| 29 | +**Products:** `MongoDB Atlas <http://mongodb.com/atlas>`__, |
| 30 | +`MongoDB Aggregation Pipeline <https://www.mongodb.com/docs/v7.0/core/aggregation-pipeline/>`__, |
| 31 | +`MongoDB Time Series <https://www.mongodb.com/products/capabilities/time-series>`__, |
| 32 | +`Atlas Charts <https://www.mongodb.com/products/platform/atlas-charts>`__, |
| 33 | +`Atlas Database <https://www.mongodb.com/products/platform/atlas-database>`__, |
| 34 | +`Atlas Data Federation <https://www.mongodb.com/products/platform/atlas-data-federation>`__, |
| 35 | +`Atlas Search <https://www.mongodb.com/products/platform/atlas-search>`__, |
| 36 | +`Atlas SQL Interface <https://www.mongodb.com/products/platform/atlas-sql-interface>`__, |
| 37 | +`Atlas Triggers <https://www.mongodb.com/docs/atlas/atlas-ui/triggers/>`__ |
| 38 | + |
| 39 | +**Partners:** `Amazon S3 <https://www.mongodb.com/products/platform/atlas-cloud-providers/aws>`__, |
| 40 | +`Tableau <https://www.mongodb.com/try/download/tableau-connector>`__ |
| 41 | + |
| 42 | +Solution Overview |
| 43 | +----------------- |
| 44 | + |
| 45 | +From manufacturing and transportation to logistics and supply chain, the |
| 46 | +ability to perform real-time analysis on live IoT data during operation, |
| 47 | +and combine it with other data sources post-operation, is valuable |
| 48 | +across many industries. This demo shows how to leverage the MongoDB |
| 49 | +modern, multi-cloud database platform to go from zero to lift-off in a |
| 50 | +rocket launch example. |
| 51 | + |
| 52 | +A typical rocket launch spans an eight-hour period from the time the |
| 53 | +initial countdown begins until the rocket payload is in orbit. During |
| 54 | +this window, approximately one million metrics are generated per second |
| 55 | +by sensors capturing the rocket’s performance. While the metrics make up |
| 56 | +the bulk of the data in this scenario, there are two other sources of |
| 57 | +data: notes and weather data. |
| 58 | + |
| 59 | +- Notes are created by both rocket engineers and an automated system. The |
| 60 | + rocket engineers create notes when they want to mark a time period or |
| 61 | + situation that they want to remember to revisit after the launch. An |
| 62 | + automated system is continuously watching the metrics as they stream in, |
| 63 | + creating notes whenever parameters reach thresholds that are out of |
| 64 | + bounds. |
| 65 | + |
| 66 | +- Weather data is retrieved from a third party, and stored in an Amazon S3 |
| 67 | + bucket, and analyzed in combination with the launch data post-launch. |
| 68 | + |
| 69 | +This solution shows how the various components of MongoDB's modern, |
| 70 | +multi-cloud database platform can be used together to support |
| 71 | +application-driven analytics on IoT data by highlighting a rocket |
| 72 | +launch, with analysis happening during the launch and after. It uses an |
| 73 | +open source data set from a Blue Origin rocket launch along with |
| 74 | +fictional data. |
| 75 | + |
| 76 | +Other Applicable Industries and Use Cases |
| 77 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 78 | + |
| 79 | +Other industries that leverage IoT sensor data for both real-time and |
| 80 | +post-event analytics include: |
| 81 | + |
| 82 | +- **Manufacturing:** Oftentimes manufacturing machinery is loaded with IoT |
| 83 | + sensors that operate for hours at a time, similar to a rocket launch. |
| 84 | + It’s important that analysis can be done while the machinery is |
| 85 | + operating and post-operation. |
| 86 | + |
| 87 | +- **Logistics:** From the modes of transportation to the packages themselves, |
| 88 | + IoT sensors enable supply chain optimization both while in-transit and |
| 89 | + post-transit. |
| 90 | + |
| 91 | +- **eCommerce:** In addition to warehouses and shipping logistics, retailers |
| 92 | + use real-time insights to drive personalization or improve business |
| 93 | + processes. |
| 94 | + |
| 95 | +Reference Architectures |
| 96 | +----------------------- |
| 97 | + |
| 98 | +Without MongoDB |
| 99 | +~~~~~~~~~~~~~~~ |
| 100 | + |
| 101 | +Without MongoDB, this solution often requires a variety of bolt-on or |
| 102 | +disparate databases to support the variety of data types being |
| 103 | +generated, and data needs to be moved and transformed to reach the |
| 104 | +systems that are used to action this data (like a BI tool). |
| 105 | + |
| 106 | +.. figure:: /includes/images/industry-solutions/rocket-launch-without-MongoDB.svg |
| 107 | + :figwidth: 1200px |
| 108 | + :alt: reference architecture without MongoDB |
| 109 | + |
| 110 | + Figure 1. Reference architecture without MongoDB |
| 111 | + |
| 112 | +With MongoDB |
| 113 | +~~~~~~~~~~~~ |
| 114 | + |
| 115 | +.. figure:: /includes/images/industry-solutions/rocket-launch-with-MongoDB.svg |
| 116 | + :figwidth: 1200px |
| 117 | + :alt: reference architecture with MongoDB |
| 118 | + |
| 119 | + Figure 2. Reference architecture with MongoDB |
| 120 | + |
| 121 | +Data Model Approach |
| 122 | +------------------- |
| 123 | + |
| 124 | +Review of the Atlas cluster deployed for the demonstration, and overview |
| 125 | +of the document model for two main Atlas collections used to store the |
| 126 | +launch data: launchData and notes. There are two main collections used |
| 127 | +in the demo: |
| 128 | + |
| 129 | +- LaunchData. |
| 130 | + |
| 131 | +- Notes. |
| 132 | + |
| 133 | +In general, the rocket metrics are produced by the rockets in 4-element |
| 134 | +tuples having the structure below: |
| 135 | + |
| 136 | +- device, timestamp, metric, value. |
| 137 | + |
| 138 | +In the real-world scenario that inspired this demonstration, an |
| 139 | +intermediate app server collected these tuples and aggregated them by |
| 140 | +device and timestamp. When all the metrics for a given device and |
| 141 | +timestamp were received, these metrics were written to the launchData |
| 142 | +collection as a single document. This document model groups the metrics |
| 143 | +into documents that match the way the data is read and analyzed by the |
| 144 | +application (by device and time). This follows the MongoDB schema design |
| 145 | +principle that data that is accessed together is stored together. This |
| 146 | +document model also was designed to work with MongoDB time-series |
| 147 | +collections. The time and metrics fields are top-level fields and the |
| 148 | +meta data (device name) is encapsulated in a field called “meta”. |
| 149 | + |
| 150 | +.. code-block:: javascript |
| 151 | + :copyable: true |
| 152 | + |
| 153 | + { |
| 154 | + _id: ObjectId("62f2f8b5800b621ee724bb94"), |
| 155 | + time: ISODate("2020-10-13T13:33:30.219Z"), |
| 156 | + meta: { device: 'truth' }, |
| 157 | + TIME_NANOSECONDS_TAI: Long("1602596010219040000"), |
| 158 | + truth_pos_CON_ECEF_ECEF_M2: -5268929.31643981, |
| 159 | + truth_pos_CON_ECEF_ECEF_M1: -1387897.36558835, |
| 160 | + truth_pos_CON_ECEF_ECEF_M3: 3306577.65409484, |
| 161 | + truth_vel_CON_ECEF_ECEF_MpS2: -0.00810950305119273, |
| 162 | + truth_vel_CON_ECEF_ECEF_MpS3: 0.00414972080992211, |
| 163 | + truth_quat_CON2ECEF1: -0.458400879273711, |
| 164 | + truth_quat_CON2ECEF2: -0.176758395646534, |
| 165 | + truth_quat_CON2ECEF3: 0.511475024782863, |
| 166 | + truth_vel_CON_ECEF_ECEF_MpS1: 0.00220006484335229, |
| 167 | + truth_quat_CON2ECEF4: 0.7049953208872 |
| 168 | + } |
| 169 | + |
| 170 | +This data can be found in the file `aerospace.archive.gz |
| 171 | +<https://github.com/mongodb-developer/rocket-analytics/blob/main/data-Atlas/aerospace.archive.gz>`__ |
| 172 | +and restored to an Atlas cluster using mongorestore as below. |
| 173 | + |
| 174 | +Building the Solution |
| 175 | +--------------------- |
| 176 | + |
| 177 | +This rocket launch solution was recorded over a three-part livestream. |
| 178 | +This `GitHub <https://github.com/jayrunkel/rocketLivestream>`__ provides |
| 179 | +all the code, data, and links to recordings you need to get started |
| 180 | +building out application-driven analytics that revolve around |
| 181 | +time-stamped IoT sensor data. |
| 182 | + |
| 183 | +.. procedure:: |
| 184 | + :style: normal |
| 185 | + |
| 186 | + .. step:: Development Environment Setup |
| 187 | + |
| 188 | + Prior to watching any of the livestreams, you’ll want to go |
| 189 | + through the setup steps listed in `GitHub readme |
| 190 | + <https://github.com/jayrunkel/rocketLivestream>`__. This will help |
| 191 | + you get started with an Atlas free tier account (if you don’t |
| 192 | + already have one) and download Compass. |
| 193 | + |
| 194 | + .. step:: Pre-launch: Rocket Launch Setup and Basic Analytical Queries with MongoDB’s Aggregation Framework |
| 195 | + |
| 196 | + The `Livestream 1 <https://www.youtube.com/live/RUTsdqehWjo>`__ |
| 197 | + recording will walk you through the data captured in a rocket |
| 198 | + launch and how to write basic aggregation queries with MongoDB’s |
| 199 | + query API. Then you’ll perform basic analytics on the IoT sensor |
| 200 | + data with Atlas Charts. `GitHub folder |
| 201 | + <https://github.com/jayrunkel/rocketLivestream/tree/main/liveStream1>`__. |
| 202 | + |
| 203 | + .. step:: During Launch: Embedding Visualizations and Building Search Capabilities in a React App for Real-Time Analytics |
| 204 | + |
| 205 | + `Livestream 2 <https://www.youtube.com/live/-jSQhlkaBCc>`__ will |
| 206 | + show you how to embed charts into a React app for real-time |
| 207 | + analytics, and how to create a search capability for finding |
| 208 | + insights for specific notes, automated or manually generated, of |
| 209 | + interest. `GitHub folder |
| 210 | + <https://github.com/jayrunkel/rocketLivestream/tree/main/liveStream2>`__. |
| 211 | + |
| 212 | + .. step:: Post-launch: Finding Insights Across Multiple Application and Third-Party Data Sources |
| 213 | + |
| 214 | + Lastly, `Livestream 3 |
| 215 | + <https://www.youtube.com/watch?v=VOxy5VRk4g0>`__ will show you how |
| 216 | + to find post-operation insights by combining rocket launch data |
| 217 | + with weather data from the same time frame and analyzing it in |
| 218 | + Tableau. `GitHub folder |
| 219 | + <https://github.com/jayrunkel/rocketLivestream/tree/main/liveStream3>`__. |
| 220 | + |
| 221 | +Key Learnings |
| 222 | +------------- |
| 223 | + |
| 224 | +MongoDB's modern, multi-cloud database platform enables |
| 225 | +application-driven IoT analytics by providing the following |
| 226 | +capabilities: |
| 227 | + |
| 228 | +- Modeling data across multiple data types. |
| 229 | + |
| 230 | +- Analyzing time series data with window functions. |
| 231 | + |
| 232 | +- Leveraging search indexes. |
| 233 | + |
| 234 | +- Doing in-place aggregation pipelines for minimizing ETL processes. |
| 235 | + |
| 236 | +- Finding insights from hot data (live application) and cold data (third party in Amazon S3). |
| 237 | + |
| 238 | +- Integrating functions to react to real-time data. |
| 239 | + |
| 240 | +Technologies and Products Used |
| 241 | +------------------------------ |
| 242 | + |
| 243 | +MongoDB Developer Data Platform |
| 244 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 245 | + |
| 246 | +- `Atlas Charts <https://www.mongodb.com/products/platform/atlas-charts>`__ with embedded charts |
| 247 | +- `Atlas Database <https://www.mongodb.com/products/platform/atlas-database>`__ with dedicated analytics nodes |
| 248 | +- `Atlas Data Federation <https://www.mongodb.com/products/platform/atlas-data-federation>`__ |
| 249 | +- `Atlas Search <https://www.mongodb.com/products/platform/atlas-search>`__ |
| 250 | +- `Atlas SQL Interface <https://www.mongodb.com/products/platform/atlas-sql-interface>`__ |
| 251 | +- `Atlas Triggers <https://www.mongodb.com/docs/atlas/atlas-ui/triggers/>`__ |
| 252 | +- `MongoDB Aggregation Pipeline <https://www.mongodb.com/docs/v7.0/core/aggregation-pipeline/>`__ |
| 253 | +- `MongoDB Time Series <https://www.mongodb.com/products/capabilities/time-series>`__ with window functions |
| 254 | + |
| 255 | +Partner Technologies |
| 256 | +~~~~~~~~~~~~~~~~~~~~ |
| 257 | + |
| 258 | +- `Amazon S3 <https://www.mongodb.com/products/platform/atlas-cloud-providers/aws>`__ |
| 259 | +- `Tableau <https://www.mongodb.com/try/download/tableau-connector>`__ |
| 260 | + |
| 261 | +Authors |
| 262 | +------- |
| 263 | + |
| 264 | +- Jay Runkel, MongoDB |
0 commit comments