Real-Time Data Sharing outline#11399
Conversation
| * **Standardized Contracts** – AsyncAPI 3.0 contracts define stream schemas and metadata | ||
|
|
||
| ### Change Event Streams | ||
|
|
There was a problem hiding this comment.
Is there additional information we should add here?
|
|
||
| Change events are published to Kafka topics and consumed by downstream systems. | ||
|
|
||
| ## Known Limitations |
There was a problem hiding this comment.
Need additional details on multi-node deployment
| ## Modeling | ||
|
|
||
| ### Modeling the Change Event Stream | ||
|
|
There was a problem hiding this comment.
Where does this occur in Studio Pro?
|
|
||
| #### Exporting the Contract | ||
|
|
||
| Contracts are exported as AsyncAPI 3.0 documents with the following mappings: |
There was a problem hiding this comment.
There is a note in the original document about further explanation on Async API 3.0 being used in change stream contracts and how the fields are mapped. Will that be added here or elsewhere?
| Event Broker and Kafka-compatible systems provide multiple configuration options beyond basic constants. Event Broker configuration is available in app settings, similar to database or workflow configuration. | ||
|
|
||
| ### Connection | ||
|
|
There was a problem hiding this comment.
Does the user need to follow any additional steps to configure? We will want to provide further instruction on where this happens in the Event Broker (where to go in the Settings, potential screenshots)
| Configure the following connection settings: | ||
|
|
||
| * **Server URL** – | ||
| * **Authentication** – Username and password |
There was a problem hiding this comment.
Any additional information needed for the server URL?
| * **Authentication Type** – Mendix Cloud, SASL, AWS, Confluent, or other compatible authentication mechanisms | ||
|
|
||
| ### Publishing | ||
|
|
There was a problem hiding this comment.
Any additional information needed?
|
|
||
| ### Topic Naming | ||
|
|
||
| Configure how streams map to Kafka topics: |
There was a problem hiding this comment.
Any additional information needed?
|
|
||
| ## Metamodel | ||
|
|
||
| The current metamodel has limited support for expressing streams as a concept. Future versions may introduce native stream representations rather than adapting existing metamodel components. |
There was a problem hiding this comment.
We should probably focus on just what the current metamodel can do, as we do want to promise what a future may or may not be able to do.
|
|
||
| ### Using an Independent CDC Component | ||
|
|
||
| An external CDC component (such as Debezium) reads database changes from the Write Ahead Log (WAL) and publishes them to Kafka. This approach has several limitations in the Mendix context: |
There was a problem hiding this comment.
The way this reads seems to encourage users to use the Mendix Runtime to implement. Is that the goal, or can a user effectively use an independent CDC component? We may want to rephrase this section to state more that it is possible to use an independent CDC but have a warning box that explains the challenges they may encounter.
|
|
||
| ## Non-Functional Requirements | ||
|
|
||
| Real-Time Data Sharing addresses the following non-functional requirements: |
There was a problem hiding this comment.
Do we want to expand upon these requirements (explain them in further depth) or just list them as they currenty are below?
|
|
||
| The outbox pattern ensures reliable delivery. Events are persisted locally before publishing, and remain in the outbox until successfully delivered. | ||
|
|
||
| ## Historical Data and Snapshots |
There was a problem hiding this comment.
There was an additional to-do in this section "ADR on snapshots." What else needs to be added and will it be here or elsewhere?
1 participant
No description provided.