-
Notifications
You must be signed in to change notification settings - Fork 18
Description
Currently, sample source data when configuring ingest is only persisted temporarily on client-side. If a user refreshes the page, or navigates away from the page, any input sample data is lost. Additionally, if a user configures ingestion and successfully inputs sample data, but then later makes some change to their ingestion configuration (after refreshing the page or navigating away), the data is also lost. This can make it difficult for a user to re-run any updates to their ingestion flow, as it requires them to go and re-upload the temporary sample source data. In the worst case, they may not have access to that original data, or just simply not remember.
Ideally, any sample data the user provides is persisted in some semi-temporary state, such that users can always refer to it when whey go to update their ingestion flow, even after navigating away from the page.
A few different options to explore include different caching mechanisms, or even providing temporary indices associated to that workflow to index that data.