docs: add FinTech data quality framework use case guide - #347
docs: add FinTech data quality framework use case guide#347mitalidaduria wants to merge 14 commits into
Conversation
Expanded the metadata standard documentation to include detailed explanations of machine learning concepts such as model drift, data lineage, feature engineering, label skew, and data quality SLA.
add business-friendly ML glossary with payments domain examples
… and styling per review
| <a href="https://openmetadatastandards.org/" target="_blank" rel="noopener noreferrer" className="link">Visit openmetadatastandards.org</a> | ||
|
|
||
| You'll find quick starts, schema references, semantic web resources, and contribution guides on the official site. | ||
| ## Machine Learning & Data Governance Expansion |
There was a problem hiding this comment.
docs.json redirects this modified route to the unchanged canonical v1.13.x/api-reference/main-concepts/metadata-standard page, so none of this replacement is published; the PR also adds no FinTech guide matching its title. Add the guide under a current versioned path with frontmatter and register it in docs.json, and revert this redirected duplicate.
ayush-shah
left a comment
There was a problem hiding this comment.
Defect-first review on head cc1f8a0:
- [P1] Resolve the
docs.jsonconflict. Conflict markers at lines 1495–1507 make the file invalid JSON, so Mintlify cannot load navigation. Retain the intended page entry and the existing Bulk Import group. - [P2] Register a publishable guide path. The new navigation target does not exist; the guide is under
content/main/...without page frontmatter. Move it to the registered versioned path with frontmatter or register an existing publishable path. - [P2] Use a referential-integrity test for orphan records.
columnValuesToNotBeNullonly rejects nulls; an orphan key can be non-null while lacking a parent. Use a custom SQL or referential comparison and keep null validation separate. - [P2] Replace the nonexistent freshness test.
tableRowCountToBeGreaterThanis unavailable, and row count does not compare event time with ingest time. Use a supported freshness test or concrete SQL overMAX(created_at). - [P2] Remove the unrelated glossary change. The canonical metadata-standard file adds an ML glossary unrelated to this FinTech guide and overlaps #331. Revert it from this PR.
Validation: jq fails on the current docs.json; the navigation target and test-definition searches were checked against the exact head.
|
Follow-up on the latest review: all five items remain on head cc1f8a0—docs.json still contains conflict markers; the registered guide target is missing; the orphan check still uses not-null; the freshness test is unavailable; and the unrelated glossary remains. Please resolve these before re-review. |
|
Follow-up on the July 27 review: head |
|
Thanks for the work here. After reviewing our documentation direction, we’ve decided not to maintain standalone FinTech or payments-domain examples in the docs. These PRs are also broader than the scope we want to support. |
This PR adds a comprehensive 'FinTech Data Quality Framework' guide to the documentation under data quality features.
It outlines how to handle 3 critical financial data failure modes (settlement mismatches, orphaned records, and latency violations) using OpenMetadata's native profiling and test suites (such as
columnValuesToBeBetween,columnValuesToBeUnique, and null prevention tests). This bridges platform capabilities with a highly requested industry use case.