Skip to content

docs: add FinTech data quality framework use case guide - #347

Open
mitalidaduria wants to merge 14 commits into
open-metadata:mainfrom
mitalidaduria:docs/fintech-data-quality-framework
Open

docs: add FinTech data quality framework use case guide#347
mitalidaduria wants to merge 14 commits into
open-metadata:mainfrom
mitalidaduria:docs/fintech-data-quality-framework

Conversation

@mitalidaduria

Copy link
Copy Markdown

This PR adds a comprehensive 'FinTech Data Quality Framework' guide to the documentation under data quality features.

It outlines how to handle 3 critical financial data failure modes (settlement mismatches, orphaned records, and latency violations) using OpenMetadata's native profiling and test suites (such as columnValuesToBeBetween, columnValuesToBeUnique, and null prevention tests). This bridges platform capabilities with a highly requested industry use case.

Expanded the metadata standard documentation to include detailed explanations of machine learning concepts such as model drift, data lineage, feature engineering, label skew, and data quality SLA.
add business-friendly ML glossary with payments domain examples
<a href="https://openmetadatastandards.org/" target="_blank" rel="noopener noreferrer" className="link">Visit openmetadatastandards.org</a>

You'll find quick starts, schema references, semantic web resources, and contribution guides on the official site.
## Machine Learning & Data Governance Expansion

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs.json redirects this modified route to the unchanged canonical v1.13.x/api-reference/main-concepts/metadata-standard page, so none of this replacement is published; the PR also adds no FinTech guide matching its title. Add the guide under a current versioned path with frontmatter and register it in docs.json, and revert this redirected duplicate.

@ayush-shah ayush-shah left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defect-first review on head cc1f8a0:

  • [P1] Resolve the docs.json conflict. Conflict markers at lines 1495–1507 make the file invalid JSON, so Mintlify cannot load navigation. Retain the intended page entry and the existing Bulk Import group.
  • [P2] Register a publishable guide path. The new navigation target does not exist; the guide is under content/main/... without page frontmatter. Move it to the registered versioned path with frontmatter or register an existing publishable path.
  • [P2] Use a referential-integrity test for orphan records. columnValuesToNotBeNull only rejects nulls; an orphan key can be non-null while lacking a parent. Use a custom SQL or referential comparison and keep null validation separate.
  • [P2] Replace the nonexistent freshness test. tableRowCountToBeGreaterThan is unavailable, and row count does not compare event time with ingest time. Use a supported freshness test or concrete SQL over MAX(created_at).
  • [P2] Remove the unrelated glossary change. The canonical metadata-standard file adds an ML glossary unrelated to this FinTech guide and overlaps #331. Revert it from this PR.

Validation: jq fails on the current docs.json; the navigation target and test-definition searches were checked against the exact head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants