[FEATURE] RDF Validate HEalth DCAT-AP with Shacl and validate the structure ShEx

[ma3u]

Summary

While SHACL is often preferred for RDF validation in this domain, ShEx could be used to validate the structure of HealthDCAT-AP datasets.

Motivation

We need quality measures that run on the data source level, before data sharing. So the researchers know a honest assessment of the existing data and bring this to the W3C DCAT Catelog.

• https://kanzo.site/
• https://kanzo-tech.github.io/keasy.github.io/
• https://www.evidenze.ai/
• https://www.healthinformationportal.eu/healthdcat-ap
• https://shex.io/

• legal and human identities
• Federated Identity is OpenID4VC
• Vocabulary Hub (Worl Ontology)

Proposed Solution

Kanzo provide tools for data providers to show the quality in the data catalog in standardized way.

Alternatives Considered

Any alternative approaches you have thought about.

Additional Context

Links to specifications (EHDS, HealthDCAT-AP, FHIR R4, OMOP CDM, Eclipse EDC), mockups,
or related issues.

[ma3u]

Assessment & plan

Short answer: SHACL yes (primary), ShEx only as a time-boxed spike.

• SHACL is where all the official artefacts live: the EU operates an ITB-based HealthDCAT-AP SHACL validator for HealthData@EU (with an API for automation), Health-RI publishes open SHACL shapes, and SEMICeu ships DCAT-AP 3.0 shapes. Mature runtimes exist for both our stacks (rdf-validate-shacl for the TypeScript proxy, pyshacl for the Python enricher).
• ShEx has no official HealthDCAT-AP schema (the old healthdcat-ap.github.io spec was decommissioned and migrated to EC infrastructure, SHACL-only). A ShEx schema would be hand-authored and maintained in parallel for little extra signal — planned as a 1-day spike with the expected outcome "covered by SHACL".

The motivation in this issue — honest, source-level quality assessment visible to researchers in the catalog before sharing — is addressed by combining SHACL conformance with W3C DQV (Data Quality Vocabulary) measurements published in the catalog itself (the standardized version of what Kanzo does).

What exists today (verified in repo)

• Catalog JSON-LD is generated in two places: the neo4j-proxy (GET /catalog/datasets, full dcat:Catalog with healthdcatap:* properties) and a smaller client-side serializer behind the "Download DCAT-AP" button (ui/src/app/catalog/page.tsx) — with two different healthdcatap: namespace URIs.
• Two divergent HealthDataset write paths (datasetId + PUBLISHED_BY from the federated enricher/seeds vs id + PUBLISHES from the UI POST).
• No SHACL/ShEx/DQV anywhere yet: the federated enricher only does envelope schema checks (models.py, deliberately loose per ADR-020); dct:conformsTo is an unchecked string.

Phased plan (detail in docs/planning/future/issue-26-healthdcat-ap-validation.md)

Phase	What	Where
0 — Hygiene (prereq)	One canonical JSON-LD serializer, one pinned healthdcatap: namespace, unify the two HealthDataset write paths — validation would instantly flag all three	ui/src/app/catalog/page.tsx, services/neo4j-proxy, migration Cypher
1 — SHACL at the source	Vendor pinned shapes (DCAT-AP 3.0 + Health-RI; swap to official EC shapes when published), POST /validate/dcat + GET /catalog/validate on the proxy, CI gate validating seeded catalog output; optional nightly cross-check against the EU ITB validator API (data is fully synthetic)	services/neo4j-proxy
2 — Ingest-side validation	pyshacl in the catalog-enricher for crawled federated catalogs — non-blocking: store shaclConformant / violation counts on the dataset + audit event + metric instead of dropping data	services/catalog-enricher
3 — DQV quality in the catalog	Per-dataset dqv:QualityMeasurement (field completeness %, SHACL conformance, freshness, existing k-anonymity signal) as graph nodes, exported in the JSON-LD, shown as quality badges on catalog cards	neo4j/init-schema.cypher, both JSON-LD endpoints, catalog UI + mocks + e2e
4 — ShEx spike	Minimal hand-authored ShEx schema (run with shex.js) vs the SHACL shapes on the same fixtures; keep only if it catches something SHACL misses	1-day time-box

Out of scope here: OpenID4VC (already the DCP/identityhub track, ADR-007 / issue #97) and the Vocabulary Hub idea (today's partial equivalent is the enricher's SNOMED/LOINC/ICD-10 theme classification — worth its own issue if wanted).

An ADR-031 "Catalog metadata validation & quality (SHACL + DQV)" will be proposed when Phase 1 starts (pinning per ADR-029).

Further sources from the issue: European Health Information Portal — HealthDCAT-AP · Kanzo · Keasy · evidenze.ai

[ma3u]

What we already have implemented (verified inventory, 2026-07-17)

More is in place than the plan above implied — several quality/validation building blocks exist, just none at the RDF/SHACL level yet:

Building block	Where	What it does today
Graph structural validation	GET /api/graph/validate (journeys J160–J164 in 15-persona-graphs.spec.ts)	Homegrown "shape validation" of the property graph: required identifying properties per label (incl. HealthDataset {datasetId, title}), allowed edge rules per layer, orphan detection, unknown-label detection, issue summary. Conceptually SHACL-on-the-property-graph — but it validates the graph, not the exported DCAT-AP RDF.
DataQualityLabelCredential (DCP)	issued by the issuerservice (README §services), seeded for both DATA_HOLDER clinics and linked to HealthDatasets in neo4j/register-ehds-credentials.cypher, NLQ-discoverable via the glossary	A verifiable credential asserting dataset quality already flows through the dataspace, and ODRL policies can require it. What's missing: nothing computes or backs the label — it's asserted, not measured.
Cohort data-quality snapshot (issue #19)	computeCohortDataQuality in the neo4j-proxy	SNOMED coverage % over Conditions + RxNorm coverage % over MedicationRequests, global and cohort-scoped, returned as dataQuality in NLQ responses. This is a source-level quality measure — currently only surfaced in NLQ, not in the catalog.
k-anonymity suppression (issue #8)	federated query path in the proxy	Dual-side suppression with aggregateSuppressed / contributor_k_violation persisted on QueryAuditEvent — a privacy-quality signal per dataset/contributor.
Ingest envelope validation	services/catalog-enricher (models.py, mapping.py)	Pydantic envelope checks, SNOMED/LOINC/ICD-10 theme classification, unknownThemes[] + metric for unrecognized vocabularies.
JSON-LD DCAT-AP export	proxy GET /catalog/datasets[/:id], UI "Download DCAT-AP"	The RDF serialization that SHACL would validate already exists (with the two-serializer/namespace divergence noted above).

Consequence for the plan

• Phase 1 is smaller than it looks: the export exists; we only add shapes + a validator over it.
• Phase 3 gets a head start: computeCohortDataQuality (coding coverage) and the k-anonymity signal are the first two DQV dimensions, already computed — Phase 3 is mostly re-publishing existing measures as dqv:QualityMeasurement in the catalog instead of inventing new ones.
• New connection worth making: Phase 3's DQV measurements should become the evidence backing the existing DataQualityLabelCredential — measured quality feeding credential issuance instead of a bare assertion. That closes the loop the issue asks for (honest, standardized quality at the source).
• /api/graph/validate stays as-is (it guards the property graph); SHACL guards the exported RDF. Complementary, not redundant.

References: plan doc docs/planning/future/issue-26-healthdcat-ap-validation.md · ADR-003 · ADR-020 · issues #8, #19