Umbrella: switch SoT to repo OWL, narrow scope, add maintenance guardrails

[turbomam]

Three coupled changes to METPO's maintenance model. Linked issues stay where they are; this tracks coordination.

1. Narrow repo scope. Extract OntoGPT / literature mining / Python codebase out of metpo; keep ontology + ROBOT templates + CI. Cleanup: #249, #279, #309, #319, #379, #380. OntoGPT relocation candidates: #136 #142 #145 #146 #147 #171 #174 #177 #180 #194 #210 #244 #256 #274 #280 #281 #295 #297 #298 #299 #313.

2. Switch SoT from Google Sheets to released OWL. Last released metpo.owl becomes the canonical input for the next cycle; ROBOT templates in the repo are the editable layer; PRs are the unit of change. #366 currently frames this as Sheet↔repo sync; needs to become a one-way handoff. Open question: keep Sheets as draft-only editing surface for undergrads, or replace. Also #378.

3. Add agentic-maintenance guardrails. ROBOT validate + reasoner check + lint on every PR. Linters motivated by existing bugs: label↔range-synonym agreement (#356, #432), formula consistency (#357), synonym formatting (#358), scope (no DB field paths as synonyms, #359), capitalization (#62, #196), Aristotelian definitions (#64), no obsolete classes as definition_source (#307), disjointness (#361), no TODO:add_citation. Infra: #46, #344, #350, #351, #377, #416, #417, #418.

Major track under (1), chromadb retirement: see #364 (Evaluate OLS4 embeddings API + linkml-store to replace DIY ChromaDB pipeline). Retiring local chromadb (~28GB) is one of the larger single wins for the scope-narrowing goal; depends on the OLS4 vs current-embedding quality comparison tracked in #364 + #256.

Resolve before merging either: #228 (no class duals for datatype-property observations) conflicts with the tolerance-class duals in #424.

🤖 Generated with Claude Code

[turbomam]

Progress on the agentic-maintenance guardrails track:

• Add metpo-proposal-lint: deterministic QC/audit guardrails (#433) #452 (merged) added metpo-proposal-lint, implementing several linters this issue calls for: formula-direction (NaCl delta high equivalent class formula is inverted #357), synonym-threshold (BactoTraits GC bin synonym assignments are swapped (3 of 4 classes) #356/METPO labels and threshold synonyms are mismatched on gc_high/gc_low/gc_mid1/gc_mid2 #432), mapping-CURIE-in-definition_source, parent/house-style/id-range checks, plus a released-content audit mode.
• Resolve CodeQL code-quality findings + metpo-edit.owl whitespace #453 (merged) cleared the CodeQL standard findings + metpo-edit.owl whitespace.

Remaining quality work is tracked separately so it doesn't get lost here:

• Clear mypy type errors and make the mypy CI step blocking #454 — clear the 223 mypy errors, then make the mypy CI step blocking.
• Critical Dependabot alert: chromadb pre-auth code injection (CVE-2026-45829, no upstream fix) #455 — critical chromadb Dependabot alert (CVE-2026-45829); durable fix is the Re-scope: replace the DIY ChromaDB pipeline (OLS4 embeddings + linkml-store + local GPU) #364 ChromaDB retirement.

🤖 Generated with Claude Code

[turbomam]

Direction confirmed (Mark, 2026-06-05): move everything related to METPO usage — past or future literature mining (OntoGPT extraction, grounding, evaluation) — out of this repo, soon. This repo should hold the ontology (Sheet/OWL source of truth, build, QC), not the consumption/literature-mining tooling. Combined with the ChromaDB re-scope (#364) and 'no ontology imports' (external terms cited as synonyms/mappings, not imported), this narrows the repo to: ontology content + build + mappings. The literature-mining cluster (#136/#142/#145-147/#171/#177/#180/#194/#210/#244/#274/#280/#281/#295/#297/#298/#299/#313) should transfer/close on extraction.

[turbomam]

Progress on the narrowing: ChromaDB pipeline removal is in PR #498 (first of the re-scope). Next slimming step is the literature-mining extraction (move metpo/literature_mining/ ~24 modules + the OntoGPT issue cluster out). Codebase is currently 110 .py files / 76 CLIs across 13 dirs; lit-mining (~24) + presentations (10, #456) + chromadb (~8) leaving brings it to ~65 in a handful of dirs.

[turbomam]

Makefile target dispositions (for the slimming, 2026-06-05):

• MetaTraits-demo targets (demo-metatraits-mongo, metatraits-helper-files, clean-metatraits-demo): build helper files for the deterministic MetaTraits → KGX transform. That's consumption (kg-microbe-facing), not ontology-building, so it leaves with the literature-mining / usage extraction. Note: the MetaTraits synonym/mapping content stays in the ontology; only the KGX transform machinery goes.
• entity-extraction (extract-all-metpo-entities, clean-entity-extracts): scans historical BioPortal submissions to extract METPO entities — this was the one-time deprecated.tsv bootstrap (Reconcile ID allocation with automated audit tool #375), not routine. Keep as a maintenance-only tool (or drop if the metadata/ontology/historical_submissions/ extracts are removed).

Also confirmed a keeper (do NOT remove): download-external-bioportal-ontologies + download-ontology CLI — they download BioPortal-only ontologies (D3O/MPO/OMP) as OWL, which is embedding-system-agnostic and is the "BioPortal-only ontologies OLS lacks" input the OLS4/linkml-store re-implementation (#364) needs. The chromadb embed step is gone (#498); the download stays.

[turbomam]

Scope-narrowing cleanup candidate (logging, not acting):

sparql/query_metpo_labels.rq is a one-off exploration query (a hardcoded VALUES list of 8 METPO IRIs, last touched in the 2024 reorg). It is not referenced by any Makefile or script, so there is no execution path to break. Removing it is a delete plus updating its catalog references:

• docs/METPO_CAPABILITIES.md
• sparql/README.md
• two archival literature_mining/**/SESSION_HANDOFF_2025-11-06.md copies

For contrast, the other top-level sparql/ exploration queries (e.g. query_metpo_entities.sparql) are still wired into the root Makefile, so they are out of scope for this cleanup.

Surfaced while adding the SPARQL QC meta-test guardrail (#500).

[turbomam]

Retirement convention (track 1, scope-narrowing): code removed from metpo is now archived to a public repo, https://github.com/turbomam/metpo-attic, instead of relying only on git history or release tags.

First instance: #456 retired metpo/presentations/ (9 ICBO 2025 figure-generation scripts plus their one test). They now live in metpo-attic under presentations/ and tests/, with a README recording provenance: source repo, original path, the removing PR, this umbrella, and the removal commit. The metpo docs that referenced the scripts (README, docs/CLI_STANDARDIZATION_PLAN.md, docs/METPO_CAPABILITIES.md) now link to the archive.

Pattern for future scope-narrowing deletions: archive the files to metpo-attic (with provenance), repoint any metpo docs that mention them, then delete in a PR. metpo-attic is a per-retirement snapshot, not a live mirror. Direct candidates for the same treatment: #441 (notebooks), #380 (non-core sheet and BioPortal fetch machinery), #249 (docs/ and notebooks/ de-bloat).

[turbomam]

Five new children under tracks 1 and 2, filed 2026-07-31 after 14 of the 16 content-spreadsheet tabs were deleted and recovered.

• #598 Record: all 16 Google Sheet tabs recovered from Drive revision 7622 after the 2026-07-31 deletion
• #599 Commit the 16 recovered Google Sheet tabs as a static, non-build archive
• #600 Stop merging deprecated terms into the released OWL; keep the archive repo-side only
• #601 Sheet fetch: curl has no -f, so a dead tab silently writes an HTML error page into the TSV
• #602 Make the committed TSVs the source of truth and retire the Google Sheet as a build input

Open decision this umbrella should settle. Track 2 here is titled "switch SoT to repo OWL" and says the last released metpo.owl becomes the canonical input with ROBOT templates as the editable layer. #602 is written the other way round: the committed src/templates/*.tsv are the source of truth and there is no OWL-as-input step. Those are different designs and only one should survive.

The TSV framing is the smaller change, because the templates are already committed and CI already builds from them; retiring the sheet is mostly deleting the fetch path. The OWL framing implies a round-trip from released OWL back into editable form, which nothing in the repo does today. Recording the divergence rather than resolving it unilaterally.

Also relevant to track 3. #600 proposes reversing #378 Decide whether deprecated entities should be in the released OWL, which closed 2026-06-08 in favour of including them. The cost is that kg-microbe, which reads metpo.owl from main, loses the ability to detect that a referenced ID is obsolete. Worth deciding at umbrella level since it is a downstream contract change.

#379 Assess all non-core Google Sheet tabs for deletion is being closed as superseded.