Retire notebooks/ and the 'notebooks' pyproject extra

[turbomam]

Drop the notebooks/ directory and the notebooks pyproject extra from metpo.

Why

The notebooks pyproject extra pulls in jupyter, notebook, jupyterlab, matplotlib, numpy, python-levenshtein, chromadb, and openai. This is the main source of Dependabot churn. The 2026-05-18 batch alone was 13 high and 7 medium alerts, all from this dependency tree.

No code under metpo/ imports any Jupyter or IPython modules. Nothing outside the notebooks themselves needs the Jupyter stack.

What is in notebooks/

Two notebooks, plus an .ipynb_checkpoints/ autosave folder and an outdated README.md. The README lists four notebooks; two of those (ols_search_example.ipynb and explore_embeddings.ipynb) have already been deleted, so the README is wrong.

query_chromadb.ipynb (12 cells, ChromaDB-dependent)

Demonstration of semantic similarity search using ChromaDB. It opens the local ChromaDB at data/chromadb/chroma_ols20_nonols4/, embeds a user query using OpenAI's embedding API, and returns the nearest-neighbor ontology terms. The stored collection covers roughly 20 OLS ontologies plus the 4 BioPortal-only ones (D3O, OMP, MPO, n4l_merged). Purely a demo of the pipeline that #364 is in the process of replacing.

assess_ontology_by_api_search.ipynb (49 cells, not ChromaDB-dependent)

Baseline ontology-popularity analysis. It reads 50 METPO term labels (a 20% sample) from data/metpo_terms/metpo_sample_labels.tsv, calls BioPortal's REST search API for each label, and tallies which external ontologies return hits. The result answers "which ontologies most often match METPO labels?" and is useful input for #364. Does not use ChromaDB or any Jupyter-specific machinery beyond the notebook wrapper.

Proposed steps

1. Delete query_chromadb.ipynb. Recoverable from git history if ever needed.
2. Convert assess_ontology_by_api_search.ipynb into a Click command-line script under metpo/scripts/, per the metpo CLAUDE.md script standards. The logic is straightforward: read a TSV of labels, loop over them, call the BioPortal search API, aggregate hits per ontology, write output TSV.
3. Delete the notebooks/ directory and the .ipynb_checkpoints/ autosave folder.
4. Remove the notebooks extra from pyproject.toml; re-run uv lock to drop the Jupyter, Notebook, and JupyterLab dependency tree.
5. Remove the install-notebooks target from the Makefile.

ChromaDB direction (context for #364)

The broader goal is to remove ChromaDB as a metpo dependency. The replacement plan tracked in #364 is a hybrid: OLS4's /api/v2/classes/llm_search endpoint for the roughly 270 OLS-hosted ontologies, and linkml-store with the LLMIndexer for the 4 BioPortal-only ontologies (D3O, OMP, MPO, n4l_merged). Any ChromaDB usage that survives the #364 cleanup must be clearly scoped to only the highest-value BioPortal-only ontologies. Broad ChromaDB indexing is not the long-term path.

Impact

• Removes the Jupyter, Notebook, and JupyterLab dependency tree from uv.lock. Most of the current Dependabot batch becomes impossible in future cycles.
• Aligns with the scope-narrowing track in umbrella issue Umbrella: switch SoT to repo OWL, narrow scope, add maintenance guardrails #433.
• Coupled with Re-scope: replace the DIY ChromaDB pipeline (OLS4 embeddings + linkml-store + local GPU) #364 (ChromaDB retirement), openai and chromadb would also become droppable from the literature pyproject extra. That follow-up work is downstream of this issue.

Related

• Umbrella: switch SoT to repo OWL, narrow scope, add maintenance guardrails #433 (umbrella: SoT switch + scope narrowing). This is one of the named scope-narrowing tracks.
• Re-scope: replace the DIY ChromaDB pipeline (OLS4 embeddings + linkml-store + local GPU) #364 (ChromaDB retirement). The query_chromadb.ipynb deletion aligns with that direction and can also proceed independently.

🤖 Generated with Claude Code

[turbomam]

Notebook status clarified (2 real notebooks, not all chromadb): query_chromadb.ipynb removed in PR #498; assess_ontology_by_api_search.ipynb is the OLS-API-search one — keep and reformulate as a Click CLI (it's the OLS4-search direction and becomes the replacement for the chromadb semantic query). Once that's a CLI, notebooks/ can be retired entirely. The chromadb part of the notebooks extra is gone in #498.