Project: Helping LLMs view IDC images #1649
Description
Draft Status
Ready - team will start page creating immediately
Category
Cloud / Web
Key Investigators
- Michael Halle (BWH, USA)
Project Description
The Imaging Data Commons is filled with imaging data, but idc-index and database searches only return textual data/metadata to LLMs. Modern LLMs have vision capabilities and can view selected images. I will complete a standalone non-interactive imaging utility that will give LLMs an image viewing capability for IDC data.
Objective
- A standalone python CLI for displaying an image or mosaic from an IDC series.
- A Claude Skill (or an extension of an existing skill) that teached an LLM to look at IDC images.
Approach and Plan
- Finish the CLI
- Get feedback
- Finish the Skill
Progress and Next Steps
1 Prototype "idc-series-preview" is written but needs knowledgable eyes on the project. It also needs some features like annotations added. This project could well be done using fancy DICOM viewer code, but for right now Python works fine for the prototype.
Illustrations
No response
Background and References
No response