Our project aims to utilize large language models to generate accurate words for the Impression section of medical reports by leveraging patients' medical findings and background information. The primary objective is to assist physicians in optimizing their documentation process, ultimately reducing the time required for generating comprehensive reports.
The data for this project was obtained from the University of Chicago Medicine, consisting of 789,280 de-identified radiology reports.
Our data is textual, well-organized, and covers multiple modalities. We chose MRI for modeling considering the data size.
Here is the workflow of our project:
Several large language base models are experimented including Alpaca-7b, Medalpaca-7b, Llama2-7b, Llama2-13b and Mistral-7b:
The selection of the model is based on:
- Open-Source Availability
- Manageable Model Size
- Proven Performance
- Use Case
To comprehensively assess the performance of the models in generating precise impressions from patients' medical findings and background information, ROUGE scores have been selected as the primary evaluation metrics. The baseline results for comparison are derived from the paper titled 'Customized Impression Prediction from Radiology Reports Using BERT and LSTMs'.[1]
Here is the result:
Below is an example demonstrating our user-friendly interface, developed with Gradio, utilizing the fine-tuned Mistral-7b model:
[1] Gundogdu, B., Pamuksuz, U., Chung, J. H., Telleria, J. M., Liu, P., Khan, F., & Chang, P. J. (2021). Customized impression prediction from radiology reports using BERT and LSTMs. IEEE Transactions on Artificial Intelligence.