🧠 Awesome Medical World Models

A curated, clinically organized list of world models for medical artificial intelligence: medical imaging, electronic health records, treatment planning, ultrasound guidance, surgical robotics, and virtual-cell / biological simulation.

This repository is designed as a one-stop research map for people building predictive, action-conditioned, counterfactual, and planning-oriented medical AI systems.

---

🚩 News & Updates

• [2026-06] Initial release. Medical world-model papers organized by clinical application.
• [2026-06] Verified links added. Paper, project, code, and dataset links were checked manually during repository creation.
• [Ongoing] Contributions welcome. Add missing papers, code, datasets, benchmarks, or corrected links through a pull request.

---

📌 Table of Contents

• 🎯 Aim of the Project
• 🧭 What Is a Medical World Model?
• 🗺️ Taxonomy
• 📖 Surveys, Reviews, and Position Papers
• 🏛️ Foundational World Model Papers
• 🧠 Medical Imaging World Models
• 🏥 EHR and Clinical Trajectory World Models
• 💊 Treatment Planning and Counterfactual Decision Support
• 🫀 Ultrasound and Probe Guidance World Models
• 🤖 Surgical and Robotic Medical World Models
• 🧬 Molecular, Biological, and Virtual Cell World Models
• 📊 Datasets and Benchmarks
• 🧪 Evaluation Metrics
• 🛠️ Repository Files
• 🤝 Contributing
• 📝 Citation

---

🎯 Aim of the Project

Medical AI is moving from static prediction toward simulation of future medical states under actions and interventions. This repository organizes that emerging direction.

It aims to:

• map the medical world-model literature into clinically meaningful categories;
• make paper, project, code, dataset, and benchmark links easy to find;
• distinguish representation learning from intervention-aware simulation and planning;
• highlight reproducible resources for medical imaging, EHR modeling, ultrasound, surgery, and virtual cells;
• help researchers identify open problems in safety, uncertainty, causality, and clinical validation.

---

🧭 What Is a Medical World Model?

A medical world model is a predictive model that learns a structured representation of a medical system and simulates how that system may evolve over time, often conditioned on clinical actions, treatments, robotic motions, or biological perturbations.

Typical inputs include:

• X-ray, CT, MRI, ultrasound, pathology, or surgical video;
• EHR events, diagnoses, medications, lab values, procedures, and notes;
• molecular profiles, perturbation descriptions, omics context, and biological knowledge graphs;
• clinical or physical actions such as treatment plans, probe movement, robot motion, or intervention protocols.

Typical outputs include:

• future medical images or masked regions;
• disease trajectory forecasts;
• post-treatment outcome simulation;
• risk, survival, or response estimates;
• treatment ranking and counterfactual recommendations;
• surgical or robotic policy evaluation;
• virtual-cell perturbation response and mechanistic hypotheses.

---

🗺️ Taxonomy

Category	Goal	Typical Inputs	Typical Outputs
Medical imaging world models	Learn anatomy / disease dynamics	X-ray, CT, MRI	future image, representation, segmentation, diagnosis
EHR trajectory world models	Simulate longitudinal patient states	EHR events, notes, labs, interventions	future events, clinical timeline, risk trajectory
Treatment planning world models	Evaluate intervention-conditioned futures	image + clinical context + treatment option	post-treatment state, survival, treatment ranking
Ultrasound / probe guidance	Model image changes under probe motion	ultrasound video + probe action	next movement, target view, navigation signal
Surgical world models	Simulate surgical scene and robot action effects	surgical video, robot state, action text	action-conditioned video, policy evaluation, synthetic data
Biological / virtual cell world models	Simulate cellular response to perturbation	gene expression, perturbation, biological KG	DE prediction, pathway reasoning, mechanistic hypotheses

---

📖 Surveys, Reviews, and Position Papers

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2026	Medical World Models Survey	A Survey on World Models: Application of World Models for Medical Artificial Intelligence	Manuscript	Survey of clinical world models	—	n/a
2025	Clinical WM Review	Beyond Generative AI: World Models for Clinical Prediction, Counterfactuals, and Planning	NeurIPS 2025 LAW Workshop / arXiv	Clinical WM taxonomy and research agenda	Paper / Project	n/a
2025	General WM Survey	Understanding World or Predicting Future? A Comprehensive Survey of World Models	ACM Computing Surveys	Comprehensive world-model survey	Paper	n/a
2023	MBRL Survey	Model-Based Reinforcement Learning: A Survey	Foundations and Trends in Machine Learning	MBRL foundations	Paper	n/a
2025	Critique	Critiques of World Models	arXiv	Conceptual critique and limitations	Paper	n/a

---

🏛️ Foundational World Model Papers

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
1943	Internal Model Hypothesis	The Nature of Explanation	Cambridge University Press	Internal predictive models	Paper	n/a
1991	Dyna	Dyna, an Integrated Architecture for Learning, Planning, and Reacting	ACM SIGART Bulletin	Learning and planning from simulated experience	Paper	n/a
2011	PILCO	PILCO: A Model-Based and Data-Efficient Approach to Policy Search	ICML	Probabilistic dynamics and policy search	Paper / Project	n/a
2015	ACVP	Action-Conditional Video Prediction using Deep Networks in Atari Games	NeurIPS	Action-conditioned video prediction	Paper	not found
2015	E2C	Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images	NeurIPS	Latent dynamics for control	Paper	not found
2018	World Models	Recurrent World Models Facilitate Policy Evolution	NeurIPS	Latent imagination for control	Paper / Project / Code	available
2019	PlaNet	Learning Latent Dynamics for Planning from Pixels	ICML	Latent planning with RSSM and CEM	Paper / Code	available
2020	Dreamer	Dreamer: Reinforcement Learning with Latent Imagination	ICLR	Actor-critic learning from imagined rollouts	Paper / Code	available
2021	DreamerV2	Mastering Atari with Discrete World Models	ICLR	Discrete stochastic latent world model	Paper / Code	available
2025	DreamerV3	Mastering Diverse Control Tasks Through World Models	Nature	General-purpose world-model control	Paper / Project / Code	available
2023	I-JEPA	Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture	CVPR	Latent representation prediction	Paper / Code	available
2024	V-JEPA	V-JEPA: Latent Video Prediction for Visual Representation Learning	arXiv / Meta AI	Video latent prediction	Paper / Project / Code	available
2025	V-JEPA 2	V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning	arXiv	Understanding, prediction, and planning	Paper / Project / Code	available
2024	Genie	Genie: Generative Interactive Environments	ICML	Interactive environment generation	Paper / Project	official code not found
2025	PAN	PAN: A World Model for General, Interactable, and Long-Horizon World Simulation	arXiv	Long-horizon action-conditioned simulation	Paper / Project	official code not found
2025	GigaWorld-0	GigaWorld-0: World Models as Data Engine to Empower Embodied AI	arXiv	Synthetic data engine for embodied AI	Paper / Code	available

---

🧠 Medical Imaging World Models

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2026	Brain-WM	Brain-WM: Brain Glioblastoma World Model	arXiv	Treatment-aware future MRI generation and treatment planning	Paper / Code	available
2025	X-WIN	X-WIN: Building Chest Radiograph World Model via Predictive Sensing	arXiv / CVPR 2026	3D-aware CXR representation learning	Paper	official code not found
2025	Xray2Xray	Xray2Xray: World Model from Chest X-rays with Volumetric Context	arXiv	Volumetric-context CXR representation learning	Paper	official code not found
2025	CheXWorld	CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning	CVPR	Self-supervised radiograph representation learning	Paper / Project / Code	available
2025	TaDiff-Net	Treatment-Aware Diffusion Probabilistic Model for Longitudinal MRI Generation and Diffuse Glioma Growth Prediction	IEEE TMI	Future MRI and tumor-mask generation under treatment	Paper / Code	available

---

🏥 EHR and Clinical Trajectory World Models

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2026	EHRWorld	EHRWorld: A Patient-Centric Medical World Model for Long-Horizon Clinical Trajectories	arXiv	Long-horizon clinical trajectory simulation	Paper	official code not found
2024	Foresight	Foresight: A Generative Pretrained Transformer for Modelling Patient Timelines Using Electronic Health Records	The Lancet Digital Health	Patient timeline forecasting and digital-twin simulation	Paper / Project / Code	available
2025	CoMET	Generative Medical Event Models Improve with Scale	arXiv / Microsoft Research	Medical event foundation model scaling	Paper / Project	official code not found

---

💊 Treatment Planning and Counterfactual Decision Support

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2025	MeWM	Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planning	ICCV / arXiv	Treatment-conditioned tumor simulation and protocol selection	Paper / Project / Code	available
2025	CLARITY	CLARITY: Medical World Model for Guiding Treatment Decisions by Modeling Context-Aware Disease Trajectories in Latent Space	arXiv	Latent treatment-conditioned disease trajectory simulation	Paper	official code not found

---

🫀 Ultrasound and Probe Guidance World Models

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2025	EchoWorld	EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance	CVPR	Motion-aware representation learning for probe guidance	Paper / Project / Code	available
2024	Cardiac Copilot / Cardiac Dreamer	Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model	MICCAI	Standard-plane navigation and probe guidance	Paper / Project	official code not found

---

🤖 Surgical and Robotic Medical World Models

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2025	Surgical Vision World Model	Surgical Vision World Model	MICCAI Workshop / arXiv	Action-controllable surgical video generation	Paper / Project / Code	repository available; code coming soon
2025	Cosmos-Surg-dVRK	Cosmos-Surg-dVRK: World Foundation Model-Based Automated Online Evaluation of Surgical Robot Policy Learning	arXiv	Automated policy evaluation through world-model rollouts	Paper	official code not found
2025	SurgWorld	SurgWorld: Learning Surgical Robot Policies from Videos via World Modeling	arXiv	Synthetic video-action data for surgical VLA policy learning	Paper / Project / Code	related model suite available
2025	Surgical WM Expert Assessment	How Far Are Surgeons from Surgical World Models? A Pilot Study on Zero-Shot Surgical Video Generation with Expert Assessment	arXiv	Expert evaluation of surgical video generation	Paper	n/a
2025	MT World Model	World Model for AI Autonomous Navigation in Mechanical Thrombectomy	MICCAI	Autonomous mechanical thrombectomy navigation	Paper / Project	official code not found

---

🧬 Molecular, Biological, and Virtual Cell World Models

Year	Model	Paper	Venue / Type	Main Task	Links	Code Status
2025	VCWorld	VCWorld: A Biological World Model for Virtual Cell Simulation	arXiv / ICLR 2026	Virtual-cell perturbation simulation and mechanistic reasoning	Paper / Code	available
2026	Lingshu-Cell	Lingshu-Cell: A Generative Cellular World Model for Transcriptome Modeling Toward Virtual Cells	arXiv	Conditional transcriptome simulation under perturbation	Paper / Project	official code not found
2026	Virtual Cell WM	A World Model of the Virtual Cell	Preprint / blog	Operational framing of virtual cells as world models	Paper	n/a

---

📊 Datasets and Benchmarks

See data/datasets.csv for a machine-readable dataset list.

Dataset	Domain	Modality	Use Case	Link
LUMIERE	Brain MRI / glioma	Longitudinal MRI	Brain-WM, glioma progression modeling	Link
MIMIC-CXR	Chest radiography	Chest X-ray + reports	X-WIN, CheXWorld, CXR foundation models	Link
CheXpert	Chest radiography	Chest X-ray	Chest pathology classification benchmark	Link
NIH ChestX-ray14	Chest radiography	Chest X-ray	CXR disease classification	Link
VinDr-CXR	Chest radiography	Chest X-ray	CXR pathology detection	Link
RSNA Pneumonia Detection Challenge	Chest radiography	Chest X-ray	Pneumonia detection benchmark	Link
JSRT	Chest radiography	Chest X-ray	Nodule detection / CXR evaluation	Link
COVIDx	Chest radiography	Chest X-ray	COVID-19 CXR evaluation	Link
HCC-TACE / HCC-TACE-Seg	Oncology treatment planning	CT / MRI + treatment outcomes	MeWM, TaDiff-style treatment simulation	—
EHRWorld-110K	EHR	Longitudinal clinical events	EHRWorld long-horizon trajectory simulation	Link
MIMIC-III	EHR / ICU	Clinical events + notes	Foresight and EHR timeline modeling	Link
MIMIC-IV	EHR / ICU	Clinical events	Clinical trajectory modeling	Link
Epic Cosmos	EHR	Medical events	CoMET scaling study	Link
SurgToolLoc-2022	Surgical video	Endoscopic surgical video	Surgical Vision World Model	Link
SATA	Surgical robot action	Surgical video + action text alignment	SurgWorld	Link
Tahoe-100M	Single-cell biology	Single-cell perturbation data	VCWorld / virtual-cell simulation	Link
GeneTAK	Biological knowledge / perturbation	Gene / pathway / perturbation information	VCWorld mechanistic reasoning	Link

---

🧪 Evaluation Metrics

See data/benchmarks.csv and docs/evaluation.md for details.

Domain	Recommended Metrics	Example Models
Temporal clinical prediction	Success@k; retention rate; event prediction accuracy; SMAPE	EHRWorld, Foresight, CoMET
Treatment planning	F1-score; precision; recall; Jaccard index; specificity; treatment ranking accuracy	MeWM, CLARITY, Brain-WM
Survival / risk prediction	C-index; Brier score; calibration error; risk stratification AUC	CLARITY, MeWM, Foresight
Medical image generation	SSIM; PSNR; LPIPS; FID; expert Turing test	Brain-WM, TaDiff, MeWM
Segmentation / lesion localization	Dice; IoU; Hausdorff distance; lesion-wise F1	TaDiff, tumor mask forecasting
Probe / robotic guidance	Navigation error; plane-guidance error; task success rate; path ratio	EchoWorld, Cardiac Copilot, MT World Model
Surgical policy evaluation	Policy rank correlation; success/failure classification; rollout fidelity	Cosmos-Surg-dVRK, SurgWorld
Virtual-cell simulation	DE prediction; perturbation ranking; pathway consistency; mechanism agreement	VCWorld, Lingshu-Cell
Counterfactual validity	Intervention sensitivity; causal consistency; off-policy agreement; uncertainty calibration	MeWM, CLARITY, clinical WM reviews

---

✅ Link and Code Status

This repository uses explicit code-status labels:

• available — official or clearly associated code repository found;
• repository available; code coming soon — repository exists but implementation is not fully released yet;
• related model suite available — related release exists but is not necessarily a paper-specific implementation;
• official code not found — no verified official implementation found at the time of curation;
• n/a — no implementation expected, usually survey/review/conceptual work.

Run the link checker locally:

python scripts/check_links.py data/papers.csv data/datasets.csv

---

🛠️ Repository Files

awesome-medical-world-models/
├── README.md
├── CONTRIBUTING.md
├── CITATION.cff
├── LICENSE
├── assets/
│   ├── medical_applications_overview.png
│   ├── medical_world_model_taxonomy.png
│   └── world_models_timeline.png
├── data/
│   ├── papers.csv
│   ├── datasets.csv
│   └── benchmarks.csv
├── docs/
│   ├── taxonomy.md
│   ├── evaluation.md
│   ├── datasets.md
│   ├── open_challenges.md
│   └── link_status.md
└── scripts/
    └── check_links.py

Recommended papers.csv fields:

year,category,model,title,venue,modality,task,paper_url,code_url,project_url,code_status,notes

---

⭐ Star History

If this repository helps your work, please consider starring it and sharing it with the medical AI, world model, and clinical foundation model communities.