# Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model

arXiv: https://arxiv.org/abs/2406.13165 · HTML: https://mlsanity.com/paper/2406.13165/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [Cross-Modal Adapter for Vision-Language Retrieval](https://mlsanity.com/paper/2211.09623/) (2022)
- [CondenseNet V2: Sparse Feature Reactivation for Deep Networks](https://mlsanity.com/paper/2104.04382/) (2021)
- [Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding](https://mlsanity.com/paper/2203.08481/) (2022)
- [Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound](https://mlsanity.com/paper/2007.04480/) (2020)
- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [Diagnostic Posture Control System for Seated-Style Echocardiography Robot](https://mlsanity.com/paper/2210.13809/) (2022)
- [Multi-Modal Interaction Control of Ultrasound Scanning Robots with Safe Human Guidance and Contact Recovery](https://mlsanity.com/paper/2302.05685/) (2023)

## Built on by (most cited first, 16 of 16)

- [EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance](https://mlsanity.com/paper/2504.13065/) (2025)
- [Surgical Vision World Model](https://mlsanity.com/paper/2503.02904/) (2025)
- [Beyond Generative AI: World Models for Clinical Prediction, Counterfactuals, and Planning](https://mlsanity.com/paper/2511.16333/) (2025)
- [Structure-aware World Model for Probe Guidance via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2406.19756/) (2024)
- [Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies](https://mlsanity.com/paper/2606.16721/) (2026)
- [Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation](https://mlsanity.com/paper/2607.25242/) (2026)
- [UltraSeP: Sequence-aware Pre-training for Echocardiography Probe Movement Guidance](https://mlsanity.com/paper/2408.15026/) (2024)
- [VA-Adapter: Adapting Ultrasound Foundation Model to Echocardiography Probe Guidance](https://mlsanity.com/paper/2510.06809/) (2025)
- [Automated Interpretable 2D Video Extraction from 3D Echocardiography](https://mlsanity.com/paper/2511.15946/) (2025)
- [Intervention-Aware Clinical World Model for Post-Op Outcome Forecasting in Cardiology](https://mlsanity.com/paper/2608.13518/) (2026)
- [UltraStar: Semantic-Aware Star Graph Modeling for Echocardiography Navigation](https://mlsanity.com/paper/2603.01461/) (2026)
- [Anatomical Prior-Driven Framework for Autonomous Robotic Cardiac Ultrasound Standard View Acquisition](https://mlsanity.com/paper/2603.21134/) (2026)
- [Follow Your Heart: Landmark-Guided Transducer Pose Scoring for Point-of-Care Echocardiography](https://mlsanity.com/paper/2603.27143/) (2026)
- [3D MRI Image Pretraining via Controllable 2D Slice Navigation Task](https://mlsanity.com/paper/2605.06487/) (2026)
- [Ultrasound Operator Guidance Using World Modeling and Retrieval Based Action Planning](https://mlsanity.com/paper/2610.06008/) (2026)
- [SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models](https://mlsanity.com/paper/2610.19327/) (2026)

## Abstract (arXiv)

Echocardiography is the only technique capable of real-time imaging of the heart and is vital for diagnosing the majority of cardiac diseases. However, there is a severe shortage of experienced cardiac sonographers, due to the heart's complex structure and significant operational challenges. To mitigate this situation, we present a Cardiac Copilot system capable of providing real-time probe movement guidance to assist less experienced sonographers in conducting freehand echocardiography. This system can enable non-experts, especially in primary departments and medically underserved areas, to perform cardiac ultrasound examinations, potentially improving global healthcare delivery. The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures. This world model can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization. We train our model with real-world ultrasound data and corresponding probe motion from 110 routine clinical scans with 151K sample pairs by three certified sonographers. Evaluations on three standard planes with 37K sample pairs demonstrate that the world model can reduce navigation errors by up to 33% and exhibit more stable performance.