# Goal-conditioned reinforcement learning for ultrasound navigation guidance

arXiv: https://arxiv.org/abs/2405.01409 · HTML: https://mlsanity.com/paper/2405.01409/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Contrastive Learning as Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2206.07568/) (2022)
- [RLlib: Abstractions for Distributed Reinforcement Learning](https://mlsanity.com/paper/1712.09381/) (2017)
- [Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound](https://mlsanity.com/paper/2007.04480/) (2020)
- [Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.00718/) (2021)
- [Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.13321/) (2020)
- [VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation](https://mlsanity.com/paper/2205.06676/) (2022)
- [Straight to the point: reinforcement learning for user guidance in ultrasound](https://mlsanity.com/paper/1903.00586/) (2019)
- [Cardiac ultrasound simulation for autonomous ultrasound navigation](https://mlsanity.com/paper/2402.06463/) (2024)

## Built on by (most cited first, 6 of 6)

- [EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance](https://mlsanity.com/paper/2504.13065/) (2025)
- [UltraSeP: Sequence-aware Pre-training for Echocardiography Probe Movement Guidance](https://mlsanity.com/paper/2408.15026/) (2024)
- [AI-driven View Guidance System in Intra-cardiac Echocardiography Imaging](https://mlsanity.com/paper/2409.16898/) (2024)
- [VA-Adapter: Adapting Ultrasound Foundation Model to Echocardiography Probe Guidance](https://mlsanity.com/paper/2510.06809/) (2025)
- [Robotic Ultrasound-Guided Femoral Artery Reconstruction of Anatomically-Representative Phantoms](https://mlsanity.com/paper/2503.06795/) (2025)
- [End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning](https://mlsanity.com/paper/2511.00114/) (2025)

## Abstract (arXiv)

Transesophageal echocardiography (TEE) plays a pivotal role in cardiology for diagnostic and interventional procedures. However, using it effectively requires extensive training due to the intricate nature of image acquisition and interpretation. To enhance the efficiency of novice sonographers and reduce variability in scan acquisitions, we propose a novel ultrasound (US) navigation assistance method based on contrastive learning as goal-conditioned reinforcement learning (GCRL). We augment the previous framework using a novel contrastive patient batching method (CPB) and a data-augmented contrastive loss, both of which we demonstrate are essential to ensure generalization to anatomical variations across patients. The proposed framework enables navigation to both standard diagnostic as well as intricate interventional views with a single model. Our method was developed with a large dataset of 789 patients and obtained an average error of 6.56 mm in position and 9.36 degrees in angle on a testing dataset of 140 patients, which is competitive or superior to models trained on individual views. Furthermore, we quantitatively validate our method's ability to navigate to interventional views such as the Left Atrial Appendage (LAA) view used in LAA closure. Our approach holds promise in providing valuable guidance during transesophageal ultrasound examinations, contributing to the advancement of skill acquisition for cardiac ultrasound practitioners.