# Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2003.13321 · HTML: https://mlsanity.com/paper/2003.13321/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 17 of 17)

- [Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.00718/) (2021)
- [Rethinking Ultrasound Augmentation: A Physics-Inspired Approach](https://mlsanity.com/paper/2105.02188/) (2021)
- [Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations](https://mlsanity.com/paper/2307.03705/) (2023)
- [VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation](https://mlsanity.com/paper/2205.06676/) (2022)
- [Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework](https://mlsanity.com/paper/2111.02167/) (2021)
- [Goal-conditioned reinforcement learning for ultrasound navigation guidance](https://mlsanity.com/paper/2405.01409/) (2024)
- [Accelerated Intravascular Ultrasound Imaging using Deep Reinforcement Learning](https://mlsanity.com/paper/2201.09522/) (2022)
- [Cardiac ultrasound simulation for autonomous ultrasound navigation](https://mlsanity.com/paper/2402.06463/) (2024)
- [Pessimistic Model Selection for Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2111.14346/) (2021)
- [Autonomous Path Planning for Intercostal Robotic Ultrasound Imaging Using Reinforcement Learning](https://mlsanity.com/paper/2404.09927/) (2024)
- [Next-Gen Machine Learning Supported Diagnostic Systems for Spacecraft](https://mlsanity.com/paper/2106.05659/) (2021)
- [Shape Completion and Real-Time Visualization in Robotic Ultrasound Spine Acquisitions](https://mlsanity.com/paper/2508.08923/) (2025)
- [End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning](https://mlsanity.com/paper/2511.00114/) (2025)
- [SonoSelect: Efficient Ultrasound Perception via Active Probe Exploration](https://mlsanity.com/paper/2604.05933/) (2026)
- [SAMe: A Semantic Anatomy Mapping Engine for Robotic Ultrasound](https://mlsanity.com/paper/2604.25646/) (2026)
- [Scanning While Imagining: A Scene-Graph World Model for Robotic Ultrasound Navigation](https://mlsanity.com/paper/2609.32837/) (2026)
- [Dual-Rate Force-Image Control with Model-Based Orientation Limits for Robotic Ultrasound](https://mlsanity.com/paper/2610.05839/) (2026)

## Abstract (arXiv)

In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task. Our method is trained and evaluated on an in-house collected data-set of 34 volunteers and when compared to pure RL and supervised learning (SL) techniques, it performs substantially better, which highlights the suitability of RL navigation for US-guided procedures. When testing our proposed model, we obtained a 82.91% chance of navigating correctly to the sacrum from 165 different starting positions on 5 different unseen simulated environments.