# Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.00718 · HTML: https://mlsanity.com/paper/2103.00718/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound](https://mlsanity.com/paper/1612.05601/) (2016)
- [Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study](https://mlsanity.com/paper/1607.08371/) (2016)
- [Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound](https://mlsanity.com/paper/2007.04480/) (2020)
- [Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1806.03228/) (2018)
- [Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.13321/) (2020)
- [Standard Plane Detection in 3D Fetal Ultrasound Using an Iterative Transformation Network](https://mlsanity.com/paper/1806.07486/) (2018)
- [Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound](https://mlsanity.com/paper/1910.04331/) (2019)
- [Straight to the point: reinforcement learning for user guidance in ultrasound](https://mlsanity.com/paper/1903.00586/) (2019)
- [Design and integration of a parallel, soft robotic end-effector for extracorporeal ultrasound](https://mlsanity.com/paper/1906.04526/) (2019)

## Built on by (most cited first, 21 of 21)

- [Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations](https://mlsanity.com/paper/2307.03705/) (2023)
- [VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation](https://mlsanity.com/paper/2205.06676/) (2022)
- [Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework](https://mlsanity.com/paper/2111.02167/) (2021)
- [Adaptive 3D Localization of 2D Freehand Ultrasound Brain Images](https://mlsanity.com/paper/2209.05477/) (2022)
- [Goal-conditioned reinforcement learning for ultrasound navigation guidance](https://mlsanity.com/paper/2405.01409/) (2024)
- [Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided Explorations](https://mlsanity.com/paper/2111.01625/) (2021)
- [Agent with Tangent-based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound](https://mlsanity.com/paper/2207.00475/) (2022)
- [Multimodal-GuideNet: Gaze-Probe Bidirectional Guidance in Obstetric Ultrasound Scanning](https://mlsanity.com/paper/2207.12833/) (2022)
- [Ultrasound Plane Pose Regression: Assessing Generalized Pose Coordinates in the Fetal Brain](https://mlsanity.com/paper/2301.08317/) (2023)
- [Next-Gen Machine Learning Supported Diagnostic Systems for Spacecraft](https://mlsanity.com/paper/2106.05659/) (2021)
- [Learning Ultrasound Scanning Skills from Human Demonstrations](https://mlsanity.com/paper/2111.09739/) (2021)
- [Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback](https://mlsanity.com/paper/2409.02337/) (2024)
- [Automatic Recognition of Abdominal Organs in Ultrasound Images based on Deep Neural Networks and K-Nearest-Neighbor Classification](https://mlsanity.com/paper/2110.04563/) (2021)
- [Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning](https://mlsanity.com/paper/2110.04564/) (2021)
- [Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization](https://mlsanity.com/paper/2310.07392/) (2023)
- [Automatic facial axes standardization of 3D fetal ultrasound images](https://mlsanity.com/paper/2409.02826/) (2024)
- [Learning Autonomous Ultrasound via Latent Task Representation and Robotic Skills Adaptation](https://mlsanity.com/paper/2307.13323/) (2023)
- [End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning](https://mlsanity.com/paper/2511.00114/) (2025)
- [Follow Your Heart: Landmark-Guided Transducer Pose Scoring for Point-of-Care Echocardiography](https://mlsanity.com/paper/2603.27143/) (2026)
- [SAMe: A Semantic Anatomy Mapping Engine for Robotic Ultrasound](https://mlsanity.com/paper/2604.25646/) (2026)
- [Scanning While Imagining: A Scene-Graph World Model for Robotic Ultrasound Navigation](https://mlsanity.com/paper/2609.32837/) (2026)

## Abstract (arXiv)

Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans. Furthermore, we propose a confidence-based approach to encode the optimization of image quality in the learning process. We validate our method in a simulation environment built with real-world data collected in the US imaging of the spine. Experimental results demonstrate that our method can perform reproducible US probe navigation towards the standard scan plane with an accuracy of 4.91mm/4.65^circ in the intra-patient setting, and accomplish the task in the intra- and inter-patient settings with a success rate of 92% and 46%, respectively. The results also show that the introduction of image quality optimization in our method can effectively improve the navigation performance.