# Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback

arXiv: https://arxiv.org/abs/2409.02337 · HTML: https://mlsanity.com/paper/2409.02337/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [End-to-End Robotic Reinforcement Learning without Reward Engineering](https://mlsanity.com/paper/1904.07854/) (2019)
- [Batch Active Preference-Based Learning of Reward Functions](https://mlsanity.com/paper/1810.04303/) (2018)
- [Shared Autonomy via Hindsight Optimization for Teleoperation and Teaming](https://mlsanity.com/paper/1706.00155/) (2017)
- [Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives](https://mlsanity.com/paper/2307.05545/) (2023)
- [Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study](https://mlsanity.com/paper/1607.08371/) (2016)
- [Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.00718/) (2021)
- [Learning Human Objectives from Sequences of Physical Corrections](https://mlsanity.com/paper/2104.00078/) (2021)
- [Quantifying Hypothesis Space Misspecification in Learning from Human-Robot Demonstrations and Physical Corrections](https://mlsanity.com/paper/2002.00941/) (2020)
- [Physical Interaction as Communication: Learning Robot Objectives Online from Human Corrections](https://mlsanity.com/paper/2107.02349/) (2021)
- [Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations](https://mlsanity.com/paper/2307.03705/) (2023)
- [Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback](https://mlsanity.com/paper/2310.20608/) (2023)
- [POMDP-lite for Robust Robot Planning under Uncertainty](https://mlsanity.com/paper/1602.04875/) (2016)
- [Robotic Sonographer: Autonomous Robotic Ultrasound using Domain Expertise in Bayesian Optimization](https://mlsanity.com/paper/2307.02442/) (2023)
- [Extending Policy from One-Shot Learning through Coaching](https://mlsanity.com/paper/1905.04841/) (2019)
- [Learning rewards for robotic ultrasound scanning using probabilistic temporal ranking](https://mlsanity.com/paper/2002.01240/) (2020)
- [Expert-Agnostic Ultrasound Image Quality Assessment using Deep Variational Clustering](https://mlsanity.com/paper/2307.02462/) (2023)
- [RUSOpt: Robotic UltraSound Probe Normalization with Bayesian Optimization for In-plane and Out-plane Scanning](https://mlsanity.com/paper/2310.03406/) (2023)
- [Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization](https://mlsanity.com/paper/2310.07392/) (2023)

## Built on by (most cited first, 2 of 2)

- [Robotic Ultrasound-Guided Femoral Artery Reconstruction of Anatomically-Representative Phantoms](https://mlsanity.com/paper/2503.06795/) (2025)
- [Point-Wise Geometry-Aware Transformer for Partial-to-Full Point Cloud Registration in Computer-Assisted Surgery](https://mlsanity.com/paper/2606.13488/) (2026)

## Abstract (arXiv)

Ultrasound is widely employed for clinical intervention and diagnosis, due to its advantages of offering non-invasive, radiation-free, and real-time imaging. However, the accessibility of this dexterous procedure is limited due to the substantial training and expertise required of operators. The robotic ultrasound (RUS) offers a viable solution to address this limitation; nonetheless, achieving human-level proficiency remains challenging. Learning from demonstrations (LfD) methods have been explored in RUS, which learns the policy prior from a dataset of offline demonstrations to encode the mental model of the expert sonographer. However, active engagement of experts, i.e. Coaching, during the training of RUS has not been explored thus far. Coaching is known for enhancing efficiency and performance in human training. This paper proposes a coaching framework for RUS to amplify its performance. The framework combines DRL (self-supervised practice) with sparse expert's feedback through coaching. The DRL employs an off-policy Soft Actor-Critic (SAC) network, with a reward based on image quality rating. The coaching by experts is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert. The validation study on phantoms showed that coaching increases the learning rate by 25% and the number of high-quality image acquisition by 74.5%.