# Autonomous Path Planning for Intercostal Robotic Ultrasound Imaging Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2404.09927 · HTML: https://mlsanity.com/paper/2404.09927/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives](https://mlsanity.com/paper/2307.05545/) (2023)
- [Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study](https://mlsanity.com/paper/1607.08371/) (2016)
- [Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.13321/) (2020)
- [Machine Learning in Robotic Ultrasound Imaging: Challenges and Perspectives](https://mlsanity.com/paper/2401.02376/) (2024)
- [Deformation-Aware Robotic 3D Ultrasound](https://mlsanity.com/paper/2107.08411/) (2021)
- [Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations](https://mlsanity.com/paper/2307.03705/) (2023)
- [VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation](https://mlsanity.com/paper/2205.06676/) (2022)
- [Towards Autonomous Atlas-based Ultrasound Acquisitions in Presence of Articulated Motion](https://mlsanity.com/paper/2208.05399/) (2022)
- [Ultrasound Video Summarization using Deep Reinforcement Learning](https://mlsanity.com/paper/2005.09531/) (2020)
- [Skeleton Graph-based Ultrasound-CT Non-rigid Registration](https://mlsanity.com/paper/2305.08228/) (2023)
- [Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound](https://mlsanity.com/paper/2105.10626/) (2021)
- [Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework](https://mlsanity.com/paper/2111.02167/) (2021)
- [Implicit Neural Representations for Breathing-compensated Volume Reconstruction in Robotic Ultrasound](https://mlsanity.com/paper/2311.04999/) (2023)
- [Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided Explorations](https://mlsanity.com/paper/2111.01625/) (2021)
- [Thoracic Cartilage Ultrasound-CT Registration using Dense Skeleton Graph](https://mlsanity.com/paper/2307.03800/) (2023)
- [Towards Safe and Collaborative Robotic Ultrasound Tissue Scanning in Neurosurgery](https://mlsanity.com/paper/2312.08409/) (2023)

## Built on by (most cited first, 3 of 3)

- [Class-Aware Cartilage Segmentation for Autonomous US-CT Registration in Robotic Intercostal Ultrasound Imaging](https://mlsanity.com/paper/2406.04100/) (2024)
- [Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies](https://mlsanity.com/paper/2606.16721/) (2026)
- [End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning](https://mlsanity.com/paper/2511.00114/) (2025)

## Abstract (arXiv)

Ultrasound (US) has been widely used in daily clinical practice for screening internal organs and guiding interventions. However, due to the acoustic shadow cast by the subcutaneous rib cage, the US examination for thoracic application is still challenging. To fully cover and reconstruct the region of interest in US for diagnosis, an intercostal scanning path is necessary. To tackle this challenge, we present a reinforcement learning (RL) approach for planning scanning paths between ribs to monitor changes in lesions on internal organs, such as the liver and heart, which are covered by rib cages. Structured anatomical information of the human skeleton is crucial for planning these intercostal paths. To obtain such anatomical insight, an RL agent is trained in a virtual environment constructed using computational tomography (CT) templates with randomly initialized tumors of various shapes and locations. In addition, task-specific state representation and reward functions are introduced to ensure the convergence of the training process while minimizing the effects of acoustic attenuation and shadows during scanning. To validate the effectiveness of the proposed approach, experiments have been carried out on unseen CTs with randomly defined single or multiple scanning targets. The results demonstrate the efficiency of the proposed RL framework in planning non-shadowed US scanning trajectories in areas with limited acoustic access.