# Deep reinforcement learning for guidewire navigation in coronary artery phantom

arXiv: https://arxiv.org/abs/2110.01840 · HTML: https://mlsanity.com/paper/2110.01840/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Noisy Networks for Exploration](https://mlsanity.com/paper/1706.10295/) (2017)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Robust Imitation of Diverse Behaviors](https://mlsanity.com/paper/1707.02747/) (2017)
- [Reinforcement Learning from Imperfect Demonstrations](https://mlsanity.com/paper/1802.05313/) (2018)
- [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017)
- [Dynamic Coronary Roadmapping via Catheter Tip Tracking in X-ray Fluoroscopy with Deep Learning Based Bayesian Filtering](https://mlsanity.com/paper/2001.03801/) (2020)
- [Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard](https://mlsanity.com/paper/2008.02646/) (2020)

## Built on by (most cited first, 10 of 10)

- [Artificial Intelligence in the Autonomous Navigation of Endovascular Interventions: A Systematic Review](https://mlsanity.com/paper/2405.03305/) (2024)
- [Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning](https://mlsanity.com/paper/2406.12499/) (2024)
- [Reinforcement Learning for Safe Autonomous Two Device Navigation of Cerebral Vessels in Mechanical Thrombectomy](https://mlsanity.com/paper/2503.24140/) (2025)
- [Autonomous Catheterization with Open-source Simulator and Expert Trajectory](https://mlsanity.com/paper/2401.09059/) (2024)
- [Robust Path Planning via Learning from Demonstrations for Robotic Catheters in Deformable Environments](https://mlsanity.com/paper/2402.00537/) (2024)
- [BronchoCopilot: Towards Autonomous Robotic Bronchoscopy via Multimodal Reinforcement Learning](https://mlsanity.com/paper/2403.01483/) (2024)
- [A Zero-Shot Reinforcement Learning Strategy for Autonomous Guidewire Navigation](https://mlsanity.com/paper/2403.02777/) (2024)
- [Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures: A Systematic Review of AI Solutions](https://mlsanity.com/paper/2504.15327/) (2025)
- [SplineFormer: An Explainable Transformer-Based Approach for Autonomous Endovascular Navigation](https://mlsanity.com/paper/2501.04515/) (2025)
- [DINO-CVA: A Multimodal Goal-Conditioned Vision-to-Action Model for Autonomous Catheter Navigation](https://mlsanity.com/paper/2510.17038/) (2025)

## Abstract (arXiv)

In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning(RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations with deep Q-learning from demonstrations (DQfD), transfer learning, and weight initialization. `State' for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in `stable' phase. Our framework opens anew direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.