# Safe Navigation for Robotic Digestive Endoscopy via Human Intervention-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2409.15688 · HTML: https://mlsanity.com/paper/2409.15688/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for clinical treatment and surgery](https://mlsanity.com/deeprl/rl-for-clinical-treatment-and-surgery/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Trial without Error: Towards Safe Reinforcement Learning via Human Intervention](https://mlsanity.com/paper/1707.05173/) (2017)
- [VR-Caps: A Virtual Environment for Capsule Endoscopy](https://mlsanity.com/paper/2008.12949/) (2020)
- [ORBSLAM-based Endoscope Tracking and 3D Reconstruction](https://mlsanity.com/paper/1608.08149/) (2016)
- [Intervention Aided Reinforcement Learning for Safe and Practical Policy Optimization in Navigation](https://mlsanity.com/paper/1811.06187/) (2018)
- [Constrained Reinforcement Learning and Formal Verification for Safe Colonoscopy Navigation](https://mlsanity.com/paper/2303.03207/) (2023)
- [Deep Reinforcement Learning-Based Control for Stomach Coverage Scanning of Wireless Capsule Endoscopy](https://mlsanity.com/paper/2305.10955/) (2023)
- [Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review](https://mlsanity.com/paper/2305.04027/) (2023)
- [Colonoscopy Navigation using End-to-End Deep Visuomotor Control: A User Study](https://mlsanity.com/paper/2206.15086/) (2022)
- [Autonomous Intraluminal Navigation of a Soft Robot using Deep-Learning-based Visual Servoing](https://mlsanity.com/paper/2207.00401/) (2022)

## Built on by (most cited first, 1 of 1)

- [Curriculum-Based Reinforcement Learning for Autonomous UAV Navigation in Unknown Curved Tubular Conduit](https://mlsanity.com/paper/2512.10934/) (2025)

## Abstract (arXiv)

With the increasing application of automated robotic digestive endoscopy (RDE), ensuring safe and efficient navigation in the unstructured and narrow digestive tract has become a critical challenge. Existing automated reinforcement learning navigation algorithms often result in potentially risky collisions due to the absence of essential human intervention, which significantly limits the safety and effectiveness of RDE in actual clinical practice. To address this limitation, we proposed a Human Intervention (HI)-based Proximal Policy Optimization (PPO) framework, dubbed HI-PPO, which incorporates expert knowledge to enhance RDE's safety. Specifically, HI-PPO combines Enhanced Exploration Mechanism (EEM), Reward-Penalty Adjustment (RPA), and Behavior Cloning Similarity (BCS) to address PPO's exploration inefficiencies for safe navigation in complex gastrointestinal environments. Comparative experiments were conducted on a simulation platform, and the results showed that HI-PPO achieved a mean ATE (Average Trajectory Error) of (8.02 mm) and a Security Score of (0.862), demonstrating performance comparable to human experts. The code will be publicly available once this paper is published.