# Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2411.14622 · HTML: https://mlsanity.com/paper/2411.14622/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [RL for clinical treatment and surgery](https://mlsanity.com/deeprl/rl-for-clinical-treatment-and-surgery/)

## Builds on (most cited first, 24 of 24)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation](https://mlsanity.com/paper/2303.02346/) (2023)
- [SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning](https://mlsanity.com/paper/2108.13035/) (2021)
- [Open-Sourced Reinforcement Learning Environments for Surgical Robotics](https://mlsanity.com/paper/1903.02090/) (2019)
- [Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery](https://mlsanity.com/paper/2406.06092/) (2024)
- [Bimanual Regrasping for Suture Needles using Reinforcement Learning for Rapid Motion Planning](https://mlsanity.com/paper/2011.04813/) (2020)
- [Learning from Demonstrations for Autonomous Soft-tissue Retraction](https://mlsanity.com/paper/2110.00336/) (2021)
- [ORBIT-Surgical: An Open-Simulation Framework for Learning Surgical Augmented Dexterity](https://mlsanity.com/paper/2404.16027/) (2024)
- [LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery](https://mlsanity.com/paper/2302.09606/) (2023)
- [Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learning](https://mlsanity.com/paper/2301.00452/) (2023)
- [Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgery](https://mlsanity.com/paper/1902.05183/) (2019)
- [Autonomous Robotic Suction to Clear the Surgical Field for Hemostasis using Image-based Blood Flow Detection](https://mlsanity.com/paper/2010.08441/) (2020)
- [3D Perception based Imitation Learning under Limited Demonstration for Laparoscope Control in Robotic Surgery](https://mlsanity.com/paper/2204.03195/) (2022)
- [A New Tensioning Method using Deep Reinforcement Learning for Surgical Pattern Cutting](https://mlsanity.com/paper/1901.03327/) (2019)
- [Sim-to-Real Surgical Robot Learning and Autonomous Planning for Internal Tissue Points Manipulation using Reinforcement Learning](https://mlsanity.com/paper/2306.14085/) (2023)
- [From Decision to Action in Surgical Autonomy: Multi-Modal Large Language Models for Robot-Assisted Blood Suction](https://mlsanity.com/paper/2408.07806/) (2024)
- [Model-Predictive Control of Blood Suction for Surgical Hemostasis using Differentiable Fluid Simulations](https://mlsanity.com/paper/2102.01436/) (2021)
- [Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robots](https://mlsanity.com/paper/2310.04676/) (2023)
- [Automating Vascular Shunt Insertion with the dVRK Surgical Robot](https://mlsanity.com/paper/2211.02293/) (2022)
- [A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kit](https://mlsanity.com/paper/2404.05888/) (2024)

## Built on by (most cited first, 1 of 1)

- [BleedOrigin: Dynamic Bleeding Source Localization in Endoscopic Submucosal Dissection via Dual-Stage Detection and Tracking](https://mlsanity.com/paper/2507.15094/) (2025)

## Abstract (arXiv)

The irrigation-suction process is a common procedure to rinse and clean up the surgical field in minimally invasive surgery (MIS). In this process, surgeons first irrigate liquid, typically saline, into the surgical scene for rinsing and diluting the contaminant, and then suction the liquid out of the surgical field. While recent advances have shown promising results in the application of reinforcement learning (RL) for automating surgical subtasks, fewer studies have explored the automation of fluid-related tasks. In this work, we explore the automation of both steps in the irrigation-suction procedure and train two vision-based RL agents to complete irrigation and suction autonomously. To achieve this, a platform is developed for creating simulated surgical robot learning environments and for training agents, and two simulated learning environments are built for irrigation and suction with visually plausible fluid rendering capabilities. With techniques such as domain randomization (DR) and carefully designed reward functions, two agents are trained in the simulator and transferred to the real world. Individual evaluations of both agents show satisfactory real-world results. With an initial amount of around 5 grams of contaminants, the irrigation agent ultimately achieved an average of 2.21 grams remaining after a manual suction. As a comparison, fully manual operation by a human results in 1.90 grams remaining. The suction agent achieved 2.64 and 2.24 grams of liquid remaining across two trial groups with more than 20 and 30 grams of initial liquid in the container. Fully autonomous irrigation-suction trials reduce the contaminant in the container from around 5 grams to an average of 2.42 grams, although yielding a higher total weight remaining (4.40) due to residual liquid not suctioned. Further information about the project is available at https://tbs-ualberta.github.io/CRESSim/.