Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for clinical treatment and surgery
Abstract (arXiv)
The irrigation-suction process is a common procedure to rinse and clean up the surgical field in minimally invasive surgery (MIS). In this process, surgeons first irrigate liquid, typically saline, into the surgical scene for rinsing and diluting the contaminant, and then suction the liquid out of the surgical field. While recent advances have shown promising results in the application of reinforcement learning (RL) for automating surgical subtasks, fewer studies have explored the automation of fluid-related tasks. In this work, we explore the automation of both steps in the irrigation-suction procedure and train two vision-based RL agents to complete irrigation and suction autonomously. To achieve this, a platform is developed for creating simulated surgical robot learning environments and for training agents, and two simulated learning environments are built for irrigation and suction with visually plausible fluid rendering capabilities. With techniques such as domain randomization (DR) and carefully designed reward functions, two agents are trained in the simulator and transferred to the real world. Individual evaluations of both agents show satisfactory real-world results. With an initial amount of around 5 grams of contaminants, the irrigation agent ultimately achieved an average of 2.21 grams remaining after a manual suction. As a comparison, fully manual operation by a human results in 1.90 grams remaining. The suction agent achieved 2.64 and 2.24 grams of liquid remaining across two trial groups with more than 20 and 30 grams of initial liquid in the container. Fully autonomous irrigation-suction trials reduce the contaminant in the container from around 5 grams to an average of 2.42 grams, although yielding a higher total weight remaining (4.40) due to residual liquid not suctioned. Further information about the project is available at https://tbs-ualberta.github.io/CRESSim/.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 54 references that are in the map. First posted on arXiv 21 Nov 2024.
Builds on
24 of 54 references.
References · 24
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
- 2018Unity: A General Platform for Intelligent Agentscited by 236
- 2023FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulationcited by 40
- 2021SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learningcited by 31
- 2019Open-Sourced Reinforcement Learning Environments for Surgical Roboticscited by 26
- 2024Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgerycited by 26
- 2020Bimanual Regrasping for Suture Needles using Reinforcement Learning for Rapid Motion Planningcited by 21
- 2021Learning from Demonstrations for Autonomous Soft-tissue Retractioncited by 18
- 2024ORBIT-Surgical: An Open-Simulation Framework for Learning Surgical Augmented Dexteritycited by 16
12 more
- 2023LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgerycited by 14
- 2023Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learningcited by 13
- 2019Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgerycited by 12
- 2020Autonomous Robotic Suction to Clear the Surgical Field for Hemostasis using Image-based Blood Flow Detectioncited by 12
- 20223D Perception based Imitation Learning under Limited Demonstration for Laparoscope Control in Robotic Surgerycited by 10
- 2019A New Tensioning Method using Deep Reinforcement Learning for Surgical Pattern Cuttingcited by 9
- 2023Sim-to-Real Surgical Robot Learning and Autonomous Planning for Internal Tissue Points Manipulation using Reinforcement Learningcited by 9
- 2024From Decision to Action in Surgical Autonomy: Multi-Modal Large Language Models for Robot-Assisted Blood Suctioncited by 8
- 2021Model-Predictive Control of Blood Suction for Surgical Hemostasis using Differentiable Fluid Simulationscited by 6
- 2023Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robotscited by 6
- 2022Automating Vascular Shunt Insertion with the dVRK Surgical Robotcited by 2
- 2024A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kitcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024SurgicAI: A Hierarchical Platform for Fine-Grained Surgical Policy Learning and Benchmarking
- 2024Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learning
- 2024SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learning
- 2025DARIL: When Imitation Learning outperforms Reinforcement Learning in Surgical Action Planning
- 2023Toward a Surgeon-in-the-Loop Ophthalmic Robotic Apprentice using Reinforcement and Imitation Learning
- 2019Autonomous Tissue Manipulation via Surgical Robot Using Learning Based Model Predictive Control
- 2024Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery
- 2026Simulation of Surgical Suturing Using Position-Based Dynamics and the Material Point Method for Robot Reinforcement Learning ↗