# SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

arXiv: https://arxiv.org/abs/2108.13035 · HTML: https://mlsanity.com/paper/2108.13035/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016)
- [Benchmarking Reinforcement Learning Algorithms on Real-World Robots](https://mlsanity.com/paper/1809.07731/) (2018)
- [SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics](https://mlsanity.com/paper/1909.05405/) (2019)
- [Open-Sourced Reinforcement Learning Environments for Surgical Robotics](https://mlsanity.com/paper/1903.02090/) (2019)
- [Robot Autonomy for Surgery](https://mlsanity.com/paper/1707.03080/) (2017)
- [Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot](https://mlsanity.com/paper/2002.06302/) (2020)

## Built on by (most cited first, 30 of 31)

- [SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning](https://mlsanity.com/paper/2505.10251/) (2025)
- [Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot](https://mlsanity.com/paper/2302.09772/) (2023)
- [Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learning](https://mlsanity.com/paper/2301.00452/) (2023)
- [3D Perception based Imitation Learning under Limited Demonstration for Laparoscope Control in Robotic Surgery](https://mlsanity.com/paper/2204.03195/) (2022)
- [Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Clouds](https://mlsanity.com/paper/2110.04685/) (2021)
- [Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robot](https://mlsanity.com/paper/2307.16503/) (2023)
- [Creating a Digital Twin of Spinal Surgery: A Proof of Concept](https://mlsanity.com/paper/2403.16736/) (2024)
- [World Models for General Surgical Grasping](https://mlsanity.com/paper/2405.17940/) (2024)
- [Integrating Artificial Intelligence and Augmented Reality in Robotic Surgery: An Initial dVRK Study Using a Surgical Education Scenario](https://mlsanity.com/paper/2201.00383/) (2022)
- [Colonoscopy Navigation using End-to-End Deep Visuomotor Control: A User Study](https://mlsanity.com/paper/2206.15086/) (2022)
- [Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robots](https://mlsanity.com/paper/2310.04676/) (2023)
- [DeformerNet: Learning Bimanual Manipulation of 3D Deformable Objects](https://mlsanity.com/paper/2305.04449/) (2023)
- [Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environments](https://mlsanity.com/paper/2310.08841/) (2023)
- [Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automation](https://mlsanity.com/paper/2405.18757/) (2024)
- [Enhanced Scale-aware Depth Estimation for Monocular Endoscopic Scenes with Geometric Modeling](https://mlsanity.com/paper/2408.07266/) (2024)
- [Autonomous Soft Tissue Retraction Using Demonstration-Guided Reinforcement Learning](https://mlsanity.com/paper/2309.00837/) (2023)
- [Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learning](https://mlsanity.com/paper/2409.02724/) (2024)
- [FF-SRL: High Performance GPU-Based Surgical Simulation For Robot Learning](https://mlsanity.com/paper/2503.18616/) (2025)
- [Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures: A Systematic Review of AI Solutions](https://mlsanity.com/paper/2504.15327/) (2025)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [Teach Me How to Learn: A Perspective Review towards User-centered Neuro-symbolic Learning for Robotic Surgical Systems](https://mlsanity.com/paper/2307.03853/) (2023)
- [Signal Temporal Logic-Guided Apprenticeship Learning](https://mlsanity.com/paper/2311.05084/) (2023)
- [Toward a Surgeon-in-the-Loop Ophthalmic Robotic Apprentice using Reinforcement and Imitation Learning](https://mlsanity.com/paper/2311.17693/) (2023)
- [Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning](https://mlsanity.com/paper/2411.14622/) (2024)
- [dARt Vinci: Egocentric Data Collection for Surgical Robot Learning at Scale](https://mlsanity.com/paper/2503.05646/) (2025)
- [Visuomotor Grasping with World Models for Surgical Robots](https://mlsanity.com/paper/2508.11200/) (2025)
- [Embedded Image-to-Image Translation for Efficient Sim-to-Real Transfer in Learning-based Robot-Assisted Soft Manipulation](https://mlsanity.com/paper/2409.10204/) (2024)
- [SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learning](https://mlsanity.com/paper/2409.15651/) (2024)
- [SSP: Safety-guaranteed Surgical Policy via Joint Optimization of Behavioral and Spatial Constraints](https://mlsanity.com/paper/2603.07032/) (2026)
- [Imitation Learning for Robot Assistance in Open Surgery: A Multi-Policy Evaluation on Suture Following](https://mlsanity.com/paper/2605.28736/) (2026)

## Abstract (arXiv)

Autonomous surgical execution relieves tedious routines and surgeon's fatigue. Recent learning-based methods, especially reinforcement learning (RL) based methods, achieve promising performance for dexterous manipulation, which usually requires the simulation to collect data efficiently and reduce the hardware cost. The existing learning-based simulation platforms for medical robots suffer from limited scenarios and simplified physical interactions, which degrades the real-world performance of learned policies. In this work, we designed SurRoL, an RL-centered simulation platform for surgical robot learning compatible with the da Vinci Research Kit (dVRK). The designed SurRoL integrates a user-friendly RL library for algorithm development and a real-time physics engine, which is able to support more PSM/ECM scenarios and more realistic physical interactions. Ten learning-based surgical tasks are built in the platform, which are common in the real autonomous surgical execution. We evaluate SurRoL using RL algorithms in simulation, provide in-depth analysis, deploy the trained policies on the real dVRK, and show that our SurRoL achieves better transferability in the real world.