mlsanityData to 7 Oct 2026

Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot

arXiv 2302.09772 ↗First posted 20 Feb 2023cs.RO cs.AI cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Task automation of surgical robot has the potentials to improve surgical efficiency. Recent reinforcement learning (RL) based approaches provide scalable solutions to surgical automation, but typically require extensive data collection to solve a task if no prior knowledge is given. This issue is known as the exploration challenge, which can be alleviated by providing expert demonstrations to an RL agent. Yet, how to make effective use of demonstration data to improve exploration efficiency still remains an open challenge. In this work, we introduce Demonstration-guided EXploration (DEX), an efficient reinforcement learning algorithm that aims to overcome the exploration problem with expert demonstrations for surgical automation. To effectively exploit demonstrations, our method estimates expert-like behaviors with higher values to facilitate productive interactions, and adopts non-parametric regression to enable such guidance at states unobserved in demonstration data. Extensive experiments on 10 surgical manipulation tasks from SurRoL, a comprehensive surgical simulation platform, demonstrate significant improvements in the exploration efficiency and task success rates of our method. Moreover, we also deploy the learned policies to the da Vinci Research Kit (dVRK) platform to show the effectiveness on the real robot. Code is available at https://github.com/med-air/DEX.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 39 of its 70 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

39 of 70 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  12. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
27 more
  1. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  2. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  4. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  5. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  6. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  7. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2019Goal-conditioned Imitation Learningcited by 159
  10. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  11. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
  12. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  13. 2021RRL: Resnet as representation for Reinforcement Learningcited by 89
  14. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  15. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  16. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  17. 2021SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learningcited by 28
  18. 2021Safe Reinforcement Learning using Formal Verification for Tissue Retraction in Autonomous Robotic-Assisted Surgerycited by 22
  19. 2020Bimanual Regrasping for Suture Needles using Reinforcement Learning for Rapid Motion Planningcited by 19
  20. 2022On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrationscited by 18
  21. 2017Robot Autonomy for Surgerycited by 16
  22. 2021Learning from Demonstrations for Autonomous Soft-tissue Retractioncited by 15
  23. 2019Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgerycited by 11
  24. 2021Robotic Surgery With Lean Reinforcement Learningcited by 11
  25. 2021Cooperative Assistance in Robotic Surgery through Multi-Agent Reinforcement Learningcited by 10
  26. 2019A New Tensioning Method using Deep Reinforcement Learning for Surgical Pattern Cuttingcited by 8
  27. 2017Using Intermittent Synchronization to Compensate for Rhythmic Body Motion During Autonomous Surgical Cutting and Debridementcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 9; 2025: 2; 2026: 4.

21 papers.

Sort:

By area:

  1. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  2. 2024General-purpose foundation models for increased autonomy in robot-assisted surgerycited by 15
  3. 2024World Models for General Surgical Graspingcited by 9
  4. 2023Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robotcited by 8
  5. 2023Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robotscited by 5
  6. 2024Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automationcited by 4
  7. 2023Autonomous Soft Tissue Retraction Using Demonstration-Guided Reinforcement Learningcited by 3
  8. 2023Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environmentscited by 3
  9. 2024SimEndoGS: Efficient Data-driven Scene Simulation using Robotic Surgery Videos via Physics-embedded 3D Gaussianscited by 3
  10. 2024Rethinking RGB-D Fusion for Semantic Segmentation in Surgical Datasetscited by 2
  11. 2024Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learningcited by 2
  12. 2023Teach Me How to Learn: A Perspective Review towards User-centered Neuro-symbolic Learning for Robotic Surgical Systemscited by 1
9 more
  1. 2023Signal Temporal Logic-Guided Apprenticeship Learningcited by 1
  2. 2024Realistic Surgical Simulation from Monocular Videoscited by 1
  3. 2025dARt Vinci: Egocentric Data Collection for Surgical Robot Learning at Scalecited by 1
  4. 2025Visuomotor Grasping with World Models for Surgical Robotscited by 1
  5. 2024SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learningcited by 0
  6. 2026Boosting deep Reinforcement Learning using pretraining with Logical Optionscited by 0
  7. 2026Match or Replay: Self Imitating Proximal Policy Optimizationcited by 0
  8. 2026E2DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulationcited by 0
  9. 2026SurgVIL: Scaling Surgical Robot Imitation Learning with Open-source Surgical Videoscited by 0