mlsanityData to 7 Oct 2026

Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robot

arXiv 2307.16503 ↗First posted 31 Jul 2023cs.RO cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is still struggling with solving long-horizon surgical robot tasks which involve multiple steps over an extended duration of time due to the policy exploration challenge. Recent methods try to tackle this problem by skill chaining, in which the long-horizon task is decomposed into multiple subtasks for easing the exploration burden and subtask policies are temporally connected to complete the whole long-horizon task. However, smoothly connecting all subtask policies is difficult for surgical robot scenarios. Not all states are equally suitable for connecting two adjacent subtasks. An undesired terminate state of the previous subtask would make the current subtask policy unstable and result in a failed execution. In this work, we introduce value-informed skill chaining (ViSkill), a novel reinforcement learning framework for long-horizon surgical robot tasks. The core idea is to distinguish which terminal state is suitable for starting all the following subtask policies. To achieve this target, we introduce a state value function that estimates the expected success probability of the entire task given a state. Based on this value function, a chaining policy is learned to instruct subtask policies to terminate at the state with the highest value so that all subsequent policies are more likely to be connected for accomplishing the task. We demonstrate the effectiveness of our method on three complex surgical robot tasks from SurRoL, a comprehensive surgical simulation platform, achieving high task success rates and execution efficiency. Code is available at hrefhttps://github.com/med-air/ViSkillhttps://github.com/med-air/ViSkill.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 31 Jul 2023.

Builds on

24 of 37 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  4. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  5. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  6. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  7. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  8. 2019Goal-conditioned Imitation Learningcited by 159
  9. 2019Planning with Goal-Conditioned Policiescited by 144
  10. 2018Self-Imitation Learningcited by 135
  11. 2017Divide-and-Conquer Reinforcement Learningcited by 46
  12. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
12 more
  1. 2021SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learningcited by 28
  2. 2021Safe Reinforcement Learning using Formal Verification for Tissue Retraction in Autonomous Robotic-Assisted Surgerycited by 22
  3. 2023Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robotcited by 21
  4. 2017Robot Autonomy for Surgerycited by 16
  5. 2021Learning from Demonstrations for Autonomous Soft-tissue Retractioncited by 15
  6. 2019Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgerycited by 11
  7. 2020Autonomous task planning and situation awareness in robotic surgerycited by 10
  8. 2021Learning to Localize, Grasp, and Hand Over Unmodified Surgical Needlescited by 10
  9. 2019A New Tensioning Method using Deep Reinforcement Learning for Surgical Pattern Cuttingcited by 8
  10. 2022Human-Robot Shared Control for Surgical Robot Based on Context-Aware Sim-to-Real Adaptationcited by 7
  11. 2021Autonomous tissue retraction with a biomechanically informed logic based frameworkcited by 6
  12. 2021Training Transition Policies via Distribution Matching for Complex Taskscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 1.

8 papers.

  1. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  2. 2025From Screens to Scenes: A Survey of Embodied AI in Healthcarecited by 10
  3. 2025SeqVLA: Sequential Task Execution for Long-Horizon Manipulation with Completion-Aware Vision-Language-Action Modelcited by 10
  4. 2024Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automationcited by 4
  5. 2024SimEndoGS: Efficient Data-driven Scene Simulation using Robotic Surgery Videos via Physics-embedded 3D Gaussianscited by 3
  6. 2024Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learningcited by 2
  7. 2025Risk-Sensitive Agent Compositionscited by 1
  8. 2026Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probescited by 0