# Cooperative Assistance in Robotic Surgery through Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2110.04857 · HTML: https://mlsanity.com/paper/2110.04857/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2021.

## Built on by (most cited first, 11 of 11)

- [Movement Primitive Diffusion: Learning Gentle Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2312.10008/) (2023)
- [Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery](https://mlsanity.com/paper/2406.06092/) (2024)
- [Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot](https://mlsanity.com/paper/2302.09772/) (2023)
- [Colonoscopy Navigation using End-to-End Deep Visuomotor Control: A User Study](https://mlsanity.com/paper/2206.15086/) (2022)
- [Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarm](https://mlsanity.com/paper/2503.03796/) (2025)
- [Skill-Aligned Fairness in Multi-Agent Learning for Collaboration in Healthcare](https://mlsanity.com/paper/2508.18708/) (2025)
- [Renaissance Robot: Optimal Transport Policy Fusion for Learning Diverse Skills](https://mlsanity.com/paper/2207.00978/) (2022)
- [Hiding in Plain Sight: Differential Privacy Noise Exploitation for Evasion-resilient Localized Poisoning Attacks in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2307.00268/) (2023)
- [Robotic Scene Segmentation with Memory Network for Runtime Surgical Context Inference](https://mlsanity.com/paper/2308.12789/) (2023)
- [On Diagnostics for Understanding Agent Training Behaviour in Cooperative MARL](https://mlsanity.com/paper/2312.08468/) (2023)
- [Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learning](https://mlsanity.com/paper/2409.02724/) (2024)

## Abstract (arXiv)

Cognitive cooperative assistance in robot-assisted surgery holds the potential to increase quality of care in minimally invasive interventions. Automation of surgical tasks promises to reduce the mental exertion and fatigue of surgeons. In this work, multi-agent reinforcement learning is demonstrated to be robust to the distribution shift introduced by pairing a learned policy with a human team member. Multi-agent policies are trained directly from images in simulation to control multiple instruments in a sub task of the minimally invasive removal of the gallbladder. These agents are evaluated individually and in cooperation with humans to demonstrate their suitability as autonomous assistants. Compared to human teams, the hybrid teams with artificial agents perform better considering completion time (44.4% to 71.2% shorter) as well as number of collisions (44.7% to 98.0% fewer). Path lengths, however, increase under control of an artificial agent (11.4% to 33.5% longer). A multi-agent formulation of the learning problem was favored over a single-agent formulation on this surgical sub task, due to the sequential learning of the two instruments. This approach may be extended to other tasks that are difficult to formulate within the standard reinforcement learning framework. Multi-agent reinforcement learning may shift the paradigm of cognitive robotic surgery towards seamless cooperation between surgeons and assistive technologies.