# From Decision to Action in Surgical Autonomy: Multi-Modal Large Language Models for Robot-Assisted Blood Suction

arXiv: https://arxiv.org/abs/2408.07806 · HTML: https://mlsanity.com/paper/2408.07806/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2024.

## Built on by (most cited first, 8 of 8)

- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning](https://mlsanity.com/paper/2411.14622/) (2024)
- [NL2GenSym: Natural Language to Generative Symbolic Rules for SOAR Cognitive Architecture via Large Language Models](https://mlsanity.com/paper/2510.09355/) (2025)
- [Benchmarking the Safety of Large Language Models for Robotic Health Attendant Control](https://mlsanity.com/paper/2604.26577/) (2026)
- [Federated Deep Reinforcement Learning for Privacy-Preserving Robotic-Assisted Surgery](https://mlsanity.com/paper/2505.12153/) (2025)
- [Agentic Surgical AI: Surgeon Style Fingerprinting and Privacy Risk Quantification via Discrete Diffusion in a Vision-Language-Action Framework](https://mlsanity.com/paper/2506.08185/) (2025)
- [Sound Source Localization for Spatial Mapping of Surgical Actions in Dynamic Scenes](https://mlsanity.com/paper/2510.24332/) (2025)
- [MedVLA: A Hierarchical Vision-Language-Action Framework for Closed-Loop Precision Medical Robot Manipulation](https://mlsanity.com/paper/2609.25756/) (2026)

## Abstract (arXiv)

The rise of Large Language Models (LLMs) has impacted research in robotics and automation. While progress has been made in integrating LLMs into general robotics tasks, a noticeable void persists in their adoption in more specific domains such as surgery, where critical factors such as reasoning, explainability, and safety are paramount. Achieving autonomy in robotic surgery, which entails the ability to reason and adapt to changes in the environment, remains a significant challenge. In this work, we propose a multi-modal LLM integration in robot-assisted surgery for autonomous blood suction. The reasoning and prioritization are delegated to the higher-level task-planning LLM, and the motion planning and execution are handled by the lower-level deep reinforcement learning model, creating a distributed agency between the two components. As surgical operations are highly dynamic and may encounter unforeseen circumstances, blood clots and active bleeding were introduced to influence decision-making. Results showed that using a multi-modal LLM as a higher-level reasoning unit can account for these surgical complexities to achieve a level of reasoning previously unattainable in robot-assisted surgeries. These findings demonstrate the potential of multi-modal LLMs to significantly enhance contextual understanding and decision-making in robotic-assisted surgeries, marking a step toward autonomous surgical systems.