mlsanityData to 13 Oct 2026

SUREON: A Benchmark and Vision-Language-Model for Surgical Reasoning

arXiv 2603.06570 ↗First posted 6 Mar 2026cs.CV cs.AIcited by 6

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Surgeons don't just see -- they interpret. When an expert observes a surgical scene, they understand not only what instrument is being used, but why it was chosen, what risk it poses, and what comes next. Current surgical AI cannot answer such questions, largely because training data that explicitly encodes surgical reasoning is immensely difficult to annotate at scale. Yet surgical video lectures already contain exactly this -- explanations of intent, rationale, and anticipation, narrated by experts for the purpose of teaching. Though inherently noisy and unstructured, these narrations encode the reasoning that surgical AI currently lacks. We introduce SUREON, a large-scale video QA dataset that systematically harvests this training signal from surgical academic videos. SUREON defines 12 question categories covering safety assessment, decision rationale, and forecasting, and uses a multi-agent pipeline to extract and structure supervision at scale. Across 134.7K clips and 170 procedure types, SUREON yields 206.8k QA pairs and an expert-validated benchmark of 354 examples. To evaluate the extent to which this supervision translates to surgical reasoning ability, we introduce two models: SureonVLM, a vision-language model adapted through supervised fine-tuning, and SureonVLM-R1, a reasoning model trained with Group Relative Policy Optimization. Both models can answer complex questions about surgery and substantially outperform larger general-domain models, exceeding 84% accuracy on the SUREON benchmark while outperforming general-domain models on standard surgical perception tasks. Qualitative analysis of SureonVLM-R1 reveals explicit reasoning behavior, such as inferring operative intent from visual context.

Key facts. Cited by 6 papers in the map (data through 13 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 32 of its 37 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

32 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models18 Mar
  2. 2025SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis13 Mar
  3. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb

Other references · 29

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  2. 2025Qwen3-VL Technical Reportcited by 2,663
  3. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,052
  4. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 466
  5. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 366
  6. 20192017 Robotic Instrument Segmentation Challengecited by 167
  7. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
  8. 20202018 Robotic Scene Segmentation Challengecited by 123
  9. 2020CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80cited by 104
  10. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 87
  11. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  12. 2021Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmarkcited by 58
17 more
  1. 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 54
  2. 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 48
  3. 2020Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Roomcited by 47
  4. 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
  5. 2022Latent Graph Representations for Critical View of Safety Assessmentcited by 37
  6. 2025EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgerycited by 37
  7. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 30
  8. 2024Distilling Vision-Language Models on Millions of Videoscited by 20
  9. 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settingscited by 19
  10. 2024CholecInstanceSeg: A Tool Instance Segmentation Dataset for Laparoscopic Surgerycited by 16
  11. 2025SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learningcited by 16
  12. 2025EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoTcited by 16
  13. 2025Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)cited by 8
  14. 2025Domain Adaptation of VLM for Soccer Video Understandingcited by 7
  15. 2025GynSurg: A Comprehensive Gynecology Laparoscopic Surgery Datasetcited by 7
  16. 2024HemoSet: The First Blood Segmentation Dataset for Automation of Hemostasis Managementcited by 4
  17. 2025Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligencecited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026MedClaw: Heuristic Agent Harness for Long-Horizon Surgical Video Reasoning
  2. 2026SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark
  3. 2026Training LLMs with Reinforcement Learning over Digital Twin Representations for Reasoning-Intensive Surgical VideoQA
  4. 2025Specialized Foundation Models for Intelligent Operating Rooms
  5. 2026STSG-VQA: Evidence-Grounded Temporal Question Answering from Surgical Spatio-Temporal Scene Graphs
  6. 2026SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video
  7. 2025Surgery-R1: Advancing Surgical-VQLA with Reasoning Multimodal Large Language Model via Reinforcement Learning
  8. 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning