mlsanityData to 7 Oct 2026

Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation

arXiv 2603.12430 ↗First posted 12 Mar 2026cs.CVcited by 2

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Surgical scene understanding demands not only accurate predictions but also interpretable reasoning that surgeons can verify against clinical expertise. However, existing surgical vision-language models generate predictions without reasoning chains, and general-purpose reasoning models fail on compositional surgical tasks without domain-specific knowledge. We present Surg-R1, a surgical Vision-Language Model that addresses this gap through hierarchical reasoning trained via a four-stage pipeline. Our approach introduces three key contributions: (1) a three-level reasoning hierarchy decomposing surgical interpretation into perceptual grounding, relational understanding, and contextual reasoning; (2) the largest surgical chain-of-thought dataset with 320,000 reasoning pairs; and (3) a four-stage training pipeline progressing from supervised fine-tuning to group relative policy optimization and iterative self-improvement. Evaluation on SurgBench, comprising six public benchmarks and six multi-center external validation datasets from five institutions, demonstrates that Surg-R1 achieves the highest Arena Score (64.9%) on public benchmarks versus Gemini 3.0 Pro (46.1%) and GPT-5.1 (37.9%), outperforming both proprietary reasoning models and specialized surgical VLMs on the majority of tasks spanning instrument localization, triplet recognition, phase recognition, action recognition, and critical view of safety assessment, with a 15.2 percentage point improvement over the strongest surgical baseline on external validation.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 49 of its 60 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

49 of 60 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026SUREON: A Benchmark and Vision-Language-Model for Surgical Reasoning6 Mar
  2. 2026MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Calling11 Feb
  3. 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model23 Jul
  4. 2025MedGround-R1: Advancing Medical Image Grounding via Spatial-Semantic Rewarded Group Relative Policy Optimization1 Jul
  5. 2025Surgery-R1: Advancing Surgical-VQLA with Reasoning Multimodal Large Language Model via Reinforcement Learning24 Jun
  6. 2025SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis13 Mar
  7. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb

Other references · 42

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  7. 2025Gemma 3 Technical Reportcited by 1,686
  8. 2025OpenAI GPT-5 System Cardcited by 910
  9. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  11. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 366
  12. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
30 more
  1. 2020Surgical Data Science -- from Concepts toward Clinical Translationcited by 105
  2. 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
  3. 2025Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thoughtcited by 69
  4. 2022Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformercited by 58
  5. 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 54
  6. 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understandingcited by 53
  7. 2023SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgerycited by 48
  8. 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 48
  9. 2022CholecTriplet2021: A benchmark challenge for surgical action triplet recognitioncited by 44
  10. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  11. 2024HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognitioncited by 41
  12. 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
  13. 2024Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgerycited by 39
  14. 2025EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgerycited by 37
  15. 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learningcited by 34
  16. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 30
  17. 2024Foundation Models in Radiology: What, How, When, Why and Why Notcited by 26
  18. 2023CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detectioncited by 22
  19. 2024PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgerycited by 22
  20. 2023CholecTrack20: A Multi-Perspective Tracking Dataset for Surgical Toolscited by 17
  21. 2024CholecInstanceSeg: A Tool Instance Segmentation Dataset for Laparoscopic Surgerycited by 16
  22. 2024CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissectioncited by 15
  23. 2024SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Predictioncited by 13
  24. 2024Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Modelscited by 13
  25. 2025EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysiscited by 13
  26. 2025SurgVidLM: Towards Multi-grained Surgical Video Understanding with Large Language Modelcited by 13
  27. 2024SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy datacited by 6
  28. 2025The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessmentcited by 6
  29. 2024MAMMAL -- Molecular Aligned Multi-Modal Architecture and Languagecited by 4
  30. 2025CARES: Collaborative Agentic Reasoning for Error Detection in Surgerycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Unified Surgical Scene Understanding:Bridging Reasoning and Grounding via MLLMs
  2. 2025Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
  3. 2024GP-VLS: A general-purpose vision language model for surgery
  4. 2025SurgXBench: Explainable Vision-Language Model Benchmark for Surgery
  5. 2026SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark
  6. 2026SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video
  7. 2025SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learning
  8. 2024ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling