mlsanityData to 14 Oct 2026

Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgery

arXiv 2509.20941 ↗First posted 25 Sep 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

As surgical AI transitions from pixel-level detection to complex reasoning, Scene Graphs (SGs) offer the structured, relational representations necessary to decode dynamic surgical environments. This PRISMA-ScR-guided scoping review systematically maps the evolving landscape of SG research in surgery, analyzing 52 primary studies to chart applications and methodological shifts. Our analysis reveals rapid growth, yet uncovers a critical 'data divide': internal-view research (e.g., triplet recognition from endoscopic video) accounts for 81% of studies and almost exclusively uses real-world 2D video, while external-view operating room modeling relies heavily on simulated data. Methodologically, we identify a decisive shift from foundational graph neural networks to specialized foundation models and generative AI, which together now account for approximately 50% of research in 2025. Crucially, our synthesis suggests that Scene Graphs are evolving from simple descriptors into essential 'neuro-symbolic guardrails', providing the structured, verifiable intermediate representation needed to prevent hallucinations in increasingly autonomous Surgical Foundation Models. Despite this promise, a major translational gap remains: none of the reviewed studies have proceeded to prospective clinical validation. We conclude that bridging this gap requires moving beyond standard computer vision metrics; we therefore propose the 'Validation Trinity' -- prioritizing Semantic Query Success, Latency-Aware Accuracy, and Safety-Critical Recall -- as the necessary evaluation framework to bring graph-based surgical AI into clinical practice.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 43 of its 95 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

43 of 95 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  3. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 367
  4. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
  5. 2017Surgical Data Science: Enabling Next-Generation Surgerycited by 126
  6. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  7. 2020Semantic Image Manipulation Using Scene Graphscited by 82
  8. 2022Scene Graph Generation: A Comprehensive Surveycited by 73
  9. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  10. 2020Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Tripletscited by 66
  11. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  12. 2023Action Scene Graphs for Long-Form Understanding of Egocentric Videoscited by 51
31 more
  1. 2018MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimationcited by 49
  2. 20224D-OR: Semantic Scene Graphs for OR Domain Modelingcited by 49
  3. 2022CholecTriplet2021: A benchmark challenge for surgical action triplet recognitioncited by 44
  4. 2023Advancing Surgical VQA with Scene Graph Knowledgecited by 41
  5. 2022Latent Graph Representations for Critical View of Safety Assessmentcited by 37
  6. 2025EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgerycited by 37
  7. 2020Learning and Reasoning with the Graph Structure Representation in Robotic Surgerycited by 29
  8. 2022Global-Reasoned Multi-Task Learning Model for Surgical Scene Understandingcited by 29
  9. 2025MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environmentscited by 28
  10. 2022Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognitioncited by 27
  11. 2023CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detectioncited by 23
  12. 2023Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactionscited by 23
  13. 2023LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Roomscited by 21
  14. 2023Dynamic Scene Graph Representation for Surgical Videocited by 21
  15. 2024EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videoscited by 20
  16. 2023Dynamic Interactive Relation Capturing via Scene Graph Learning for Robotic Surgical Report Generationcited by 14
  17. 2024ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modelingcited by 14
  18. 2022Concept Graph Neural Networks for Surgical Video Understandingcited by 13
  19. 2024SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Predictioncited by 12
  20. 2024VISAGE: Video Synthesis using Action Graphs for Surgerycited by 11
  21. 2024Instrument-tissue Interaction Detection Framework for Surgical Video Understandingcited by 8
  22. 2021Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedurescited by 6
  23. 2024Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Roomscited by 6
  24. 2025Surgery-R1: Advancing Surgical-VQLA with Reasoning Multimodal Large Language Model via Reinforcement Learningcited by 5
  25. 2019Compact Scene Graphs for Layout Composition and Patch Retrievalcited by 4
  26. 2022Task-Aware Asynchronous Multi-Task Model with Class Incremental Contrastive Learning for Surgical Scene Understandingcited by 4
  27. 2024Voxel Scene Graph for Intracranial Hemorrhagecited by 4
  28. 2025SurGrID: Controllable Surgical Simulation via Scene Graph to Image Diffusioncited by 3
  29. 2025Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglementcited by 3
  30. 2025Specialized Foundation Models for Intelligent Operating Roomscited by 3
  31. 2025Surgical-MambaLLM: Mamba2-enhanced Multimodal Large Language Model for VQLA in Robotic Surgerycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
  2. 2025Recognizing Surgical Phases Anywhere: Few-Shot Test-time Adaptation and Task-graph Guided Refinement
  3. 2026SurgGraph: Quantitative Laparoscopic Video Understanding via Geometry-Grounded Scene Graphs ↗
  4. 2026Surgical Video Generation From Diffusion to World Models: A Survey
  5. 2026A vision-language model and platform for temporally mapping surgery from video
  6. 2026A Comparative Study in Surgical AI: Potential and Limitations of Data, Compute, and Scaling
  7. 2025SG2VID: Scene Graphs Enable Fine-Grained Control for Video Synthesis
  8. 2025SurgXBench: Explainable Vision-Language Model Benchmark for Surgery