ProtoFlow: Interpretable and Robust Surgical Workflow Modeling with Learned Dynamic Scene Graph Prototypes
Not on the map. Why?
Abstract (arXiv)
Purpose: Detailed surgical recognition is critical for advancing AI-assisted surgery, yet progress is hampered by high annotation costs, data scarcity, and a lack of interpretable models. While scene graphs offer a structured abstraction of surgical events, their full potential remains untapped. In this work, we introduce ProtoFlow, a novel framework that learns dynamic scene graph prototypes to model complex surgical workflows in an interpretable and robust manner. Methods: ProtoFlow leverages a graph neural network (GNN) encoder-decoder architecture that combines self-supervised pretraining for rich representation learning with a prototype-based fine-tuning stage. This process discovers and refines core prototypes that encapsulate recurring, clinically meaningful patterns of surgical interaction, forming an explainable foundation for workflow analysis. Results: We evaluate our approach on the fine-grained CAT-SG dataset. ProtoFlow not only outperforms standard GNN baselines in overall accuracy but also demonstrates exceptional robustness in limited-data, few-shot scenarios, maintaining strong performance when trained on as few as one surgical video. Our qualitative analyses further show that the learned prototypes successfully identify distinct surgical sub-techniques and provide clear, interpretable insights into workflow deviations and rare complications. Conclusion: By uniting robust representation learning with inherent explainability, ProtoFlow represents a significant step toward developing more transparent, reliable, and data-efficient AI systems, accelerating their potential for clinical adoption in surgical training, real-time decision support, and workflow optimization.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 21 references that are in the map. First posted on arXiv 16 Dec 2025.
Builds on
16 of 21 references.
References · 16
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
- 2021How Attentive are Graph Attention Networks?cited by 835
- 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 367
- 2018Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learningcited by 81
- 2018Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networkscited by 57
- 2019Video-based surgical skill assessment using 3D convolutional neural networkscited by 57
- 20224D-OR: Semantic Scene Graphs for OR Domain Modelingcited by 49
- 2023Dynamic Scene Graph Representation for Surgical Videocited by 21
- 2022Towards Prototype-Based Self-Explainable Graph Neural Networkcited by 14
- 2023Encoding Surgical Videos as Latent Spatiotemporal Graphs for Object and Anatomy-Driven Reasoningcited by 14
4 more
- 2019Offline identification of surgical deviations in laparoscopic rectopexycited by 12
- 2024SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Predictioncited by 12
- 2025Watch and Learn: Leveraging Expert Knowledge and Language for Surgical Video Understandingcited by 4
- 2025CAT-SG: A Large Dynamic Scene Graph Dataset for Fine-Grained Understanding of Cataract Surgerycited by 4
Built on by
Not cited yet.