mlsanityData to 14 Oct 2026

Concept Graph Neural Networks for Surgical Video Understanding

arXiv 2202.13402 ↗First posted 27 Feb 2022cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

We constantly integrate our knowledge and understanding of the world to enhance our interpretation of what we see. This ability is crucial in application domains which entail reasoning about multiple entities and concepts, such as AI-augmented surgery. In this paper, we propose a novel way of integrating conceptual knowledge into temporal analysis tasks via temporal concept graph networks. In the proposed networks, a global knowledge graph is incorporated into the temporal analysis of surgical instances, learning the meaning of concepts and relations as they apply to the data. We demonstrate our results in surgical video data for tasks such as verification of critical view of safety, as well as estimation of Parkland grading scale. The results show that our method improves the recognition and detection of complex benchmarks as well as enables other analytic applications of interest.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 46 of its 97 references that are in the map. First posted on arXiv 27 Feb 2022.

Builds on

46 of 97 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  5. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  6. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  7. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  8. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  9. 2016Geometric deep learning: going beyond Euclidean datacited by 1,048
  10. 2020Object-Centric Learning with Slot Attentioncited by 876
  11. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  12. 2017Dynamic Routing Between Capsulescited by 689
34 more
  1. 2018Hypergraph Neural Networkscited by 488
  2. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 367
  3. 2015A Review of Relational Machine Learning for Knowledge Graphscited by 265
  4. 2016Molecular Graph Convolutions: Moving Beyond Fingerprintscited by 238
  5. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  6. 2018Videos as Space-Time Region Graphscited by 221
  7. 2019Contrastive Learning of Structured World Modelscited by 209
  8. 2017Social Attention: Modeling Attention in Human Crowdscited by 186
  9. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
  10. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
  11. 2016Graph-Structured Representations for Visual Question Answeringcited by 123
  12. 2018Graph-Based Global Reasoning Networkscited by 120
  13. 2016Composing graphical models with neural networks for structured representations and fast inferencecited by 113
  14. 2019Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systemscited by 107
  15. 2020DeepGMR: Learning Latent Gaussian Mixture Models for Registrationcited by 91
  16. 2019Unsupervised speech representation learning using WaveNet autoencoderscited by 90
  17. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  18. 2019Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANscited by 79
  19. 2019Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysiscited by 76
  20. 2020Spatiotemporal Relationship Reasoning for Pedestrian Intent Predictioncited by 71
  21. 2020Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Tripletscited by 66
  22. 2019Dual Graph Convolutional Network for Semantic Segmentationcited by 59
  23. 2019VideoGraph: Recognizing Minutes-Long Human Activities in Videoscited by 57
  24. 2017Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gatescited by 54
  25. 2016Stick-Breaking Variational Autoencoderscited by 44
  26. 2020Graphical Normalizing Flowscited by 37
  27. 2018RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotationscited by 26
  28. 2019Uncertainty-Aware Driver Trajectory Prediction at Urban Intersectionscited by 26
  29. 2019Attention routing between capsulescited by 17
  30. 2020Graph Convolutional Neural Networks with Node Transition Probability-based Message Passing and DropNode Regularizationcited by 15
  31. 2020Latent World Models For Intrinsically Motivated Explorationcited by 15
  32. 2021HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Samplingcited by 12
  33. 2020Aggregating Long-Term Context for Learning Laparoscopic and Robot-Assisted Surgical Workflowscited by 10
  34. 2021SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgerycited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 6; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmarkcited by 21
  2. 2024VS-Assistant: Versatile Surgery Assistant on the Demand of Surgeonscited by 7
  3. 2025The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessmentcited by 6
  4. 2024Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgerycited by 5
  5. 2024Surgical Triplet Recognition via Diffusion Modelcited by 4
  6. 2025Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgerycited by 3
  7. 2026SurgSigma: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligencecited by 2
  8. 2025SurgVisAgent: Multimodal Agentic Model for Versatile Surgical Visual Enhancementcited by 1
  9. 2023SurGNN: Explainable visual scene understanding and assessment of surgical skill using graph neural networkscited by 0
  10. 2025Tracking-Aware Deformation Field Estimation for Non-rigid 3D Reconstruction in Robotic Surgeriescited by 0
  11. 2025Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistancecited by 0
  12. 2025Learning from Single Timestamps: Complexity Estimation in Laparoscopic Cholecystectomycited by 0
1 more
  1. 2026SAGE-OR: Semi-supervised Adaptive Scene Graph Generation for Operating Roomscited by 0