# SurGNN: Explainable visual scene understanding and assessment of surgical skill using graph neural networks

arXiv: https://arxiv.org/abs/2308.13073 · HTML: https://mlsanity.com/paper/2308.13073/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Graph Clustering with Graph Neural Networks](https://mlsanity.com/paper/2006.16904/) (2020)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention](https://mlsanity.com/paper/2004.01389/) (2020)
- [PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection](https://mlsanity.com/paper/2012.10412/) (2020)
- [Towards Holistic Surgical Scene Understanding](https://mlsanity.com/paper/2212.04582/) (2022)
- [Automated Surgical Skill Assessment in RMIS Training](https://mlsanity.com/paper/1712.08604/) (2017)
- [Surgical Activity Recognition in Robot-Assisted Radical Prostatectomy using Deep Learning](https://mlsanity.com/paper/1806.00466/) (2018)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Video-based assessment of intraoperative surgical skill](https://mlsanity.com/paper/2205.06416/) (2022)
- [wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data](https://mlsanity.com/paper/2209.10399/) (2022)
- [RefiNeRF: Modelling dynamic neural radiance fields with inconsistent or missing camera parameters](https://mlsanity.com/paper/2303.08695/) (2023)

## Abstract (arXiv)

This paper explores how graph neural networks (GNNs) can be used to enhance visual scene understanding and surgical skill assessment. By using GNNs to analyze the complex visual data of surgical procedures represented as graph structures, relevant features can be extracted and surgical skill can be predicted. Additionally, GNNs provide interpretable results, revealing the specific actions, instruments, or anatomical structures that contribute to the predicted skill metrics. This can be highly beneficial for surgical educators and trainees, as it provides valuable insights into the factors that contribute to successful surgical performance and outcomes. SurGNN proposes two concurrent approaches -- one supervised and the other self-supervised. The paper also briefly discusses other automated surgical skill evaluation techniques and highlights the limitations of hand-crafted features in capturing the intricacies of surgical expertise. We use the proposed methods to achieve state-of-the-art results on EndoVis19, and custom datasets. The working implementation of the code can be found at https://github.com/<redacted>.