# Deep Neural Networks for the Assessment of Surgical Skills: A Systematic Review

arXiv: https://arxiv.org/abs/2103.05113 · HTML: https://mlsanity.com/paper/2103.05113/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 67 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks](https://mlsanity.com/paper/1802.08774/) (2018)
- [Video-based surgical skill assessment using 3D convolutional neural networks](https://mlsanity.com/paper/1903.02306/) (2019)
- [The TUM LapChole dataset for the M2CAI 2016 workflow challenge](https://mlsanity.com/paper/1610.09278/) (2016)
- [Manipulation-skill Assessment from Videos with Spatial Attention Network](https://mlsanity.com/paper/1901.02579/) (2019)
- [Video and Accelerometer-Based Motion Analysis for Automated Surgical Skills Assessment](https://mlsanity.com/paper/1702.07772/) (2017)
- [Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection](https://mlsanity.com/paper/2008.00936/) (2020)
- [Automated Surgical Skill Assessment in RMIS Training](https://mlsanity.com/paper/1712.08604/) (2017)
- [Evaluating surgical skills from kinematic data using convolutional neural networks](https://mlsanity.com/paper/1806.02750/) (2018)
- [Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks](https://mlsanity.com/paper/1908.07319/) (2019)
- [Deep Learning with Convolutional Neural Network for Objective Skill Evaluation in Robot-assisted Surgery](https://mlsanity.com/paper/1806.05796/) (2018)
- [SATR-DL: Improving Surgical Skill Assessment and Task Recognition in Robot-assisted Surgery with Deep Neural Networks](https://mlsanity.com/paper/1806.05798/) (2018)
- [Machine Learning Approach for Skill Evaluation in Robotic-Assisted Surgery](https://mlsanity.com/paper/1611.05136/) (2016)
- [Who's Better? Who's Best? Pairwise Deep Ranking for Skill Determination](https://mlsanity.com/paper/1703.09913/) (2017)
- [Scoring and Assessment in Medical VR Training Simulators with Dynamic Time Series Classification](https://mlsanity.com/paper/2006.12366/) (2020)
- [Automated Performance Assessment in Transoesophageal Echocardiography with Convolutional Neural Networks](https://mlsanity.com/paper/1806.05154/) (2018)
- [Recurrent and Spiking Modeling of Sparse Surgical Kinematics](https://mlsanity.com/paper/2005.05868/) (2020)

## Built on by (most cited first, 10 of 10)

- [Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning](https://mlsanity.com/paper/2203.09589/) (2022)
- [Cataract-1K: Cataract Surgery Dataset for Scene Segmentation, Phase Recognition, and Irregularity Detection](https://mlsanity.com/paper/2312.06295/) (2023)
- [One-shot skill assessment in high-stakes domains with limited data via meta learning](https://mlsanity.com/paper/2301.00812/) (2022)
- [ReCAP: Recursive Cross Attention Network for Pseudo-Label Generation in Robotic Surgical Skill Assessment](https://mlsanity.com/paper/2407.05180/) (2024)
- [End-to-End Deep Learning for Real-Time Neuroimaging-Based Assessment of Bimanual Motor Skills](https://mlsanity.com/paper/2504.03681/) (2025)
- [Cognitive-Motor Integration in Assessing Bimanual Motor Skills](https://mlsanity.com/paper/2404.10889/) (2024)
- [Airway Skill Assessment with Spatiotemporal Attention Mechanisms Using Human Gaze](https://mlsanity.com/paper/2506.19306/) (2025)
- [ExpOS: Explainable Open-Surgery Skills Assessment Using 3D Hand Reconstruction](https://mlsanity.com/paper/2605.23653/) (2026)
- [Constructing Predictive Surgical Path for AI-based Capsulorhexis Skill Transfer](https://mlsanity.com/paper/2608.21441/) (2026)
- [Spatiotemporal Recognition of Abdominal Wall Layers During Optical Trocar Entry](https://mlsanity.com/paper/2610.14512/) (2026)

## Abstract (arXiv)

Surgical training in medical school residency programs has followed the apprenticeship model. The learning and assessment process is inherently subjective and time-consuming. Thus, there is a need for objective methods to assess surgical skills. Here, we use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to systematically survey the literature on the use of Deep Neural Networks for automated and objective surgical skill assessment, with a focus on kinematic data as putative markers of surgical competency. There is considerable recent interest in deep neural networks (DNN) due to the availability of powerful algorithms, multiple datasets, some of which are publicly available, as well as efficient computational hardware to train and host them. We have reviewed 530 papers, of which we selected 25 for this systematic review. Based on this review, we concluded that DNNs are powerful tools for automated, objective surgical skill assessment using both kinematic and video data. The field would benefit from large, publicly available, annotated datasets that are representative of the surgical trainee and expert demographics and multimodal data beyond kinematics and videos.