# Video-based Surgical Skill Assessment Using Dynamics-and-Uncertainty-Aware Tree-based Gaussian Process Classifier

arXiv: https://arxiv.org/abs/2609.24619 · HTML: https://mlsanity.com/paper/2609.24619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 16 of 16)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Video-based surgical skill assessment using 3D convolutional neural networks](https://mlsanity.com/paper/1903.02306/) (2019)
- [Representation Flow for Action Recognition](https://mlsanity.com/paper/1810.01455/) (2018)
- [Manipulation-skill Assessment from Videos with Spatial Attention Network](https://mlsanity.com/paper/1901.02579/) (2019)
- [Video and Accelerometer-Based Motion Analysis for Automated Surgical Skills Assessment](https://mlsanity.com/paper/1702.07772/) (2017)
- [Dependent Multinomial Models Made Easy: Stick Breaking with the P'olya-Gamma Augmentation](https://mlsanity.com/paper/1506.05843/) (2015)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art](https://mlsanity.com/paper/2304.13014/) (2023)
- [Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation](https://mlsanity.com/paper/1802.06383/) (2018)
- [Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning](https://mlsanity.com/paper/2203.09589/) (2022)
- [Objective Surgical Skills Assessment and Tool Localization: Results from the MICCAI 2021 SimSurgSkill Challenge](https://mlsanity.com/paper/2212.04448/) (2022)
- [Scalable Multi-Class Gaussian Process Classification using Expectation Propagation](https://mlsanity.com/paper/1706.07258/) (2017)
- [Accounting for Input Noise in Gaussian Process Parameter Retrieval](https://mlsanity.com/paper/2005.09907/) (2020)
- [Cataract-LMM Large-Scale Multi-Source Multi-Task Benchmark for Deep Learning in Surgical Video Analysis](https://mlsanity.com/paper/2510.16371/) (2025)
- [Real-time Informative Surgical Skill Assessment with Gaussian Process Learning](https://mlsanity.com/paper/2112.02598/) (2021)

## Abstract (arXiv)

The proposed pipeline integrates a representation-flow convolutional neural network with a dynamics- and uncertainty-aware tree-based Gaussian Process classifier. In this framework, latent motion dynamics are exploited both as discriminative representations and as a source of input uncertainty, enhancing robustness against temporal variations and abnormal motion transitions. Compared with conventional deep learning approaches, the proposed strategy requires less training data and offers improved computational efficiency. To further improve classification performance, we introduce novel semantic-aware compound kernels that effectively capture semantic, flow, and dynamic information embedded in surgical video features. In addition, uncertainty-aware kernels are developed to strengthen the robustness and practical applicability of the compound kernel framework. The proposed method is evaluated on two benchmark datasets, namely the JIGSAWS and the Cataract-LMM (Capsulorhexis) datasets. Experimental results demonstrate strong performance across both datasets, including the LOSO and LOUO evaluation protocols on JIGSAWS, including the subject-independent LOUO protocol on JIGSAWS, on which the framework attains a mean accuracy of ph96.9%; results under the within-subject LOSO protocol are reported for comparability with prior work, achieving competitive accuracy while substantially reducing computational cost. Overall, the proposed pipeline provides an efficient and accurate framework for video-based surgical skill assessment.