# Weakly Supervised Recognition of Surgical Gestures

arXiv: https://arxiv.org/abs/1907.10993 · HTML: https://mlsanity.com/paper/1907.10993/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2019.

## Built on by (most cited first, 6 of 6)

- [The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods](https://mlsanity.com/paper/2104.03178/) (2021)
- [Gesture Recognition in Robotic Surgery: a Review](https://mlsanity.com/paper/2102.00027/) (2021)
- [Surgical Gesture Recognition Based on Bidirectional Multi-Layer Independently RNN with Explainable Spatial Feature Extraction](https://mlsanity.com/paper/2105.00460/) (2021)
- [Runtime Detection of Executional Errors in Robot-Assisted Surgery](https://mlsanity.com/paper/2203.00737/) (2022)
- [Unsupervised identification of surgical robotic actions from small non homogeneous datasets](https://mlsanity.com/paper/2105.08488/) (2021)
- [Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback](https://mlsanity.com/paper/2501.07507/) (2025)

## Abstract (arXiv)

Kinematic trajectories recorded from surgical robots contain information about surgical gestures and potentially encode cues about surgeon's skill levels. Automatic segmentation of these trajectories into meaningful action units could help to develop new metrics for surgical skill assessment as well as to simplify surgical automation. State-of-the-art methods for action recognition relied on manual labelling of large datasets, which is time consuming and error prone. Unsupervised methods have been developed to overcome these limitations. However, they often rely on tedious parameter tuning and perform less well than supervised approaches, especially on data with high variability such as surgical trajectories. Hence, the potential of weak supervision could be to improve unsupervised learning while avoiding manual annotation of large datasets. In this paper, we used at a minimum one expert demonstration and its ground truth annotations to generate an appropriate initialization for a GMM-based algorithm for gesture recognition. We showed on real surgical demonstrations that the latter significantly outperforms standard task-agnostic initialization methods. We also demonstrated how to improve the recognition accuracy further by redefining the actions and optimising the inputs.