# Unsupervised identification of surgical robotic actions from small non homogeneous datasets

arXiv: https://arxiv.org/abs/2105.08488 · HTML: https://mlsanity.com/paper/2105.08488/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 47 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 10 of 10)

- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Autonomous task planning and situation awareness in robotic surgery](https://mlsanity.com/paper/2004.08911/) (2020)
- [Real-time analysis of cataract surgery videos using statistical models](https://mlsanity.com/paper/1610.05465/) (2016)
- [Weakly Supervised Recognition of Surgical Gestures](https://mlsanity.com/paper/1907.10993/) (2019)
- [Dynamic Movement Primitives: Volumetric Obstacle Avoidance Using Dynamic Potential Functions](https://mlsanity.com/paper/2007.00518/) (2020)
- [Overcoming Some Drawbacks of Dynamic Movement Primitives](https://mlsanity.com/paper/1908.10608/) (2019)
- [Improving rigid 3D calibration for robotic surgery](https://mlsanity.com/paper/2007.08427/) (2020)
- [Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment](https://mlsanity.com/paper/2103.04075/) (2021)
- [Knowledge transfer for surgical activity prediction](https://mlsanity.com/paper/1711.05848/) (2017)
- [Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations](https://mlsanity.com/paper/1810.00599/) (2018)

## Built on by (most cited first, 3 of 3)

- [Towards Surgical Context Inference and Translation to Gestures](https://mlsanity.com/paper/2302.14237/) (2023)
- [Robotic Constrained Imitation Learning for the Peg Transfer Task in Fundamentals of Laparoscopic Surgery](https://mlsanity.com/paper/2405.03440/) (2024)
- [Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback](https://mlsanity.com/paper/2501.07507/) (2025)

## Abstract (arXiv)

Robot-assisted surgery is an established clinical practice. The automatic identification of surgical actions is needed for a range of applications, including performance assessment of trainees and surgical process modeling for autonomous execution and monitoring. However, supervised action identification is not feasible, due to the burden of manually annotating recordings of potentially complex and long surgical executions. Moreover, often few example executions of a surgical procedure can be recorded. This paper proposes a novel fast algorithm for unsupervised identification of surgical actions in a standard surgical training task, the ring transfer, executed with da Vinci Research Kit. Exploiting kinematic and semantic visual features automatically extracted from a very limited dataset of executions, we are able to significantly outperform state-of-the-art results on a dataset of non-expert executions (58% vs. 24% F1-score), and improve performance in the presence of noise, short actions and non-homogeneous workflows, i.e. non repetitive action sequences.