# Unsupervised Learning for Surgical Motion by Learning to Predict the Future

arXiv: https://arxiv.org/abs/1806.03318 · HTML: https://mlsanity.com/paper/1806.03318/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2018.

## Built on by (most cited first, 1 of 1)

- [Mixture Density Networks for Classification with an Application to Product Bundling](https://mlsanity.com/paper/2402.05428/) (2024)

## Abstract (arXiv)

We show that it is possible to learn meaningful representations of surgical motion, without supervision, by learning to predict the future. An architecture that combines an RNN encoder-decoder and mixture density networks (MDNs) is developed to model the conditional distribution over future motion given past motion. We show that the learned encodings naturally cluster according to high-level activities, and we demonstrate the usefulness of these learned encodings in the context of information retrieval, where a database of surgical motion is searched for suturing activity using a motion-based query. Future prediction with MDNs is found to significantly outperform simpler baselines as well as the best previously-published result for this task, advancing state-of-the-art performance from an F1 score of 0.60 +- 0.14 to 0.77 +- 0.05.