# Surgical Gesture Recognition with Optical Flow only

arXiv: https://arxiv.org/abs/1904.01143 · HTML: https://mlsanity.com/paper/1904.01143/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [MIcro-Surgical Anastomose Workflow recognition challenge report](https://mlsanity.com/paper/2103.13111/) (2021)
- [Gesture Recognition in Robotic Surgery: a Review](https://mlsanity.com/paper/2102.00027/) (2021)
- [PEg TRAnsfer Workflow recognition challenge report: Does multi-modal data improve recognition?](https://mlsanity.com/paper/2202.05821/) (2022)
- [Know your sensORs -- A Modality Study For Surgical Action Classification](https://mlsanity.com/paper/2203.08674/) (2022)
- [Using Hand Pose Estimation To Automate Open Surgery Training Feedback](https://mlsanity.com/paper/2211.07021/) (2022)
- [Zero-shot Prompt-based Video Encoder for Surgical Gesture Recognition](https://mlsanity.com/paper/2403.19786/) (2024)
- [On the Evaluation Protocol of Gesture Recognition for UAV-based Rescue Operation based on Deep Learning: A Subject-Independence Perspective](https://mlsanity.com/paper/2602.17854/) (2026)

## Abstract (arXiv)

In this paper, we address the open research problem of surgical gesture recognition using motion cues from video data only. We adapt Optical flow ConvNets initially proposed by Simonyan et al.. While Simonyan uses both RGB frames and dense optical flow, we use only dense optical flow representations as input to emphasize the role of motion in surgical gesture recognition, and present it as a robust alternative to kinematic data. We also overcome one of the limitations of Optical flow ConvNets by initializing our model with cross modality pre-training. A large number of promising studies that address surgical gesture recognition highly rely on kinematic data which requires additional recording devices. To our knowledge, this is the first paper that addresses surgical gesture recognition using dense optical flow information only. We achieve competitive results on JIGSAWS dataset, moreover, our model achieves more robust results with less standard deviation, which suggests optical flow information can be used as an alternative to kinematic data for the recognition of surgical gestures.