# Event Recognition in Laparoscopic Gynecology Videos with Hybrid Transformers

arXiv: https://arxiv.org/abs/2312.00593 · HTML: https://mlsanity.com/paper/2312.00593/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks](https://mlsanity.com/paper/2003.10751/) (2020)
- [Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis](https://mlsanity.com/paper/1907.06099/) (2019)
- [Video-based surgical skill assessment using 3D convolutional neural networks](https://mlsanity.com/paper/1903.02306/) (2019)
- [Relevance Detection in Cataract Surgery Videos by Spatio-Temporal Action Localization](https://mlsanity.com/paper/2104.14280/) (2021)
- [LensID: A CNN-RNN-Based Framework Towards Lens Irregularity Detection in Cataract Surgery Videos](https://mlsanity.com/paper/2107.00875/) (2021)
- [An Empirical Study on Activity Recognition in Long Surgical Videos](https://mlsanity.com/paper/2205.02805/) (2022)
- [Recognition and Prediction of Surgical Gestures and Trajectories Using Transformer Models in Robot-Assisted Surgery](https://mlsanity.com/paper/2212.01683/) (2022)
- [Action Recognition in Video Recordings from Gynecologic Laparoscopy](https://mlsanity.com/paper/2311.18666/) (2023)
- [A Smoke Removal Method for Laparoscopic Images](https://mlsanity.com/paper/1803.08410/) (2018)

## Built on by (most cited first, 5 of 5)

- [SurgBench: A Unified Large-Scale Benchmark for Surgical Video Analysis](https://mlsanity.com/paper/2506.07603/) (2025)
- [GynSurg: A Comprehensive Gynecology Laparoscopic Surgery Dataset](https://mlsanity.com/paper/2506.11356/) (2025)
- [Dual Invariance Self-training for Reliable Semi-supervised Surgical Phase Recognition](https://mlsanity.com/paper/2501.17628/) (2025)
- [WetCat: Enabling Automated Skill Assessment in Wet-Lab Cataract Surgery Videos](https://mlsanity.com/paper/2506.08896/) (2025)
- [Data-Efficient Learning for Generalizable Surgical Video Understanding](https://mlsanity.com/paper/2508.10215/) (2025)

## Abstract (arXiv)

Analyzing laparoscopic surgery videos presents a complex and multifaceted challenge, with applications including surgical training, intra-operative surgical complication prediction, and post-operative surgical assessment. Identifying crucial events within these videos is a significant prerequisite in a majority of these applications. In this paper, we introduce a comprehensive dataset tailored for relevant event recognition in laparoscopic gynecology videos. Our dataset includes annotations for critical events associated with major intra-operative challenges and post-operative complications. To validate the precision of our annotations, we assess event recognition performance using several CNN-RNN architectures. Furthermore, we introduce and evaluate a hybrid transformer architecture coupled with a customized training-inference framework to recognize four specific events in laparoscopic surgery videos. Leveraging the Transformer networks, our proposed architecture harnesses inter-frame dependencies to counteract the adverse effects of relevant content occlusion, motion blur, and surgical scene variation, thus significantly enhancing event recognition accuracy. Moreover, we present a frame sampling strategy designed to manage variations in surgical scenes and the surgeons' skill level, resulting in event recognition with high temporal resolution. We empirically demonstrate the superiority of our proposed methodology in event recognition compared to conventional CNN-RNN architectures through a series of extensive experiments.