# Surgical Workflow Recognition: from Analysis of Challenges to Architectural Study

arXiv: https://arxiv.org/abs/2203.09230 · HTML: https://mlsanity.com/paper/2203.09230/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)
- [TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks](https://mlsanity.com/paper/2003.10751/) (2020)
- [Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer](https://mlsanity.com/paper/2103.09712/) (2021)
- [A Short Note on the Kinetics-700-2020 Human Action Dataset](https://mlsanity.com/paper/2010.10864/) (2020)
- [Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis](https://mlsanity.com/paper/1907.06099/) (2019)
- [OperA: Attention-Regularized Transformers for Surgical Phase Recognition](https://mlsanity.com/paper/2103.03873/) (2021)
- [Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark](https://mlsanity.com/paper/2109.14956/) (2021)
- [MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimation](https://mlsanity.com/paper/1808.08180/) (2018)
- [The TUM LapChole dataset for the M2CAI 2016 workflow challenge](https://mlsanity.com/paper/1610.09278/) (2016)
- [Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery](https://mlsanity.com/paper/2006.16166/) (2020)
- [Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis](https://mlsanity.com/paper/2107.02568/) (2021)
- [MIcro-Surgical Anastomose Workflow recognition challenge report](https://mlsanity.com/paper/2103.13111/) (2021)
- [Offline identification of surgical deviations in laparoscopic rectopexy](https://mlsanity.com/paper/1909.10790/) (2019)
- [Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis](https://mlsanity.com/paper/1702.03684/) (2017)
- [Prediction of laparoscopic procedure duration using unlabeled, multimodal sensor data](https://mlsanity.com/paper/1811.03384/) (2018)

## Built on by (most cited first, 6 of 6)

- [LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Rooms](https://mlsanity.com/paper/2303.13293/) (2023)
- [Metrics Matter in Surgical Phase Recognition](https://mlsanity.com/paper/2305.13961/) (2023)
- [TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognition](https://mlsanity.com/paper/2307.09997/) (2023)
- [Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Room](https://mlsanity.com/paper/2503.13028/) (2025)
- [TrackOR: Towards Personalized Intelligent Operating Rooms Through Robust Tracking](https://mlsanity.com/paper/2508.07968/) (2025)
- [Self-Supervised Uncalibrated Multi-View Video Anonymization in the Operating Room](https://mlsanity.com/paper/2602.02850/) (2026)

## Abstract (arXiv)

Algorithmic surgical workflow recognition is an ongoing research field and can be divided into laparoscopic (Internal) and operating room (External) analysis. So far many different works for the internal analysis have been proposed with the combination of a frame-level and an additional temporal model to address the temporal ambiguities between different workflow phases. For the External recognition task, Clip-level methods are in the focus of researchers targeting the local ambiguities present in the OR scene. In this work we evaluate combinations of different model architectures for the task of surgical workflow recognition to provide a fair comparison of the methods for both Internal and External analysis. We show that methods designed for the Internal analysis can be transferred to the external task with comparable performance gains for different architectures.