# Retrieval of surgical phase transitions using reinforcement learning

arXiv: https://arxiv.org/abs/2208.00902 · HTML: https://mlsanity.com/paper/2208.00902/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks](https://mlsanity.com/paper/2003.10751/) (2020)
- [Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer](https://mlsanity.com/paper/2103.09712/) (2021)
- [OperA: Attention-Regularized Transformers for Surgical Phase Recognition](https://mlsanity.com/paper/2103.03873/) (2021)
- [Recognizing Surgical Activities with Recurrent Neural Networks](https://mlsanity.com/paper/1606.06329/) (2016)
- [DAISI: Database for AI Surgical Instruction](https://mlsanity.com/paper/2004.02809/) (2020)
- [Towards Generalizable Surgical Activity Recognition Using Spatial Temporal Graph Convolutional Networks](https://mlsanity.com/paper/2001.03728/) (2020)

## Built on by (most cited first, 7 of 7)

- [Scaling up self-supervised learning for improved surgical foundation models](https://mlsanity.com/paper/2501.09436/) (2025)
- [Metrics Matter in Surgical Phase Recognition](https://mlsanity.com/paper/2305.13961/) (2023)
- [Exploring the Effect of Dataset Diversity in Self-Supervised Learning for Surgical Computer Vision](https://mlsanity.com/paper/2407.17904/) (2024)
- [TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognition](https://mlsanity.com/paper/2307.09997/) (2023)
- [Surgical Temporal Action-aware Network with Sequence Regularization for Phase Recognition](https://mlsanity.com/paper/2311.12603/) (2023)
- [Robust Surgical Phase Recognition From Annotation Efficient Supervision](https://mlsanity.com/paper/2406.18481/) (2024)
- [TRCoRSurg: Temporal-Relational Co-Reasoning for Surgical Video Triplet Recognition](https://mlsanity.com/paper/2608.04606/) (2026)

## Abstract (arXiv)

In minimally invasive surgery, surgical workflow segmentation from video analysis is a well studied topic. The conventional approach defines it as a multi-class classification problem, where individual video frames are attributed a surgical phase label. We introduce a novel reinforcement learning formulation for offline phase transition retrieval. Instead of attempting to classify every video frame, we identify the timestamp of each phase transition. By construction, our model does not produce spurious and noisy phase transitions, but contiguous phase blocks. We investigate two different configurations of this model. The first does not require processing all frames in a video (only <60% and <20% of frames in 2 different applications), while producing results slightly under the state-of-the-art accuracy. The second configuration processes all video frames, and outperforms the state-of-the art at a comparable computational cost. We compare our method against the recent top-performing frame-based approaches TeCNO and Trans-SVNet on the public dataset Cholec80 and also on an in-house dataset of laparoscopic sacrocolpopexy. We perform both a frame-based (accuracy, precision, recall and F1-score) and an event-based (event ratio) evaluation of our algorithms.