# LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition

arXiv: https://arxiv.org/abs/2004.09845 · HTML: https://mlsanity.com/paper/2004.09845/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding](https://mlsanity.com/paper/2406.07471/) (2024)
- [On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysis](https://mlsanity.com/paper/2203.07976/) (2022)
- [Less is More: Surgical Phase Recognition from Timestamp Supervision](https://mlsanity.com/paper/2202.08199/) (2022)
- [Correlation-aware active learning for surgery video segmentation](https://mlsanity.com/paper/2311.08811/) (2023)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [Robust Surgical Phase Recognition From Annotation Efficient Supervision](https://mlsanity.com/paper/2406.18481/) (2024)

## Abstract (arXiv)

Automatic surgical workflow recognition in video is an essentially fundamental yet challenging problem for developing computer-assisted and robotic-assisted surgery. Existing approaches with deep learning have achieved remarkable performance on analysis of surgical videos, however, heavily relying on large-scale labelled datasets. Unfortunately, the annotation is not often available in abundance, because it requires the domain knowledge of surgeons. In this paper, we propose a novel active learning method for cost-effective surgical video analysis. Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency (LRTD) among continuous frames. We then formulate an intra-clip dependency score to represent the overall dependency within this clip. By ranking scores among clips in unlabelled data pool, we select the clips with weak dependencies to annotate, which indicates the most informative ones to better benefit network training. We validate our approach on a large surgical video dataset (Cholec80) by performing surgical workflow recognition task. By using our LRTD based selection strategy, we can outperform other state-of-the-art active learning methods. Using only up to 50% of samples, our approach can exceed the performance of full-data training.