# RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations

arXiv: https://arxiv.org/abs/1802.03243 · HTML: https://mlsanity.com/paper/1802.03243/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2018.

## Built on by (most cited first, 26 of 26)

- [Temporal Memory Relation Network for Workflow Recognition from Surgical Video](https://mlsanity.com/paper/2103.16327/) (2021)
- [OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding](https://mlsanity.com/paper/2406.07471/) (2024)
- [Scaling up self-supervised learning for improved surgical foundation models](https://mlsanity.com/paper/2501.09436/) (2025)
- [On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysis](https://mlsanity.com/paper/2203.07976/) (2022)
- [CholecInstanceSeg: A Tool Instance Segmentation Dataset for Laparoscopic Surgery](https://mlsanity.com/paper/2406.16039/) (2024)
- [CataNet: Predicting remaining cataract surgery duration](https://mlsanity.com/paper/2106.11048/) (2021)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Preserving Privacy in Surgical Video Analysis Using Artificial Intelligence: A Deep Learning Classifier to Identify Out-of-Body Scenes in Endoscopic Videos](https://mlsanity.com/paper/2301.07053/) (2023)
- [Cataract-1K: Cataract Surgery Dataset for Scene Segmentation, Phase Recognition, and Irregularity Detection](https://mlsanity.com/paper/2312.06295/) (2023)
- [SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery](https://mlsanity.com/paper/2105.04642/) (2021)
- [Towards Graph Representation Learning Based Surgical Workflow Anticipation](https://mlsanity.com/paper/2208.03824/) (2022)
- [Bounded Future MS-TCN++ for surgical gesture recognition](https://mlsanity.com/paper/2209.14647/) (2022)
- [Multitask Learning in Minimally Invasive Surgical Vision: A Review](https://mlsanity.com/paper/2401.08256/) (2024)
- [Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery](https://mlsanity.com/paper/2402.01974/) (2024)
- [Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence](https://mlsanity.com/paper/2504.02799/) (2025)
- [SWAG: Long-term Surgical Workflow Prediction with Generative-based Anticipation](https://mlsanity.com/paper/2412.18849/) (2024)
- [PitRSDNet: Predicting Intra-operative Remaining Surgery Duration in Endoscopic Pituitary Surgery](https://mlsanity.com/paper/2409.16998/) (2024)
- [Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation](https://mlsanity.com/paper/2412.06454/) (2024)
- [Is there progress in activity progress prediction?](https://mlsanity.com/paper/2308.05533/) (2023)
- [Multiview Progress Prediction of Robot Activities](https://mlsanity.com/paper/2603.00151/) (2026)
- [Deep-Learning-Assisted Analysis of Cataract Surgery Videos](https://mlsanity.com/paper/2312.05900/) (2023)
- [CoStoDet-DDPM: Collaborative Training of Stochastic and Deterministic Models Improves Surgical Workflow Anticipation and Recognition](https://mlsanity.com/paper/2503.10216/) (2025)
- [Multi-Stage Boundary-Aware Transformer Network for Action Segmentation in Untrimmed Surgical Videos](https://mlsanity.com/paper/2504.18756/) (2025)
- [State-Change Learning for Prediction of Future Events in Endoscopic Videos](https://mlsanity.com/paper/2510.12904/) (2025)
- [Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis](https://mlsanity.com/paper/2605.26483/) (2026)
- [AI for Quality Assurance in the Operating Room](https://mlsanity.com/paper/2606.30657/) (2026)

## Abstract (arXiv)

Accurate surgery duration estimation is necessary for optimal OR planning, which plays an important role in patient comfort and safety as well as resource optimization. It is, however, challenging to preoperatively predict surgery duration since it varies significantly depending on the patient condition, surgeon skills, and intraoperative situation. In this paper, we propose a deep learning pipeline, referred to as RSDNet, which automatically estimates the remaining surgery duration (RSD) intraoperatively by using only visual information from laparoscopic videos. Previous state-of-the-art approaches for RSD prediction are dependent on manual annotation, whose generation requires expensive expert knowledge and is time-consuming, especially considering the numerous types of surgeries performed in a hospital and the large number of laparoscopic videos available. A crucial feature of RSDNet is that it does not depend on any manual annotation during training, making it easily scalable to many kinds of surgeries. The generalizability of our approach is demonstrated by testing the pipeline on two large datasets containing different types of surgeries: 120 cholecystectomy and 170 gastric bypass videos. The experimental results also show that the proposed network significantly outperforms a traditional method of estimating RSD without utilizing manual annotation. Further, this work provides a deeper insight into the deep learning network through visualization and interpretation of the features that are automatically learned.