mlsanityData to 7 Oct 2026

EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos

arXiv 1602.03012 ↗First posted 9 Feb 2016cs.CVcited by 366

Not on the map. Why?

Cited by 25 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Surgical workflow recognition has numerous potential medical applications, such as the automatic indexing of surgical video databases and the optimization of real-time operating room scheduling, among others. As a result, phase recognition has been studied in the context of several kinds of surgeries, such as cataract, neurological, and laparoscopic surgeries. In the literature, two types of features are typically used to perform this task: visual features and tool usage signals. However, the visual features used are mostly handcrafted. Furthermore, the tool usage signals are usually collected via a manual annotation process or by using additional equipment. In this paper, we propose a novel method for phase recognition that uses a convolutional neural network (CNN) to automatically learn features from cholecystectomy videos and that relies uniquely on visual information. In previous studies, it has been shown that the tool signals can provide valuable information in performing the phase recognition task. Thus, we present a novel CNN architecture, called EndoNet, that is designed to carry out the phase recognition and tool presence detection tasks in a multi-task manner. To the best of our knowledge, this is the first work proposing to use a CNN for multiple recognition tasks on laparoscopic videos. Extensive experimental comparisons to other methods show that EndoNet yields state-of-the-art results for both tasks.

Key facts. Cited by 366 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 35; 2023: 52; 2024: 68; 2025: 98; 2026: 77.

366 papers.

Sort:

By area:

  1. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
  2. 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 93
  3. 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
  4. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  5. 2021Temporal Memory Relation Network for Workflow Recognition from Surgical Videocited by 66
  6. 2021OperA: Attention-Regularized Transformers for Surgical Phase Recognitioncited by 60
  7. 2021Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmarkcited by 58
  8. 2022Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformercited by 58
  9. 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
  10. 2022Dissecting Self-Supervised Learning Methods for Surgical Computer Visioncited by 48
  11. 2023SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgerycited by 48
  12. 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 48
38 more
  1. 2022CholecTriplet2021: A benchmark challenge for surgical action triplet recognitioncited by 44
  2. 2021Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedurescited by 42
  3. 2023Advancing Surgical VQA with Scene Graph Knowledgecited by 41
  4. 2024HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognitioncited by 41
  5. 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
  6. 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentationcited by 38
  7. 2021The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methodscited by 37
  8. 2025EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgerycited by 37
  9. 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learningcited by 34
  10. 2022Towards Holistic Surgical Scene Understandingcited by 33
  11. 2024Surgformer: Surgical Transformer with Hierarchical Temporal Attention for Surgical Phase Recognitioncited by 32
  12. 2024OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understandingcited by 31
  13. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 30
  14. 2025MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environmentscited by 28
  15. 2021MIcro-Surgical Anastomose Workflow recognition challenge reportcited by 27
  16. 2022Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognitioncited by 27
  17. 2024General surgery vision transformer: A video pre-trained foundation model for general surgerycited by 25
  18. 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
  19. 2023What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentationcited by 24
  20. 2023Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectivescited by 24
  21. 2021Gesture Recognition in Robotic Surgery: a Reviewcited by 23
  22. 2021Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videoscited by 23
  23. 2022Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasetscited by 23
  24. 2023Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactionscited by 23
  25. 2023CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detectioncited by 22
  26. 2023Self-Supervised Learning for Endoscopic Video Analysiscited by 22
  27. 2023LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Roomscited by 21
  28. 2023Dynamic Scene Graph Representation for Surgical Videocited by 21
  29. 2023The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmarkcited by 21
  30. 2021Not End-to-End: Explore Multi-Stage Architecture for Online Surgical Phase Recognitioncited by 20
  31. 2022FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentationcited by 20
  32. 2024EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videoscited by 20
  33. 2022On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysiscited by 19
  34. 2024SurGen: Text-Guided Diffusion Model for Surgical Video Generationcited by 19
  35. 2022Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phasescited by 18
  36. 2023Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the artcited by 18
  37. 2025Datasets and Recipes for Video Temporal Grounding via Reinforcement Learningcited by 18
  38. 2023CholecTrack20: A Multi-Perspective Tracking Dataset for Surgical Toolscited by 17

50 most cited of 366 All 366 (JSON)