mlsanityData to 14 Oct 2026

Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer

arXiv 2103.09712 ↗First posted 17 Mar 2021cs.CV cs.AIcited by 94

Not on the map. Why?

Abstract (arXiv)

Real-time surgical phase recognition is a fundamental task in modern operating rooms. Previous works tackle this task relying on architectures arranged in spatio-temporal order, however, the supportive benefits of intermediate spatial features are not considered. In this paper, we introduce, for the first time in surgical workflow analysis, Transformer to reconsider the ignored complementary effects of spatial and temporal features for accurate surgical phase recognition. Our hybrid embedding aggregation Transformer fuses cleverly designed spatial and temporal embeddings by allowing for active queries based on spatial information from temporal embedding sequences. More importantly, our framework processes the hybrid embeddings in parallel to achieve a high inference speed. Our method is thoroughly validated on two large surgical video datasets, i.e., Cholec80 and M2CAI16 Challenge datasets, and outperforms the state-of-the-art approaches at a processing speed of 91 fps.

Key facts. Cited by 94 papers in the map (data through 14 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

21 of 33 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2016Layer Normalizationcited by 5,669
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  5. 2021Transformers in Vision: A Surveycited by 1,013
  6. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 508
  7. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
  8. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 367
  9. 2019Deep Closest Point: Learning Representations for Point Cloud Registrationcited by 321
  10. 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
  11. 2020TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networkscited by 141
  12. 2017Surgical Data Science: Enabling Next-Generation Surgerycited by 126
9 more
  1. 2019Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysiscited by 76
  2. 2019Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Videocited by 64
  3. 2018DeepPhase: Surgical Phase Recognition in CATARACTS Videoscited by 47
  4. 2019Using 3D Convolutional Neural Networks to Learn Spatiotemporal Features for Automatic Surgical Gesture Recognition in Videocited by 36
  5. 2020Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Videocited by 35
  6. 2020Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Searchcited by 23
  7. 2016Single- and Multi-Task Architectures for Surgical Workflow Challenge at M2CAI 2016cited by 19
  8. 2020Symmetric Dilated Convolution for Surgical Gesture Recognitioncited by 11
  9. 2016Real-time analysis of cataract surgery videos using statistical modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 18; 2023: 15; 2024: 20; 2025: 19; 2026: 19.

94 papers.

Sort:

By area:

  1. 2022Transformers in Medical Imaging: A Surveycited by 245
  2. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
  3. 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
  4. 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
  5. 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 56
  6. 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 49
  7. 2022TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgerycited by 40
  8. 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentationcited by 39
  9. 2022Towards Holistic Surgical Scene Understandingcited by 33
  10. 2024Surgformer: Surgical Transformer with Hierarchical Temporal Attention for Surgical Phase Recognitioncited by 32
  11. 2022Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentationcited by 27
  12. 2022Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognitioncited by 27
38 more
  1. 2021Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videoscited by 23
  2. 2022Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasetscited by 23
  3. 2023CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detectioncited by 23
  4. 2023LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Roomscited by 21
  5. 2023Dynamic Scene Graph Representation for Surgical Videocited by 21
  6. 2021Not End-to-End: Explore Multi-Stage Architecture for Online Surgical Phase Recognitioncited by 20
  7. 2024EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videoscited by 20
  8. 2022On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysiscited by 19
  9. 2022Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phasescited by 18
  10. 2025SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learningcited by 16
  11. 2023Metrics Matter in Surgical Phase Recognitioncited by 14
  12. 2023Encoding Surgical Videos as Latent Spatiotemporal Graphs for Object and Anatomy-Driven Reasoningcited by 14
  13. 2022Less is More: Surgical Phase Recognition from Timestamp Supervisioncited by 13
  14. 2022Recognition and Prediction of Surgical Gestures and Trajectories Using Transformer Models in Robot-Assisted Surgerycited by 12
  15. 2023SurgPLAN: Surgical Phase Localization Network for Phase Recognitioncited by 12
  16. 2024VidLPRO: A underlineVideo-underlineLanguage underlinePre-training Framework for underlineRobotic and Laparoscopic Surgerycited by 12
  17. 2025EndoMamba: An Efficient Foundation Model for Endoscopic Videos via Hierarchical Pre-trainingcited by 11
  18. 2023Self-distillation for surgical action recognitioncited by 9
  19. 2022Free Lunch for Surgical Video Understanding by Distilling Self-Supervisionscited by 8
  20. 2024MuST: Multi-Scale Transformers for Surgical Phase Recognitioncited by 8
  21. 2025Comparative validation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in endoscopy: Results of the PhaKIR 2024 challengecited by 8
  22. 2022Retrieval of surgical phase transitions using reinforcement learningcited by 7
  23. 2025Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registrationcited by 7
  24. 2022PEg TRAnsfer Workflow recognition challenge report: Does multi-modal data improve recognition?cited by 6
  25. 2022Surgical Workflow Recognition: from Analysis of Challenges to Architectural Studycited by 6
  26. 2022ARST: Auto-Regressive Surgical Transformer for Phase Recognition from Laparoscopic Videoscited by 6
  27. 2023TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognitioncited by 6
  28. 2023Surgical Temporal Action-aware Network with Sequence Regularization for Phase Recognitioncited by 6
  29. 2024Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Roomscited by 6
  30. 2024SR-Mamba: Effective Surgical Phase Recognition with State Space Modelcited by 6
  31. 2023GLSFormer: Gated - Long, Short Sequence Transformer for Step Recognition in Surgical Videoscited by 5
  32. 2024Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgerycited by 5
  33. 2024Neural Finite-State Machines for Surgical Phase Recognitioncited by 5
  34. 2024SWAG: Long-term Surgical Workflow Prediction with Generative-based Anticipationcited by 4
  35. 2022MURPHY: Relations Matter in Surgical Workflow Analysiscited by 3
  36. 2024SEDMamba: Enhancing Selective State Space Modelling with Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgerycited by 3
  37. 2024DACAT: Dual-stream Adaptive Clip-aware Time Modeling for Robust Online Surgical Phase Recognitioncited by 3
  38. 2024SPRMamba: Surgical Phase Recognition for Endoscopic Submucosal Dissection with Mambacited by 3

50 most cited of 94 All 94 (JSON)