mlsanityData to 7 Oct 2026

CoStoDet-DDPM: Collaborative Training of Stochastic and Deterministic Models Improves Surgical Workflow Anticipation and Recognition

arXiv 2503.10216 ↗First posted 13 Mar 2025cs.CVcited by 0

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Anticipating and recognizing surgical workflows are critical for intelligent surgical assistance systems. However, existing methods rely on deterministic decision-making, struggling to generalize across the large anatomical and procedural variations inherent in real-world surgeries.In this paper, we introduce an innovative framework that incorporates stochastic modeling through a denoising diffusion probabilistic model (DDPM) into conventional deterministic learning for surgical workflow analysis. At the heart of our approach is a collaborative co-training paradigm: the DDPM branch captures procedural uncertainties to enrich feature representations, while the task branch focuses on predicting surgical phases and instrument usage.Theoretically, we demonstrate that this mutual refinement mechanism benefits both branches: the DDPM reduces prediction errors in uncertain scenarios, and the task branch directs the DDPM toward clinically meaningful representations. Notably, the DDPM branch is discarded during inference, enabling real-time predictions without sacrificing accuracy.Experiments on the Cholec80 dataset show that for the anticipation task, our method achieves a 16% reduction in eMAE compared to state-of-the-art approaches, and for phase recognition, it improves the Jaccard score by 1.0%. Additionally, on the AutoLaparo dataset, our method achieves a 1.5% improvement in the Jaccard score for phase recognition, while also exhibiting robust generalization to patient-specific variations. Our code and weight are available at https://github.com/kk42yy/CoStoDet-DDPM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 48 of its 70 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

48 of 70 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  12. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
36 more
  1. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  3. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  4. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  5. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  6. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 366
  7. 2022Understanding Diffusion Models: A Unified Perspectivecited by 337
  8. 2021Anticipative Video Transformercited by 197
  9. 2020TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networkscited by 139
  10. 2017Surgical Data Science: Enabling Next-Generation Surgerycited by 126
  11. 2020Surgical Data Science -- from Concepts toward Clinical Translationcited by 105
  12. 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 93
  13. 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
  14. 2019Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysiscited by 75
  15. 2023Diffusion Action Segmentationcited by 74
  16. 2021Temporal Memory Relation Network for Workflow Recognition from Surgical Videocited by 66
  17. 2021OperA: Attention-Regularized Transformers for Surgical Phase Recognitioncited by 60
  18. 2022Real-time Online Video Detection with Temporal Smoothing Transformerscited by 58
  19. 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
  20. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 37
  21. 2019Using 3D Convolutional Neural Networks to Learn Spatiotemporal Features for Automatic Surgical Gesture Recognition in Videocited by 36
  22. 2024Unveil Inversion and Invariance in Flow Transformer for Versatile Image Editingcited by 34
  23. 2024Surgformer: Surgical Transformer with Hierarchical Temporal Attention for Surgical Phase Recognitioncited by 32
  24. 2024OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understandingcited by 31
  25. 2024FlexiEdit: Frequency-Aware Latent Refinement for Enhanced Non-Rigid Editingcited by 31
  26. 2018RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotationscited by 26
  27. 2024GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understandingcited by 26
  28. 2018Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networkscited by 24
  29. 2020Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Searchcited by 23
  30. 2022On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysiscited by 19
  31. 2021CataNet: Predicting remaining cataract surgery durationcited by 15
  32. 2020Rethinking Anticipation Tasks: Uncertainty-aware Anticipation of Sparse Surgical Instrument Usage for Context-aware Assistancecited by 13
  33. 2021SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgerycited by 7
  34. 2022Towards Graph Representation Learning Based Surgical Workflow Anticipationcited by 6
  35. 2024DACAT: Dual-stream Adaptive Clip-aware Time Modeling for Robust Online Surgical Phase Recognitioncited by 3
  36. 2024Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Comparative Study in Surgical AI: Potential and Limitations of Data, Compute, and Scaling
  2. 2025Recognizing Surgical Phases Anywhere: Few-Shot Test-time Adaptation and Task-graph Guided Refinement
  3. 2026Surgical Video Generation From Diffusion to World Models: A Survey
  4. 2026Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning
  5. 2025Holistic Surgical Phase Recognition with Hierarchical Input Dependent State Space Models
  6. 2026Scaling Video Pretraining for Surgical Foundation Models
  7. 2025Towards Suturing World Models: Learning Predictive Models for Robotic Surgical Tasks
  8. 2025Mitigating Surgical Data Imbalance with Dual-Prediction Video Diffusion Model