mlsanityData to 14 Oct 2026

Diffusion Action Segmentation

arXiv 2303.17959 ↗First posted 31 Mar 2023cs.CV eess.IVcited by 74

Not on the map. Why?

Abstract (arXiv)

Temporal action segmentation is crucial for understanding long-form videos. Previous works on this task commonly adopt an iterative refinement paradigm by using multi-stage models. We propose a novel framework via denoising diffusion models, which nonetheless shares the same inherent spirit of such iterative refinement. In this framework, action predictions are iteratively generated from random noise with input video features as conditions. To enhance the modeling of three striking characteristics of human actions, including the position prior, the boundary ambiguity, and the relational dependency, we devise a unified masking strategy for the conditioning inputs in our framework. Extensive experiments on three benchmark datasets, i.e., GTEA, 50Salads, and Breakfast, are performed and the proposed method achieves superior or comparable results to state-of-the-art methods, showing the effectiveness of a generative approach for action segmentation.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 49 of its 79 references that are in the map. First posted on arXiv 31 Mar 2023.

Builds on

49 of 79 references.

References · 49

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  10. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  11. 2022Diffusion Models in Vision: A Surveycited by 1,015
  12. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
37 more
  1. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 508
  2. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  3. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  4. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
  5. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  6. 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
  7. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  8. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  9. 2022Diffusion Probabilistic Modeling for Video Generationcited by 238
  10. 2022MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Modelcited by 180
  11. 2021ASFormer: Transformer for Action Segmentationcited by 168
  12. 2022Diffusion Models for Video Prediction and Infillingcited by 135
  13. 2022Person Image Synthesis via Denoising Diffusion Modelcited by 104
  14. 2020Alleviating Over-segmentation Errors by Detecting Action Boundariescited by 87
  15. 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
  16. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  17. 2016Segmental Spatiotemporal CNNs for Fine-grained Action Segmentationcited by 78
  18. 2022Deep Model Reassemblycited by 78
  19. 2020Action Segmentation with Joint Self-Supervised Temporal Domain Adaptationcited by 74
  20. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  21. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesiscited by 64
  22. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  23. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
  24. 2021Towards Unified Surgical Skill Assessmentcited by 49
  25. 2022BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesiscited by 44
  26. 2021Global2Local: Efficient Structure Search for Video Action Segmentationcited by 42
  27. 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
  28. 2021Action Segmentation with Mixed Temporal Domain Adaptationcited by 21
  29. 2022Refined Semantic Enhancement towards Frequency Diffusion for Video Captioningcited by 18
  30. 2021FIFA: Fast Inference Approximation for Action Segmentationcited by 16
  31. 2017TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentationcited by 14
  32. 2022Cross-Enhancement Transformer for Action Segmentationcited by 14
  33. 2022Do we really need temporal convolutions in action segmentation?cited by 10
  34. 2018Learning Motion in Feature Space: Locally-Consistent Deformable Convolution Networks for Fine-Grained Action Detectioncited by 9
  35. 2019Coupled Generative Adversarial Network for Continuous Fine-grained Action Segmentationcited by 7
  36. 2022Diffusing Surrogate Dreams of Video Scenes to Predict Video Memorabilitycited by 4
  37. 2023Temporal Segment Transformer for Action Segmentationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 23; 2025: 22; 2026: 14.

74 papers.

Sort:

By area:

  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  2. 2023A Survey on Video Diffusion Modelscited by 206
  3. 2023Structural Pruning for Diffusion Modelscited by 195
  4. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 70
  5. 2023Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairscited by 17
  6. 2024Efficient and Effective Weakly-Supervised Action Segmentation via Action-Transition-Aware Boundary Alignmentcited by 13
  7. 2024Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Modelscited by 12
  8. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  9. 2024Action Detection via an Image Diffusion Processcited by 11
  10. 2024Training-free Composite Scene Generation for Layout-to-Image Synthesiscited by 11
  11. 2024Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Groundingcited by 10
  12. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Modelscited by 10
38 more
  1. 2024Efficient Temporal Action Segmentation via Boundary-aware Query Votingcited by 10
  2. 2023DiffAnt: Diffusion Models for Action Anticipationcited by 9
  3. 2023A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentationcited by 8
  4. 2024OnlineTAS: An Online Baseline for Temporal Action Segmentationcited by 7
  5. 2023Boosting Diffusion Models with an Adaptive Momentum Samplercited by 6
  6. 2024Gated Temporal Diffusion for Stochastic Long-Term Dense Anticipationcited by 6
  7. 2025What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learningcited by 6
  8. 2025Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentationcited by 6
  9. 2023SF-TMN: SlowFast Temporal Modeling Network for Surgical Phase Recognitioncited by 5
  10. 2023A Survey on Deep Learning-based Spatio-temporal Action Detectioncited by 5
  11. 2024Diffusing in Someone Else's Shoes: Robotic Perspective Taking with Diffusioncited by 4
  12. 2024Surgical Triplet Recognition via Diffusion Modelcited by 4
  13. 2024ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipationcited by 4
  14. 2025Condensing Action Segmentation Datasets via Generative Network Inversioncited by 4
  15. 2025SummDiff: Generative Modeling of Video Summarization with Diffusioncited by 4
  16. 2024An Analysis on Quantizing Diffusion Transformerscited by 3
  17. 2024Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustmentcited by 3
  18. 2023DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detectioncited by 2
  19. 2023Diffusion-Based Particle-DETR for BEV Perceptioncited by 2
  20. 2024Robotic Imitation of Human Actionscited by 2
  21. 2024Con4m: Context-aware Consistency Learning Framework for Segmented Time Series Classificationcited by 2
  22. 2024Faster Diffusion Action Segmentationcited by 2
  23. 2024Language-Assisted Human Part Motion Learning for Skeleton-Based Temporal Action Segmentationcited by 2
  24. 2025PointDiffuse: A Dual-Conditional Diffusion Model for Enhanced Point Cloud Semantic Segmentationcited by 2
  25. 2025Text-Derived Relational Graph-Enhanced Network for Skeleton-Based Action Segmentationcited by 2
  26. 2025Generic Event Boundary Detection via Denoising Diffusioncited by 2
  27. 2026ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarioscited by 2
  28. 2026The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detectioncited by 2
  29. 2023End-to-End Streaming Video Temporal Action Segmentation with Reinforce Learningcited by 1
  30. 2023SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentationcited by 1
  31. 2023SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentationcited by 1
  32. 2024Feature Fusion for Human Activity Recognition using Parameter-Optimized Multi-Stage Graph Convolutional Network and Transformer Modelscited by 1
  33. 2024Temporal2Seq: A Unified Framework for Temporal Video Understanding Taskscited by 1
  34. 2025Localization-Aware Multi-Scale Representation Learning for Repetitive Action Countingcited by 1
  35. 2025End-to-End Action Segmentation Transformercited by 1
  36. 2025Exploring Ordinal Bias in Action Recognition for Instructional Videoscited by 1
  37. 2025M2R2: MultiModal Robotic Representation for Temporal Action Segmentationcited by 1
  38. 2025Looking into the Unknown: Exploring Action Discovery for Segmentation of Known and Unknown Actionscited by 1

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion
  2. 2024ActionDiffusion: An Action-aware Diffusion Model for Procedure Planning in Instructional Videos
  3. 2022Flexible Diffusion Modeling of Long Videos
  4. 2025HopaDIFF: Holistic-Partial Aware Fourier Conditioned Diffusion for Referring Human Action Segmentation in Multi-Person Scenarios
  5. 2026Masked Diffusion Vision-Language Models for Temporal Action Localization
  6. 2025Target-Aware Video Diffusion Models
  7. 2024AICL: Action In-Context Learning for Video Diffusion Model
  8. 2024Progressive Autoregressive Video Diffusion Models