mlsanityData to 14 Oct 2026

Action Detection via an Image Diffusion Process

arXiv 2404.01051 ↗First posted 1 Apr 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Action detection aims to localize the starting and ending points of action instances in untrimmed videos, and predict the classes of those instances. In this paper, we make the observation that the outputs of the action detection task can be formulated as images. Thus, from a novel perspective, we tackle action detection via a three-image generation process to generate starting point, ending point and action-class predictions as images via our proposed Action Detection Image Diffusion (ADI-Diff) framework. Furthermore, since our images differ from natural images and exhibit special properties, we further explore a Discrete Action-Detection Diffusion Process and a Row-Column Transformer design to better handle their processing. Our ADI-Diff framework achieves state-of-the-art results on two widely-used datasets.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 59 of its 74 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

59 of 74 references.

References · 59

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  10. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
38 more
  1. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  2. 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 382
  3. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  4. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 329
  5. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  6. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  7. 2017Temporal Action Detection with Structured Segment Networkscited by 267
  8. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 249
  9. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
  10. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  11. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  12. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  13. 2021Learning Salient Boundary Feature for Anchor-free Temporal Action Localizationcited by 156
  14. 2021End-to-end Temporal Action Detection with Transformercited by 151
  15. 2018SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videoscited by 150
  16. 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 142
  17. 2019Gaussian Temporal Awareness Networks for Action Localizationcited by 135
  18. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 117
  19. 2017TURN TAP: Temporal Unit Regression Network for Temporal Action Proposalscited by 113
  20. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 112
  21. 2020Video Self-Stitching Graph Network for Temporal Action Localizationcited by 104
  22. 2015End-to-end Learning of Action Detection from Frame Glimpses in Videoscited by 101
  23. 2022DiffPose: Toward More Reliable 3D Pose Estimationcited by 100
  24. 2019Fast Learning of Temporal Action Proposal via Dense Boundary Generatorcited by 89
  25. 2020Boundary Content Graph Neural Network for Temporal Action Proposal Generationcited by 88
  26. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 82
  27. 2018Multi-granularity Generator for Temporal Action Proposalcited by 77
  28. 2020Revisiting Anchor Mechanisms for Temporal Action Localizationcited by 75
  29. 2023Diffusion Action Segmentationcited by 74
  30. 2021Enriching Local and Global Contexts for Temporal Action Localizationcited by 64
  31. 2020Multi-shot Temporal Event Localization: a Benchmarkcited by 52
  32. 2022ReAct: Temporal Action Detection with Relational Queriescited by 48
  33. 2021MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detectioncited by 42
  34. 2021Class Semantics-based Attention for Action Detectioncited by 36
  35. 2021DCAN: Improving Temporal Action Detection via Dual Context Aggregationcited by 36
  36. 2021Deep Learning-based Action Detection in Untrimmed Videos: A Surveycited by 32
  37. 2022Proposal-Free Temporal Action Detection via Global Segmentation Mask Learningcited by 29
  38. 2023Distribution-Aligned Diffusion for Human Mesh Recoverycited by 27

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 3.

11 papers.

  1. 2025LongDiff: Training-Free Long Video Generation in One Gocited by 13
  2. 2024Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognitioncited by 9
  3. 2025Visual Prompting for One-shot Controllable Video Editing without Inversioncited by 8
  4. 2025An Image-like Diffusion Method for Human-Object Interaction Detectioncited by 5
  5. 2024UniAV: Unified Audio-Visual Perception for Multi-Task Video Event Localizationcited by 4
  6. 2025Generic Event Boundary Detection via Denoising Diffusioncited by 2
  7. 2026Towards Mitigating Modality Bias in Vision-Language Models for Temporal Action Localizationcited by 1
  8. 2026Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Trackingcited by 1
  9. 2024An Effective-Efficient Approach for Dense Multi-Label Action Detectioncited by 0
  10. 2025Action-Dynamics Modeling and Cross-Temporal Interaction for Online Action Understandingcited by 0
  11. 2026Denoise and Align: Diffusion-Driven Foreground Knowledge Prompting for Open-Vocabulary Temporal Action Detectioncited by 0