mlsanityData to 7 Oct 2026

Adaptive Perception Transformer for Temporal Action Localization

arXiv 2208.11908 ↗First posted 25 Aug 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Temporal action localization aims to predict the boundary and category of each action instance in untrimmed long videos. Most of previous methods based on anchors or proposals neglect the global-local context interaction in entire video sequences. Besides, their multi-stage designs cannot generate action boundaries and categories straightforwardly. To address the above issues, this paper proposes a end-to-end model, called Adaptive Perception transformer (AdaPerFormer for short). Specifically, AdaPerFormer explores a dual-branch attention mechanism. One branch takes care of the global perception attention, which can model entire video sequences and aggregate global relevant contexts. While the other branch concentrates on the local convolutional shift to aggregate intra-frame and inter-frame information through our bidirectional shift operation. The end-to-end nature produces the boundaries and categories of video actions without extra steps. Extensive experiments together with ablation studies are provided to reveal the effectiveness of our design. Our method obtains competitive performance on the THUMOS14 and ActivityNet-1.3 dataset.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 53 of its 63 references that are in the map. First posted on arXiv 25 Aug 2022.

Builds on

53 of 63 references.

References · 53

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  10. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  11. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  12. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
38 more
  1. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  2. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
  3. 2021Early Convolutions Help Transformers See Bettercited by 395
  4. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  5. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 327
  6. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  7. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  8. 2017Temporal Action Detection with Structured Segment Networkscited by 265
  9. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 248
  10. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
  11. 2020Understanding the Difficulty of Training Transformerscited by 210
  12. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  13. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  14. 2020Dense Regression Network for Video Groundingcited by 176
  15. 2021Conformer: Local Features Coupling Global Representations for Visual Recognitioncited by 164
  16. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  17. 2015Sequence to Sequence -- Video to Textcited by 157
  18. 2021Learning Salient Boundary Feature for Anchor-free Temporal Action Localizationcited by 156
  19. 2021End-to-end Temporal Action Detection with Transformercited by 150
  20. 2018Gather-Excite: Exploiting Feature Context in Convolutional Neural Networkscited by 138
  21. 2019Gaussian Temporal Awareness Networks for Action Localizationcited by 135
  22. 2017Single Shot Temporal Action Detectioncited by 123
  23. 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 116
  24. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 116
  25. 2017TURN TAP: Temporal Unit Regression Network for Temporal Action Proposalscited by 113
  26. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 111
  27. 2020Video Self-Stitching Graph Network for Temporal Action Localizationcited by 104
  28. 2016Spatiotemporal Residual Networks for Video Action Recognitioncited by 91
  29. 2019Fast Learning of Temporal Action Proposal via Dense Boundary Generatorcited by 89
  30. 2020Boundary Content Graph Neural Network for Temporal Action Proposal Generationcited by 88
  31. 2018Multi-granularity Generator for Temporal Action Proposalcited by 77
  32. 2020TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Taskscited by 77
  33. 2020Revisiting Anchor Mechanisms for Temporal Action Localizationcited by 75
  34. 2021On the Integration of Self-Attention and Convolutioncited by 74
  35. 2018Diagnosing Error in Temporal Action Detectorscited by 71
  36. 2021Enriching Local and Global Contexts for Temporal Action Localizationcited by 64
  37. 2020Multi-shot Temporal Event Localization: a Benchmarkcited by 52
  38. 2021DCAN: Improving Temporal Action Detection via Dual Context Aggregationcited by 36

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.