mlsanityData to 14 Oct 2026

A Circular Window-based Cascade Transformer for Online Action Detection

arXiv 2208.14209 ↗First posted 30 Aug 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. Codes will be available after acceptance.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 76 of its 110 references that are in the map. First posted on arXiv 30 Aug 2022.

Builds on

76 of 110 references.

References · 76

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  11. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  12. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
38 more
  1. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  2. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  3. 2017The Kinetics Human Action Video Datasetcited by 1,909
  4. 2018SlowFast Networks for Video Recognitioncited by 1,867
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  6. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  7. 2021ViViT: A Video Vision Transformercited by 1,620
  8. 2020Rethinking Attention with Performerscited by 1,601
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  10. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  11. 2018Self-Attention with Relative Position Representationscited by 1,120
  12. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  13. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  14. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 941
  15. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  16. 2018Image Transformercited by 772
  17. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  18. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 508
  19. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
  20. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
  21. 2019Axial Attention in Multidimensional Transformerscited by 358
  22. 2021Random Feature Attentioncited by 354
  23. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 329
  24. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  25. 2021Video Transformer Networkcited by 272
  26. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  27. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 249
  28. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  29. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
  30. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  31. 2020Synthesizer: Rethinking Self-Attention in Transformer Modelscited by 205
  32. 2020Dense Regression Network for Video Groundingcited by 176
  33. 2021ASFormer: Transformer for Action Segmentationcited by 168
  34. 2021Learning Salient Boundary Feature for Anchor-free Temporal Action Localizationcited by 156
  35. 2019Gaussian Temporal Awareness Networks for Action Localizationcited by 135
  36. 2017Single Shot Temporal Action Detectioncited by 123
  37. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 117
  38. 2017CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videoscited by 112

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 1.

4 papers.

  1. 2023Memory-and-Anticipation Transformer for Online Action Understandingcited by 39
  2. 2024CYCLO: Cyclic Graph Transformer Approach to Multi-Object Relationship Modeling in Aerial Videoscited by 11
  3. 2022Learning Point-Language Hierarchical Alignment for 3D Visual Groundingcited by 6
  4. 2025Driving behavior recognition via self-discovery learningcited by 0