mlsanityData to 14 Oct 2026

A Survey on Backbones for Deep Video Action Recognition

arXiv 2405.05584 ↗First posted 9 May 2024cs.CV cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Action recognition is a key technology in building interactive metaverses. With the rapid development of deep learning, methods in action recognition have also achieved great advancement. Researchers design and implement the backbones referring to multiple standpoints, which leads to the diversity of methods and encountering new challenges. This paper reviews several action recognition methods based on deep neural networks. We introduce these methods in three parts: 1) Two-Streams networks and their variants, which, specifically in this paper, use RGB video frame and optical flow modality as input; 2) 3D convolutional networks, which make efforts in taking advantage of RGB modality directly while extracting different motion information is no longer necessary; 3) Transformer-based methods, which introduce the model from natural language processing into computer vision and video understanding. We offer objective sights in this review and hopefully provide a reference for future research.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 37 of its 46 references that are in the map. First posted on arXiv 9 May 2024.

Builds on

37 of 46 references.

References · 37

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2014Going Deeper with Convolutionscited by 4,241
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  4. 2017Non-local Neural Networkscited by 2,293
  5. 2017The Kinetics Human Action Video Datasetcited by 1,909
  6. 2018SlowFast Networks for Video Recognitioncited by 1,867
  7. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  8. 2021ViViT: A Video Vision Transformercited by 1,620
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  10. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  11. 2021Video Swin Transformercited by 1,091
  12. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
25 more
  1. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  2. 2021Multiscale Vision Transformerscited by 897
  3. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  4. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  5. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  6. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  7. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  8. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  9. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 387
  10. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  11. 2021Video Transformer Networkcited by 272
  12. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  13. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  14. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 184
  15. 2022Multiview Transformers for Video Recognitioncited by 148
  16. 2022Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learningcited by 90
  17. 2021An Image is Worth 16x16 Words, What is a Video Worth?cited by 87
  18. 2021Space-time Mixing Attention for Video Transformercited by 84
  19. 2017ActionVLAD: Learning spatio-temporal aggregation for action classificationcited by 73
  20. 2015Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptorscited by 58
  21. 2015P-CNN: Pose-based CNN Features for Action Recognitioncited by 58
  22. 2015Contextual Action Recognition with R*CNNcited by 34
  23. 2016Deep Temporal Linear Encoding Networkscited by 27
  24. 2023Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learningcited by 11
  25. 2017Deep Local Video Feature for Action Recognitioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Boosting Skeleton-based Zero-Shot Action Recognition with Training-Free Test-Time Adaptationcited by 3
  2. 2026Video-STLayout Pre-trainingcited by 0