mlsanityData to 7 Oct 2026

A Closer Look at Spatiotemporal Convolutions for Action Recognition

arXiv 1711.11248 ↗First posted 30 Nov 2017cs.CVcited by 1,139

Not on the map. Why?

Cited by 25 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper we discuss several forms of spatiotemporal convolutions for video analysis and study their effects on action recognition. Our motivation stems from the observation that 2D CNNs applied to individual frames of the video have remained solid performers in action recognition. In this work we empirically demonstrate the accuracy advantages of 3D CNNs over 2D CNNs within the framework of residual learning. Furthermore, we show that factorizing the 3D convolutional filters into separate spatial and temporal components yields significantly advantages in accuracy. Our empirical study leads to the design of a new spatiotemporal convolutional block "R(2+1)D" which gives rise to CNNs that achieve results comparable or superior to the state-of-the-art on Sports-1M, Kinetics, UCF101 and HMDB51.

Key facts. Cited by 1,139 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 267; 2022: 275; 2023: 195; 2024: 177; 2025: 131; 2026: 94.

1,139 papers.

Sort:

By area:

  1. 2021ViViT: A Video Vision Transformercited by 1,610
  2. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  3. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  4. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  5. 2021Video Swin Transformercited by 1,090
  6. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  7. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  8. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  9. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  10. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  11. 2022Multimodal Learning with Transformers: A Surveycited by 399
  12. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
38 more
  1. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  2. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  3. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  4. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  5. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  6. 2021Revisiting Skeleton-based Action Recognitioncited by 256
  7. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  8. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  9. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  10. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  11. 2021Anticipative Video Transformercited by 197
  12. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  13. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
  14. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  15. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  16. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  17. 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
  18. 2022Multiview Transformers for Video Recognitioncited by 148
  19. 2023AIM: Adapting Image Models for Efficient Video Action Recognitioncited by 145
  20. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  21. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  22. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  23. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  24. 2021Video Frame Interpolation Transformercited by 122
  25. 2022UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormercited by 116
  26. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  27. 2022HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modelingcited by 115
  28. 2021VidTr: Video Transformer Without Convolutionscited by 112
  29. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 111
  30. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  31. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  32. 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
  33. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  34. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  35. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
  36. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  37. 2022Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learningcited by 90
  38. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90

50 most cited of 1,139 All 1,139 (JSON)