mlsanityData to 7 Oct 2026

Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?

arXiv 1711.09577 ↗First posted 27 Nov 2017cs.CVcited by 551

Not on the map. Why?

Abstract (arXiv)

The purpose of this study is to determine whether current video datasets have sufficient data for training very deep convolutional neural networks (CNNs) with spatio-temporal three-dimensional (3D) kernels. Recently, the performance levels of 3D CNNs in the field of action recognition have improved significantly. However, to date, conventional research has only explored relatively shallow 3D architectures. We examine the architectures of various 3D CNNs from relatively shallow to very deep ones on current video datasets. Based on the results of those experiments, the following conclusions could be obtained: (i) ResNet-18 training resulted in significant overfitting for UCF-101, HMDB-51, and ActivityNet but not for Kinetics. (ii) The Kinetics dataset has sufficient data for training of deep 3D CNNs, and enables training of up to 152 ResNets layers, interestingly similar to 2D ResNets on ImageNet. ResNeXt-101 achieved 78.4% average accuracy on the Kinetics test set. (iii) Kinetics pretrained simple 3D architectures outperforms complex 2D architectures, and the pretrained ResNeXt-101 achieved 94.5% and 70.2% on UCF-101 and HMDB-51, respectively. The use of 2D CNNs trained on ImageNet has produced significant progress in various tasks in image. We believe that using deep 3D CNNs together with Kinetics will retrace the successful history of 2D CNNs and ImageNet, and stimulate advances in computer vision for videos. The codes and pretrained models used in this study are publicly available. https://github.com/kenshohara/3D-ResNets-PyTorch

Key facts. Cited by 551 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 148; 2022: 117; 2023: 85; 2024: 81; 2025: 77; 2026: 43.

551 papers.

Sort:

By area:

  1. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  2. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  3. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  4. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  5. 2021Revisiting Skeleton-based Action Recognitioncited by 256
  6. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  7. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
  8. 2022Human-to-Robot Imitation in the Wildcited by 157
  9. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  10. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  11. 2021VidTr: Video Transformer Without Convolutionscited by 112
  12. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
38 more
  1. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
  2. 2022Generative Cooperative Learning for Unsupervised Video Anomaly Detectioncited by 102
  3. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  4. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  5. 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 84
  6. 2021Everything at Once -- Multi-modal Fusion Transformer for Video Retrievalcited by 79
  7. 2021Adaptive Focus for Efficient Video Recognitioncited by 76
  8. 2021Video as Conditional Graph Hierarchy for Multi-Granular Question Answeringcited by 68
  9. 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
  10. 2022MEDFAIR: Benchmarking Fairness for Medical Imagingcited by 66
  11. 2021ACTION-Net: Multipath Excitation for Action Recognitioncited by 63
  12. 2021iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysiscited by 63
  13. 2021Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videoscited by 62
  14. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 61
  15. 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
  16. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  17. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  18. 2021Distilling Audio-Visual Knowledge by Compositional Contrastive Learningcited by 53
  19. 2021Home Action Genome: Cooperative Compositional Action Understandingcited by 52
  20. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answeringcited by 51
  21. 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
  22. 2023MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videoscited by 46
  23. 2021MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognitioncited by 44
  24. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  25. 2023UnIVAL: Unified Model for Image, Video, Audio and Language Taskscited by 44
  26. 2021Deep Learning for Micro-expression Recognition: A Surveycited by 43
  27. 2023Prompting Visual-Language Models for Dynamic Facial Expression Recognitioncited by 43
  28. 2022Anomaly Detection-Inspired Few-Shot Medical Image Segmentation Through Self-Supervision With Supervoxelscited by 42
  29. 2023MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognitioncited by 42
  30. 2023Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challengescited by 42
  31. 2022Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wildcited by 41
  32. 2021Video Contrastive Learning with Global Contextcited by 40
  33. 2024Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Promptscited by 40
  34. 2023A Deep Neural Architecture for Harmonizing 3-D Input Data Analysis and Decision Making in Medical Imagingcited by 39
  35. 2023Black Box Few-Shot Adaptation for Vision-Language modelscited by 39
  36. 2023AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perceptioncited by 39
  37. 2024Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessmentcited by 39
  38. 2024ViViD: Video Virtual Try-on using Diffusion Modelscited by 38

50 most cited of 551 All 551 (JSON)