mlsanityData to 14 Oct 2026

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

arXiv 1705.07750 ↗First posted 22 May 2017cs.CV cs.LGcited by 3,270

Not on the map. Why?

Cited by 98 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The paucity of videos in current action classification datasets (UCF-101 and HMDB-51) has made it difficult to identify good video architectures, as most methods obtain similar performance on existing small-scale benchmarks. This paper re-evaluates state-of-the-art architectures in light of the new Kinetics Human Action Video dataset. Kinetics has two orders of magnitude more data, with 400 human action classes and over 400 clips per class, and is collected from realistic, challenging YouTube videos. We provide an analysis on how current architectures fare on the task of action classification on this dataset and how much performance improves on the smaller benchmark datasets after pre-training on Kinetics. We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on 2D ConvNet inflation: filters and pooling kernels of very deep image classification ConvNets are expanded into 3D, making it possible to learn seamless spatio-temporal feature extractors from video while leveraging successful ImageNet architecture designs and even their parameters. We show that, after pre-training on Kinetics, I3D models considerably improve upon the state-of-the-art in action classification, reaching 80.9% on HMDB-51 and 98.0% on UCF-101.

Key facts. Cited by 3,270 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 623; 2022: 677; 2023: 611; 2024: 558; 2025: 487; 2026: 312.

3,270 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2022Video Diffusion Modelscited by 2,305
  4. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  5. 2021ViViT: A Video Vision Transformercited by 1,620
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  8. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  9. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  10. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  11. 2021Video Swin Transformercited by 1,091
  12. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
38 more
  1. 2021Multiscale Vision Transformerscited by 897
  2. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  3. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  4. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  5. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  6. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  7. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  8. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  9. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  10. 2023CoTracker: It is Better to Track Togethercited by 522
  11. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  12. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  13. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  14. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  15. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  16. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  17. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  18. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  19. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  20. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  21. 2023Emergent Correspondence from Image Diffusioncited by 446
  22. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  23. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  24. 2021Dynamic Neural Networks: A Surveycited by 408
  25. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  26. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  27. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  28. 2022MAGVIT: Masked Generative Video Transformercited by 346
  29. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 328
  30. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  31. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
  32. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  33. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  34. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  35. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  36. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 301
  37. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  38. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287

50 most cited of 3,270 All 3,270 (JSON)