mlsanityData to 7 Oct 2026

AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions

arXiv 1705.08421 ↗First posted 23 May 2017cs.CVcited by 449

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper introduces a video dataset of spatio-temporally localized Atomic Visual Actions (AVA). The AVA dataset densely annotates 80 atomic visual actions in 430 15-minute video clips, where actions are localized in space and time, resulting in 1.58M action labels with multiple labels per person occurring frequently. The key characteristics of our dataset are: (1) the definition of atomic visual actions, rather than composite actions; (2) precise spatio-temporal annotations with possibly multiple annotations for each person; (3) exhaustive annotation of these atomic actions over 15-minute video clips; (4) people temporally linked across consecutive segments; and (5) using movies to gather a varied set of action representations. This departs from existing datasets for spatio-temporal action recognition, which typically provide sparse annotations for composite actions in short video clips. We will release the dataset publicly. AVA, with its realistic scene and action complexity, exposes the intrinsic difficulty of action recognition. To benchmark this, we present a novel approach for action localization that builds upon the current state-of-the-art methods, and demonstrates better performance on JHMDB and UCF101-24 categories. While setting a new state of the art on existing datasets, the overall results on AVA are low at 15.6% mAP, underscoring the need for developing new approaches for video understanding.

Key facts. Cited by 449 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 80; 2022: 79; 2023: 91; 2024: 86; 2025: 71; 2026: 42.

449 papers.

Sort:

By area:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  3. 2021Multiscale Vision Transformerscited by 896
  4. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  5. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  6. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  7. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  8. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  9. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  10. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  11. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  12. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
38 more
  1. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  2. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  3. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  4. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  5. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  6. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 287
  7. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  8. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  9. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  10. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  11. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  12. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  13. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  14. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  15. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  16. 2021Towards Long-Form Video Understandingcited by 146
  17. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  18. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
  19. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  20. 2022HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modelingcited by 115
  21. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  22. 2022FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessmentcited by 106
  23. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  24. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98
  25. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 93
  26. 2021Negative Sample Matters: A Renaissance of Metric Learning for Temporal Groundingcited by 90
  27. 2022Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learningcited by 90
  28. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  29. 2021MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actionscited by 82
  30. 2021Object-Region Video Transformerscited by 73
  31. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
  32. 2021Temporal Query Networks for Fine-grained Video Understandingcited by 68
  33. 2021Video as Conditional Graph Hierarchy for Multi-Granular Question Answeringcited by 68
  34. 2022BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Videocited by 66
  35. 2024CameraHMR: Aligning People with Perspectivecited by 64
  36. 2021SiamMOT: Siamese Multi-Object Trackingcited by 62
  37. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  38. 2021FineAction: A Fine-Grained Video Dataset for Temporal Action Localizationcited by 60

50 most cited of 449 All 449 (JSON)