# The AVA-Kinetics Localized Human Actions Video Dataset

arXiv: https://arxiv.org/abs/2005.00214 · HTML: https://mlsanity.com/paper/2005.00214/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 30 of 82)

- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://mlsanity.com/paper/2212.03191/) (2022)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations](https://mlsanity.com/paper/2209.13064/) (2022)
- [VideoPrism: A Foundational Visual Encoder for Video Understanding](https://mlsanity.com/paper/2402.13217/) (2024)
- [Self-Supervised Learning for Videos: A Survey](https://mlsanity.com/paper/2207.00419/) (2022)
- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [FineAction: A Fine-Grained Video Dataset for Temporal Action Localization](https://mlsanity.com/paper/2105.11107/) (2021)
- [Visual Semantic Role Labeling for Video Understanding](https://mlsanity.com/paper/2104.00990/) (2021)
- [Pixel-Wise Recognition for Holistic Surgical Scene Understanding](https://mlsanity.com/paper/2401.11174/) (2024)
- [Foundation Models for Video Understanding: A Survey](https://mlsanity.com/paper/2405.03770/) (2024)
- [Spotting Temporally Precise, Fine-Grained Events in Video](https://mlsanity.com/paper/2207.10213/) (2022)
- [JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection](https://mlsanity.com/paper/2106.08827/) (2021)
- [On the Benefits of 3D Pose and Tracking for Human Action Recognition](https://mlsanity.com/paper/2304.01199/) (2023)
- [Masked Image Modeling: A Survey](https://mlsanity.com/paper/2408.06687/) (2024)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [Semi-TCL: Semi-Supervised Track Contrastive Representation Learning](https://mlsanity.com/paper/2107.02396/) (2021)
- [Accurate and Fast Compressed Video Captioning](https://mlsanity.com/paper/2309.12867/) (2023)
- [End-to-End Spatio-Temporal Action Localisation with Video Transformers](https://mlsanity.com/paper/2304.12160/) (2023)
- [What to Say and When to Say it: Live Fitness Coaching as a Testbed for Situated Interaction](https://mlsanity.com/paper/2407.08101/) (2024)
- [Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks](https://mlsanity.com/paper/2501.08326/) (2025)
- [Contextualized Spatio-Temporal Contrastive Learning with Self-Supervision](https://mlsanity.com/paper/2112.05181/) (2021)
- [SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videos](https://mlsanity.com/paper/2404.04565/) (2024)
- [VideoGLUE: Video General Understanding Evaluation of Foundation Models](https://mlsanity.com/paper/2307.03166/) (2023)
- [BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos](https://mlsanity.com/paper/2405.17698/) (2024)
- [Common Ground Tracking in Multimodal Dialogue](https://mlsanity.com/paper/2403.17284/) (2024)
- [Discrete neural representations for explainable anomaly detection](https://mlsanity.com/paper/2112.05585/) (2021)
- [Movies2Scenes: Using Movie Metadata to Learn Scene Representation](https://mlsanity.com/paper/2202.10650/) (2022)
- [X-MAN: Explaining multiple sources of anomalies in video](https://mlsanity.com/paper/2106.08856/) (2021)
- [What, when, and where? -- Self-Supervised Spatio-Temporal Grounding in Untrimmed Multi-Action Videos from Narrated Instructions](https://mlsanity.com/paper/2303.16990/) (2023)

## Abstract (arXiv)

This paper describes the AVA-Kinetics localized human actions video dataset. The dataset is collected by annotating videos from the Kinetics-700 dataset using the AVA annotation protocol, and extending the original AVA dataset with these new AVA annotated Kinetics clips. The dataset contains over 230k clips annotated with the 80 AVA action classes for each of the humans in key-frames. We describe the annotation process and provide statistics about the new dataset. We also include a baseline evaluation using the Video Action Transformer Network on the AVA-Kinetics dataset, demonstrating improved performance for action classification on the AVA test set. The dataset can be downloaded from https://research.google.com/ava/