# HAA500: Human-Centric Atomic Action Dataset with Curated Videos

arXiv: https://arxiv.org/abs/2009.05224 · HTML: https://mlsanity.com/paper/2009.05224/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2020.

## Built on by (most cited first, 30 of 39)

- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022)
- [HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation](https://mlsanity.com/paper/2304.04269/) (2023)
- [Learning from Massive Human Videos for Universal Humanoid Pose Control](https://mlsanity.com/paper/2412.14172/) (2024)
- [MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls](https://mlsanity.com/paper/2407.21136/) (2024)
- [Scaling Large Motion Models with Million-Level Human Motions](https://mlsanity.com/paper/2410.03311/) (2024)
- [From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding](https://mlsanity.com/paper/2304.00553/) (2023)
- [HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation](https://mlsanity.com/paper/2503.24026/) (2025)
- [Referring Atomic Video Action Recognition](https://mlsanity.com/paper/2407.01872/) (2024)
- [MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotations](https://mlsanity.com/paper/2410.13790/) (2024)
- [Video Action Differencing](https://mlsanity.com/paper/2503.07860/) (2025)
- [Multi-Query Video Retrieval](https://mlsanity.com/paper/2201.03639/) (2022)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space](https://mlsanity.com/paper/2406.11253/) (2024)
- [A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction](https://mlsanity.com/paper/2502.05503/) (2025)
- [Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning](https://mlsanity.com/paper/2108.06647/) (2021)
- [MotionEdit: Benchmarking and Learning Motion-Centric Image Editing](https://mlsanity.com/paper/2512.10284/) (2025)
- [MOVE: Motion-Guided Few-Shot Video Object Segmentation](https://mlsanity.com/paper/2507.22061/) (2025)
- [HAA4D: Few-Shot Human Atomic Action Recognition via 3D Spatio-Temporal Skeletal Alignment](https://mlsanity.com/paper/2202.07308/) (2022)
- [Joint Learning On The Hierarchy Representation for Fine-Grained Human Action Recognition](https://mlsanity.com/paper/2110.05853/) (2021)
- [MOOSE: Pay Attention to Temporal Dynamics for Video Understanding via Optical Flows](https://mlsanity.com/paper/2506.01119/) (2025)
- [Toward Rich Video Human-Motion2D Generation](https://mlsanity.com/paper/2506.14428/) (2025)
- [MESH -- Understanding Videos Like Human: Measuring Hallucinations in Large Video Models](https://mlsanity.com/paper/2509.08538/) (2025)
- [Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition](https://mlsanity.com/paper/2511.03725/) (2025)
- [Seeing Without Eyes: 4D Human-Scene Understanding from Wearable IMUs](https://mlsanity.com/paper/2604.21926/) (2026)
- [O-TALC: Steps Towards Combating Oversegmentation within Online Action Segmentation](https://mlsanity.com/paper/2404.06894/) (2024)
- [Learning to Animate Images from A Few Videos to Portray Delicate Human Actions](https://mlsanity.com/paper/2503.00276/) (2025)
- [PCBEAR: Pose Concept Bottleneck for Explainable Action Recognition](https://mlsanity.com/paper/2504.13140/) (2025)
- [TxP: Reciprocal Generation of Ground Pressure Dynamics and Activity Descriptions for Improving Human Activity Recognition](https://mlsanity.com/paper/2505.02052/) (2025)

## Abstract (arXiv)

We contribute HAA500, a manually annotated human-centric atomic action dataset for action recognition on 500 classes with over 591K labeled frames. To minimize ambiguities in action classification, HAA500 consists of highly diversified classes of fine-grained atomic actions, where only consistent actions fall under the same label, e.g., "Baseball Pitching" vs "Free Throw in Basketball". Thus HAA500 is different from existing atomic action datasets, where coarse-grained atomic actions were labeled with coarse action-verbs such as "Throw". HAA500 has been carefully curated to capture the precise movement of human figures with little class-irrelevant motions or spatio-temporal label noises. The advantages of HAA500 are fourfold: 1) human-centric actions with a high average of 69.7% detectable joints for the relevant human poses; 2) high scalability since adding a new class can be done under 20-60 minutes; 3) curated videos capturing essential elements of an atomic action without irrelevant frames; 4) fine-grained atomic action classes. Our extensive experiments including cross-data validation using datasets collected in the wild demonstrate the clear benefits of human-centric and atomic characteristics of HAA500, which enable training even a baseline deep learning model to improve prediction by attending to atomic human poses. We detail the HAA500 dataset statistics and collection methodology and compare quantitatively with existing action recognition datasets.