# NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding

arXiv: https://arxiv.org/abs/1905.04757 · HTML: https://mlsanity.com/paper/1905.04757/

## Key facts

- Cited by 533 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2019.

## Built on by (most cited first, 30 of 533)

- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [DisCo: Disentangled Control for Realistic Human Dance Generation](https://mlsanity.com/paper/2307.00040/) (2023)
- [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022)
- [Gait Recognition in the Wild with Dense 3D Representations and A Benchmark](https://mlsanity.com/paper/2204.02569/) (2022)
- [3D Human Action Representation Learning via Cross-View Consistency Pursuit](https://mlsanity.com/paper/2104.14466/) (2021)
- [HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation](https://mlsanity.com/paper/2304.04269/) (2023)
- [HumanTOMATO: Text-aligned Whole-body Motion Generation](https://mlsanity.com/paper/2310.12978/) (2023)
- [Inter-X: Towards Versatile Human-Human Interaction Analysis](https://mlsanity.com/paper/2312.16051/) (2023)
- [UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles](https://mlsanity.com/paper/2104.00946/) (2021)
- [Human Motion Generation: A Survey](https://mlsanity.com/paper/2307.10894/) (2023)
- [CHORE: Contact, Human and Object REconstruction from a single RGB image](https://mlsanity.com/paper/2204.02445/) (2022)
- [Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition](https://mlsanity.com/paper/2112.03590/) (2021)
- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [Hierarchically Decomposed Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2208.10741/) (2022)
- [PYSKL: Towards Good Practices for Skeleton Action Recognition](https://mlsanity.com/paper/2205.09443/) (2022)
- [ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation](https://mlsanity.com/paper/2203.07706/) (2022)
- [Skeleton-Contrastive 3D Action Representation Learning](https://mlsanity.com/paper/2108.03656/) (2021)

## Abstract (arXiv)

Research on depth-based human activity analysis achieved outstanding performance and demonstrated the effectiveness of 3D representation for action recognition. The existing depth-based and RGB+D-based action recognition benchmarks have a number of limitations, including the lack of large-scale training samples, realistic number of distinct class categories, diversity in camera views, varied environmental conditions, and variety of human subjects. In this work, we introduce a large-scale dataset for RGB+D human action recognition, which is collected from 106 distinct subjects and contains more than 114 thousand video samples and 8 million frames. This dataset contains 120 different action classes including daily, mutual, and health-related activities. We evaluate the performance of a series of existing 3D activity analysis methods on this dataset, and show the advantage of applying deep learning methods for 3D-based human action recognition. Furthermore, we investigate a novel one-shot 3D activity recognition problem on our dataset, and a simple yet effective Action-Part Semantic Relevance-aware (APSR) framework is proposed for this task, which yields promising results for recognition of the novel action classes. We believe the introduction of this large-scale dataset will enable the community to apply, adapt, and develop various data-hungry learning techniques for depth-based and RGB+D-based human activity understanding. [The dataset is available at: http://rose1.ntu.edu.sg/Datasets/actionRecognition.asp]