# NTU-X: An Enhanced Large-scale Dataset for Improving Pose-based Recognition of Subtle Human Actions

arXiv: https://arxiv.org/abs/2101.11529 · HTML: https://mlsanity.com/paper/2101.11529/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Monocular Expressive Body Regression through Body-Driven Attention](https://mlsanity.com/paper/2008.09062/) (2020)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition](https://mlsanity.com/paper/2010.09978/) (2020)
- [A Survey on 3D Skeleton-Based Action Recognition Using Learning Method](https://mlsanity.com/paper/2002.05907/) (2020)
- [Mimetics: Towards Understanding Human Actions Out of Context](https://mlsanity.com/paper/1912.07249/) (2019)
- [Quo Vadis, Skeleton Action Recognition ?](https://mlsanity.com/paper/2007.02072/) (2020)
- [A Large-scale Varying-view RGB-D Action Dataset for Arbitrary-view Human Action Recognition](https://mlsanity.com/paper/1904.10681/) (2019)
- [IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos](https://mlsanity.com/paper/2007.06317/) (2020)

## Built on by (most cited first, 11 of 11)

- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022)
- [ReGenNet: Towards Human Action-Reaction Synthesis](https://mlsanity.com/paper/2403.11882/) (2024)
- [MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls](https://mlsanity.com/paper/2407.21136/) (2024)
- [PSUMNet: Unified Modality Part Streams are All You Need for Efficient Pose-based Action Recognition](https://mlsanity.com/paper/2208.05775/) (2022)
- [ARFlow: Human Action-Reaction Flow Matching with Physical Guidance](https://mlsanity.com/paper/2503.16973/) (2025)
- [Using joint angles based on the international biomechanical standards for human action recognition and related tasks](https://mlsanity.com/paper/2406.17443/) (2024)
- [Fusing Hand and Body Skeletons for Human Action Recognition in Assembly](https://mlsanity.com/paper/2307.09238/) (2023)
- [Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition](https://mlsanity.com/paper/2308.10557/) (2023)
- [3D Pose Nowcasting: Forecast the Future to Improve the Present](https://mlsanity.com/paper/2308.12914/) (2023)
- [Skarimva: Skeleton-based Action Recognition is a Multi-view Application](https://mlsanity.com/paper/2602.23231/) (2026)

## Abstract (arXiv)

The lack of fine-grained joints (facial joints, hand fingers) is a fundamental performance bottleneck for state of the art skeleton action recognition models. Despite this bottleneck, community's efforts seem to be invested only in coming up with novel architectures. To specifically address this bottleneck, we introduce two new pose based human action datasets - NTU60-X and NTU120-X. Our datasets extend the largest existing action recognition dataset, NTU-RGBD. In addition to the 25 body joints for each skeleton as in NTU-RGBD, NTU60-X and NTU120-X dataset includes finger and facial joints, enabling a richer skeleton representation. We appropriately modify the state of the art approaches to enable training using the introduced datasets. Our results demonstrate the effectiveness of these NTU-X datasets in overcoming the aforementioned bottleneck and improve state of the art performance, overall and on previously worst performing action categories. Code and pretrained models can be found at https://github.com/skelemoa/ntu-x .