# Unsupervised 3D Pose Estimation for Hierarchical Dance Video Recognition

arXiv: https://arxiv.org/abs/2109.09166 · HTML: https://mlsanity.com/paper/2109.09166/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 19 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model](https://mlsanity.com/paper/1803.08225/) (2018)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach](https://mlsanity.com/paper/1704.02447/) (2017)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [Self-Supervised Learning of 3D Human Pose using Multi-view Geometry](https://mlsanity.com/paper/1903.02330/) (2019)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)
- [RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation](https://mlsanity.com/paper/1902.09868/) (2019)
- [XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera](https://mlsanity.com/paper/1907.00837/) (2019)
- [Dance with Flow: Two-in-One Stream Action Detection](https://mlsanity.com/paper/1904.00696/) (2019)
- [3D Human Pose Machines with Self-supervised Learning](https://mlsanity.com/paper/1901.03798/) (2019)
- [Robust Estimation of Similarity Transformation for Visual Object Tracking](https://mlsanity.com/paper/1712.05231/) (2017)
- [Anatomy-aware 3D Human Pose Estimation with Bone-based Pose Decomposition](https://mlsanity.com/paper/2002.10322/) (2020)
- [Unsupervised Cross-Modal Alignment for Multi-Person 3D Pose Estimation](https://mlsanity.com/paper/2008.01388/) (2020)
- [Let's Dance: Learning From Online Dance Videos](https://mlsanity.com/paper/1801.07388/) (2018)

## Built on by (most cited first, 9 of 9)

- [PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision](https://mlsanity.com/paper/2203.15625/) (2022)
- [Fine-Grained Sports, Yoga, and Dance Postures Recognition: A Benchmark Analysis](https://mlsanity.com/paper/2308.00323/) (2023)
- [Pose Prior Learner: Unsupervised Categorical Prior Learning for Pose Estimation](https://mlsanity.com/paper/2410.03858/) (2024)
- [Unsupervised 3D Keypoint Discovery with Multi-View Geometry](https://mlsanity.com/paper/2211.12829/) (2022)
- [EgoMusic-driven Human Dance Motion Estimation with Skeleton Mamba](https://mlsanity.com/paper/2508.10522/) (2025)
- [TaiChi Action Capture and Performance Analysis with Multi-view RGB Cameras](https://mlsanity.com/paper/2306.14490/) (2023)
- [Component attention network for multimodal dance improvisation recognition](https://mlsanity.com/paper/2310.05938/) (2023)
- [EPOCH: Jointly Estimating the 3D Pose of Cameras and Humans](https://mlsanity.com/paper/2406.19726/) (2024)
- [Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle](https://mlsanity.com/paper/2309.06313/) (2023)

## Abstract (arXiv)

Dance experts often view dance as a hierarchy of information, spanning low-level (raw images, image sequences), mid-levels (human poses and bodypart movements), and high-level (dance genre). We propose a Hierarchical Dance Video Recognition framework (HDVR). HDVR estimates 2D pose sequences, tracks dancers, and then simultaneously estimates corresponding 3D poses and 3D-to-2D imaging parameters, without requiring ground truth for 3D poses. Unlike most methods that work on a single person, our tracking works on multiple dancers, under occlusions. From the estimated 3D pose sequence, HDVR extracts body part movements, and therefrom dance genre. The resulting hierarchical dance representation is explainable to experts. To overcome noise and interframe correspondence ambiguities, we enforce spatial and temporal motion smoothness and photometric continuity over time. We use an LSTM network to extract 3D movement subsequences from which we recognize the dance genre. For experiments, we have identified 154 movement types, of 16 body parts, and assembled a new University of Illinois Dance (UID) Dataset, containing 1143 video clips of 9 genres covering 30 hours, annotated with movement and genre labels. Our experimental results demonstrate that our algorithms outperform the state-of-the-art 3D pose estimation methods, which also enhances our dance recognition performance.