# Unsupervised Human Pose Estimation through Transforming Shape Templates

arXiv: https://arxiv.org/abs/2105.04154 · HTML: https://mlsanity.com/paper/2105.04154/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 64 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017)
- [DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model](https://mlsanity.com/paper/1605.03170/) (2016)
- [3D Human Pose Estimation in the Wild by Adversarial Learning](https://mlsanity.com/paper/1803.09722/) (2018)
- [Unsupervised Discovery of Object Landmarks as Structural Representations](https://mlsanity.com/paper/1804.04412/) (2018)
- [Unsupervised Part-Based Disentangling of Object Shape and Appearance](https://mlsanity.com/paper/1903.06946/) (2019)
- [Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance](https://mlsanity.com/paper/1806.06503/) (2018)
- [Flowing ConvNets for Human Pose Estimation in Videos](https://mlsanity.com/paper/1506.02897/) (2015)
- [Human pose estimation via Convolutional Part Heatmap Regression](https://mlsanity.com/paper/1609.01743/) (2016)
- [Self-supervised learning of a facial attribute embedding from video](https://mlsanity.com/paper/1808.06882/) (2018)
- [3D Guided Fine-Grained Face Manipulation](https://mlsanity.com/paper/1902.08900/) (2019)
- [Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis](https://mlsanity.com/paper/2004.04400/) (2020)

## Built on by (most cited first, 26 of 26)

- [Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects](https://mlsanity.com/paper/2112.11347/) (2021)
- [Human Pose as Compositional Tokens](https://mlsanity.com/paper/2303.11638/) (2023)
- [DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2307.16687/) (2023)
- [Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2203.15227/) (2022)
- [Self-Supervised Keypoint Discovery in Behavioral Videos](https://mlsanity.com/paper/2112.05121/) (2021)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints](https://mlsanity.com/paper/2205.10636/) (2022)
- [Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video](https://mlsanity.com/paper/2303.08475/) (2023)
- [Unsupervised Keypoints from Pretrained Diffusion Models](https://mlsanity.com/paper/2312.00065/) (2023)
- [3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels](https://mlsanity.com/paper/2306.04745/) (2023)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [Pose Prior Learner: Unsupervised Categorical Prior Learning for Pose Estimation](https://mlsanity.com/paper/2410.03858/) (2024)
- [A survey of top-down approaches for human pose estimation](https://mlsanity.com/paper/2202.02656/) (2022)
- [SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2501.15073/) (2025)
- [Bottom-up approaches for multi-person pose estimation and it's applications: A brief review](https://mlsanity.com/paper/2112.11834/) (2021)
- [Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video](https://mlsanity.com/paper/2202.08418/) (2022)
- [Self-Supervised 3D Human Pose Estimation in Static Video Via Neural Rendering](https://mlsanity.com/paper/2210.04514/) (2022)
- [Unsupervised 3D Keypoint Discovery with Multi-View Geometry](https://mlsanity.com/paper/2211.12829/) (2022)
- [Mask as Supervision: Leveraging Unified Mask Information for Unsupervised 3D Pose Estimation](https://mlsanity.com/paper/2312.07051/) (2023)
- [3D Human Pose Estimation Based on 2D-3D Consistency with Synchronized Adversarial Training](https://mlsanity.com/paper/2106.04274/) (2021)
- [Decanus to Legatus: Synthetic training for 2D-3D human pose lifting](https://mlsanity.com/paper/2210.02231/) (2022)
- [Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning](https://mlsanity.com/paper/2311.02815/) (2023)
- [Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation](https://mlsanity.com/paper/2405.11448/) (2024)
- [Depth-Guided Self-Supervised Human Keypoint Detection via Cross-Modal Distillation](https://mlsanity.com/paper/2410.14700/) (2024)
- [X as Supervision: Contending with Depth Ambiguity in Unsupervised Monocular 3D Pose Estimation](https://mlsanity.com/paper/2411.13026/) (2024)
- [Multi-Grained Feature Pruning for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2503.05365/) (2025)

## Abstract (arXiv)

Human pose estimation is a major computer vision problem with applications ranging from augmented reality and video capture to surveillance and movement tracking. In the medical context, the latter may be an important biomarker for neurological impairments in infants. Whilst many methods exist, their application has been limited by the need for well annotated large datasets and the inability to generalize to humans of different shapes and body compositions, e.g. children and infants. In this paper we present a novel method for learning pose estimators for human adults and infants in an unsupervised fashion. We approach this as a learnable template matching problem facilitated by deep feature extractors. Human-interpretable landmarks are estimated by transforming a template consisting of predefined body parts that are characterized by 2D Gaussian distributions. Enforcing a connectivity prior guides our model to meaningful human shape representations. We demonstrate the effectiveness of our approach on two different datasets including adults and infants.