# Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry

arXiv: https://arxiv.org/abs/2108.07777 · HTML: https://mlsanity.com/paper/2108.07777/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 51 references that are in the map.
- First posted on arXiv 17 Aug 2021.

## Builds on (most cited first, 30 of 35)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model](https://mlsanity.com/paper/1803.08225/) (2018)
- [Epipolar Transformers](https://mlsanity.com/paper/2005.04551/) (2020)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [Self-Supervised Learning of 3D Human Pose using Multi-view Geometry](https://mlsanity.com/paper/1903.02330/) (2019)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)
- [Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows](https://mlsanity.com/paper/2003.10350/) (2020)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)
- [Cross View Fusion for 3D Human Pose Estimation](https://mlsanity.com/paper/1909.01203/) (2019)
- [RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation](https://mlsanity.com/paper/1902.09868/) (2019)
- [3D Human Pose Estimation from a Single Image via Distance Matrix Regression](https://mlsanity.com/paper/1611.09010/) (2016)
- [Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation](https://mlsanity.com/paper/1804.01110/) (2018)
- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/1511.09439/) (2015)
- [Can 3D Pose be Learned from 2D Projections Alone?](https://mlsanity.com/paper/1808.07182/) (2018)
- [Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis](https://mlsanity.com/paper/2004.04400/) (2020)
- [OriNet: A Fully Convolutional Network for 3D Human Pose Estimation](https://mlsanity.com/paper/1811.04989/) (2018)
- [Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision](https://mlsanity.com/paper/1705.11166/) (2017)
- [Traffic Control Gesture Recognition for Autonomous Vehicles](https://mlsanity.com/paper/2007.16072/) (2020)
- [Robust Estimation of 3D Human Poses from a Single Image](https://mlsanity.com/paper/1406.2282/) (2014)
- [Single Image 3D Interpreter Network](https://mlsanity.com/paper/1604.08685/) (2016)
- [Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations](https://mlsanity.com/paper/1803.08244/) (2018)

## Built on by (most cited first, 10 of 10)

- [MotionMixer: MLP-based 3D Human Body Pose Forecasting](https://mlsanity.com/paper/2207.00499/) (2022)
- [Generalizable Human Pose Triangulation](https://mlsanity.com/paper/2110.00280/) (2021)
- [Weakly-supervised Camera Localization by Ground-to-satellite Image Registration](https://mlsanity.com/paper/2409.06471/) (2024)
- [Weakly Supervised Multi-Modal 3D Human Body Pose Estimation for Autonomous Driving](https://mlsanity.com/paper/2307.14889/) (2023)
- [A Spatio-Temporal Multilayer Perceptron for Gesture Recognition](https://mlsanity.com/paper/2204.11511/) (2022)
- [Learning Temporal 3D Human Pose Estimation with Pseudo-Labels](https://mlsanity.com/paper/2110.07578/) (2021)
- [Self-supervised Secondary Landmark Detection via 3D Representation Learning](https://mlsanity.com/paper/2110.00543/) (2021)
- [Multi-person 3D pose estimation from unlabelled data](https://mlsanity.com/paper/2212.08731/) (2022)
- [Gesture Recognition with Keypoint and Radar Stream Fusion for Automated Vehicles](https://mlsanity.com/paper/2302.09998/) (2023)
- [Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning](https://mlsanity.com/paper/2311.02815/) (2023)

## Abstract (arXiv)

We present a self-supervised learning algorithm for 3D human pose estimation of a single person based on a multiple-view camera system and 2D body pose estimates for each view. To train our model, represented by a deep neural network, we propose a four-loss function learning algorithm, which does not require any 2D or 3D body pose ground-truth. The proposed loss functions make use of the multiple-view geometry to reconstruct 3D body pose estimates and impose body pose constraints across the camera views. Our approach utilizes all available camera views during training, while the inference is single-view. In our evaluations, we show promising performance on Human3.6M and HumanEva benchmarks, while we also present a generalization study on MPI-INF-3DHP dataset, as well as several ablation results. Overall, we outperform all self-supervised learning methods and reach comparable results to supervised and weakly-supervised learning approaches. Our code and models are publicly available