# PoseNet3D: Learning Temporally Consistent 3D Human Pose via Knowledge Distillation

arXiv: https://arxiv.org/abs/2003.03473 · HTML: https://mlsanity.com/paper/2003.03473/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2020.

## Built on by (most cited first, 8 of 8)

- [Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving](https://mlsanity.com/paper/2112.12141/) (2021)
- [HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media](https://mlsanity.com/paper/2110.07235/) (2021)
- [Monocular 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks](https://mlsanity.com/paper/2104.01797/) (2021)
- [Dual networks based 3D Multi-Person Pose Estimation from Monocular Video](https://mlsanity.com/paper/2205.00748/) (2022)
- [Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry](https://mlsanity.com/paper/2108.07777/) (2021)
- [Learning Temporal 3D Human Pose Estimation with Pseudo-Labels](https://mlsanity.com/paper/2110.07578/) (2021)
- [Self-supervised Secondary Landmark Detection via 3D Representation Learning](https://mlsanity.com/paper/2110.00543/) (2021)
- [Capsules as viewpoint learners for human pose estimation](https://mlsanity.com/paper/2302.06194/) (2023)

## Abstract (arXiv)

Recovering 3D human pose from 2D joints is a highly unconstrained problem. We propose a novel neural network framework, PoseNet3D, that takes 2D joints as input and outputs 3D skeletons and SMPL body model parameters. By casting our learning approach in a student-teacher framework, we avoid using any 3D data such as paired/unpaired 3D data, motion capture sequences, depth images or multi-view images during training. We first train a teacher network that outputs 3D skeletons, using only 2D poses for training. The teacher network distills its knowledge to a student network that predicts 3D pose in SMPL representation. Finally, both the teacher and the student networks are jointly fine-tuned in an end-to-end manner using temporal, self-consistency and adversarial losses, improving the accuracy of each individual network. Results on Human3.6M dataset for 3D human pose estimation demonstrate that our approach reduces the 3D joint prediction error by 18% compared to previous unsupervised methods. Qualitative results on in-the-wild datasets show that the recovered 3D poses and meshes are natural, realistic, and flow smoothly over consecutive frames.