# Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory

arXiv: https://arxiv.org/abs/2012.14739 · HTML: https://mlsanity.com/paper/2012.14739/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Balanced MSE for Imbalanced Visual Regression](https://mlsanity.com/paper/2203.16427/) (2022)
- [Scene-aware Generative Network for Human Motion Synthesis](https://mlsanity.com/paper/2105.14804/) (2021)
- [Playing for 3D Human Recovery](https://mlsanity.com/paper/2110.07588/) (2021)
- [Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements](https://mlsanity.com/paper/2111.00763/) (2021)
- [H3WB: Human3.6M 3D WholeBody Dataset and Benchmark](https://mlsanity.com/paper/2211.15692/) (2022)
- [VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds](https://mlsanity.com/paper/2110.08729/) (2021)
- [FuRPE: Learning Full-body Reconstruction from Part Experts](https://mlsanity.com/paper/2212.00731/) (2022)
- [Noise-in, Bias-out: Balanced and Real-time MoCap Solving](https://mlsanity.com/paper/2309.14330/) (2023)

## Abstract (arXiv)

Deep neural networks have achieved great progress in single-image 3D human reconstruction. However, existing methods still fall short in predicting rare poses. The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses. In this work, we 1) identify and analyze this learning obstacle and 2) propose a prototype memory-augmented network, PM-Net, that effectively improves performances of predicting rare poses. The core of our framework is a memory module that learns and stores a set of 3D human prototypes capturing local distributions for either common poses or rare poses. With this formulation, the regression starts from a better initialization, which is relatively easier to converge. Extensive experiments on several widely employed datasets demonstrate the proposed framework's effectiveness compared to other state-of-the-art methods. Notably, our approach significantly improves the models' performances on rare poses while generating comparable results on other samples.