# Probabilistic 3D Human Shape and Pose Estimation from Multiple Unconstrained Images in the Wild

arXiv: https://arxiv.org/abs/2103.10978 · HTML: https://mlsanity.com/paper/2103.10978/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 58 references that are in the map.
- First posted on arXiv 19 Mar 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation](https://mlsanity.com/paper/1808.05942/) (2018)
- [BodyNet: Volumetric Inference of 3D Human Body Shapes](https://mlsanity.com/paper/1804.04875/) (2018)
- [Exploiting temporal context for 3D human pose estimation in the wild](https://mlsanity.com/paper/1905.04266/) (2019)
- [DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare](https://mlsanity.com/paper/1910.00116/) (2019)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [Self-supervised Learning of Motion Capture](https://mlsanity.com/paper/1712.01337/) (2017)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)

## Built on by (most cited first, 30 of 40)

- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Probabilistic Modeling for Human Mesh Recovery](https://mlsanity.com/paper/2108.11944/) (2021)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2407.07532/) (2024)
- [FLAG: Flow-based 3D Avatar Generation from Sparse Observations](https://mlsanity.com/paper/2203.05789/) (2022)
- [Accurate 3D Body Shape Regression using Metric and Semantic Attributes](https://mlsanity.com/paper/2206.07036/) (2022)
- [Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild](https://mlsanity.com/paper/2110.00990/) (2021)
- [SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos](https://mlsanity.com/paper/2311.10812/) (2023)
- [Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms](https://mlsanity.com/paper/2209.10529/) (2022)
- [HuManiFlow: Ancestor-Conditioned Normalising Flows on SO(3) Manifolds for Human Pose and Shape Distribution Estimation](https://mlsanity.com/paper/2305.06968/) (2023)
- [Look Ma, no markers: holistic performance capture without the hassle](https://mlsanity.com/paper/2410.11520/) (2024)
- [Coarse-to-fine Animal Pose and Shape Estimation](https://mlsanity.com/paper/2111.08176/) (2021)
- [H4D: Human 4D Modeling by Learning Neural Compositional Representation](https://mlsanity.com/paper/2203.01247/) (2022)
- [MEGA: Masked Generative Autoencoder for Human Mesh Recovery](https://mlsanity.com/paper/2405.18839/) (2024)
- [VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space](https://mlsanity.com/paper/2312.08291/) (2023)
- [Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery](https://mlsanity.com/paper/2301.06020/) (2023)
- [Self-supervised Human Mesh Recovery with Cross-Representation Alignment](https://mlsanity.com/paper/2209.04596/) (2022)
- [VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation](https://mlsanity.com/paper/2210.06028/) (2022)
- [Progressive Multi-view Human Mesh Recovery with Self-Supervision](https://mlsanity.com/paper/2212.05223/) (2022)
- [SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation](https://mlsanity.com/paper/2404.15276/) (2024)
- [Body Knowledge and Uncertainty Modeling for Monocular 3D Human Body Reconstruction](https://mlsanity.com/paper/2308.00799/) (2023)
- [Normalizing Flows on the Product Space of SO(3) Manifolds for Probabilistic Human Pose Modeling](https://mlsanity.com/paper/2404.05675/) (2024)
- [KBody: Towards general, robust, and aligned monocular whole-body estimation](https://mlsanity.com/paper/2304.11542/) (2023)
- [Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model](https://mlsanity.com/paper/2111.15404/) (2021)
- [GloPro: Globally-Consistent Uncertainty-Aware 3D Human Pose Estimation & Tracking in the Wild](https://mlsanity.com/paper/2309.10369/) (2023)
- [StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset](https://mlsanity.com/paper/2407.20545/) (2024)
- [Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images](https://mlsanity.com/paper/2409.18364/) (2024)
- [CondiMen: Conditional Multi-Person Mesh Recovery](https://mlsanity.com/paper/2412.13058/) (2024)
- [ShapeBoost: Boosting Human Shape Estimation with Part-Based Parameterization and Clothing-Preserving Augmentation](https://mlsanity.com/paper/2403.01345/) (2024)

## Abstract (arXiv)

This paper addresses the problem of 3D human body shape and pose estimation from RGB images. Recent progress in this field has focused on single images, video or multi-view images as inputs. In contrast, we propose a new task: shape and pose estimation from a group of multiple images of a human subject, without constraints on subject pose, camera viewpoint or background conditions between images in the group. Our solution to this task predicts distributions over SMPL body shape and pose parameters conditioned on the input images in the group. We probabilistically combine predicted body shape distributions from each image to obtain a final multi-image shape prediction. We show that the additional body shape information present in multi-image input groups improves 3D human shape estimation metrics compared to single-image inputs on the SSP-3D dataset and a private dataset of tape-measured humans. In addition, predicting distributions over 3D bodies allows us to quantify pose prediction uncertainty, which is useful when faced with challenging input images with significant occlusion. Our method demonstrates meaningful pose uncertainty on the 3DPW dataset and is competitive with the state-of-the-art in terms of pose estimation metrics.