# Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms

arXiv: https://arxiv.org/abs/2209.10529 · HTML: https://mlsanity.com/paper/2209.10529/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 103 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 30 of 71)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop](https://mlsanity.com/paper/2103.16507/) (2021)
- [Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose](https://mlsanity.com/paper/2008.09047/) (2020)
- [CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation](https://mlsanity.com/paper/2208.00571/) (2022)

## Built on by (most cited first, 28 of 28)

- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2309.17448/) (2023)
- [One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer](https://mlsanity.com/paper/2303.16160/) (2023)
- [Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2407.07532/) (2024)
- [SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling](https://mlsanity.com/paper/2303.17368/) (2023)
- [ReFit: Recurrent Fitting Network for 3D Human Recovery](https://mlsanity.com/paper/2308.11184/) (2023)
- [SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2501.09782/) (2025)
- [FLAG3D: A 3D Fitness Activity Dataset with Language Instruction](https://mlsanity.com/paper/2212.04638/) (2022)
- [MEGA: Masked Generative Autoencoder for Human Mesh Recovery](https://mlsanity.com/paper/2405.18839/) (2024)
- [VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space](https://mlsanity.com/paper/2312.08291/) (2023)
- [You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception](https://mlsanity.com/paper/2312.05525/) (2023)
- [Diffusion Models are Efficient Data Generators for Human Mesh Recovery](https://mlsanity.com/paper/2403.11111/) (2024)
- [Rethinking Self-Supervised Visual Representation Learning in Pre-training for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2303.05370/) (2023)
- [Learning Dense UV Completion for Human Mesh Recovery](https://mlsanity.com/paper/2307.11074/) (2023)
- [FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions](https://mlsanity.com/paper/2309.05073/) (2023)
- [KBody: Towards general, robust, and aligned monocular whole-body estimation](https://mlsanity.com/paper/2304.11542/) (2023)
- [Towards Robust and Expressive Whole-body Human Pose and Shape Estimation](https://mlsanity.com/paper/2312.08730/) (2023)
- [HumanMM: Global Human Motion Recovery from Multi-shot Videos](https://mlsanity.com/paper/2503.07597/) (2025)
- [Personalized 3D Human Pose and Shape Refinement](https://mlsanity.com/paper/2403.11634/) (2024)
- [Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction](https://mlsanity.com/paper/2402.19197/) (2024)
- [Unveiling Hidden Vulnerabilities in Digital Human Generation via Adversarial Attacks](https://mlsanity.com/paper/2504.17457/) (2025)
- [Shape-aware Inertial Poser: Motion Tracking for Humans with Diverse Shapes Using Sparse Inertial Sensors](https://mlsanity.com/paper/2510.17101/) (2025)
- [A Modular Multi-stage Lightweight Graph Transformer Network for Human Pose and Shape Estimation from 2D Human Pose](https://mlsanity.com/paper/2301.13403/) (2023)
- [SHARE: Single-view Human Adversarial REconstruction](https://mlsanity.com/paper/2401.00343/) (2023)
- [LatentStealth: Unnoticeable and Efficient Adversarial Attacks on Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2505.12009/) (2025)
- [Paving the Way Towards Kinematic Assessment Using Monocular Video: A Preclinical Benchmark of State-of-the-Art Deep-Learning-Based 3D Human Pose Estimators Against Inertial Sensors in Daily Living Activities](https://mlsanity.com/paper/2510.02264/) (2025)
- [On the Use of Hierarchical Vision Foundation Models for Low-Cost Human Mesh Recovery and Pose Estimation](https://mlsanity.com/paper/2510.12660/) (2025)

## Abstract (arXiv)

3D human pose and shape estimation (a.k.a. "human mesh recovery") has achieved substantial progress. Researchers mainly focus on the development of novel algorithms, while less attention has been paid to other critical factors involved. This could lead to less optimal baselines, hindering the fair and faithful evaluations of newly designed methodologies. To address this problem, this work presents the first comprehensive benchmarking study from three under-explored perspectives beyond algorithms. 1) Datasets. An analysis on 31 datasets reveals the distinct impacts of data samples: datasets featuring critical attributes (i.e. diverse poses, shapes, camera characteristics, backbone features) are more effective. Strategical selection and combination of high-quality datasets can yield a significant boost to the model performance. 2) Backbones. Experiments with 10 backbones, ranging from CNNs to transformers, show the knowledge learnt from a proximity task is readily transferable to human mesh recovery. 3) Training strategies. Proper augmentation techniques and loss designs are crucial. With the above findings, we achieve a PA-MPJPE of 47.3 mm on the 3DPW test set with a relatively simple model. More importantly, we provide strong baselines for fair comparisons of algorithms, and recommendations for building effective training configurations in the future. Codebase is available at http://github.com/smplbody/hmr-benchmarks