# SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation

arXiv: https://arxiv.org/abs/2501.09782 · HTML: https://mlsanity.com/paper/2501.09782/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 92 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 30 of 60)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Sapiens: Foundation for Human Vision Models](https://mlsanity.com/paper/2408.12569/) (2024)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Capturing and Inferring Dense Full-Body Human-Scene Contact](https://mlsanity.com/paper/2206.09553/) (2022)
- [Generating Holistic 3D Human Motion from Speech](https://mlsanity.com/paper/2212.04420/) (2022)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)

## Built on by (most cited first, 30 of 32)

- [SAM 3D Body: Robust Full-Body Human Mesh Recovery](https://mlsanity.com/paper/2602.15989/) (2026)
- [The Quest for Generalizable Motion Generation: Data, Model, and Evaluation](https://mlsanity.com/paper/2510.26794/) (2025)
- [PlayerOne: Egocentric World Simulator](https://mlsanity.com/paper/2506.09995/) (2025)
- [UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections](https://mlsanity.com/paper/2509.24817/) (2025)
- [Human Mesh Modeling for Anny Body](https://mlsanity.com/paper/2511.03589/) (2025)
- [UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass](https://mlsanity.com/paper/2601.01222/) (2026)
- [ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization](https://mlsanity.com/paper/2505.10250/) (2025)
- [PhysHMR: Learning Humanoid Control Policies from Vision for Physically Plausible Human Motion Reconstruction](https://mlsanity.com/paper/2510.02566/) (2025)
- [DyaDiT: A Multi-Modal Diffusion Transformer for Socially Favorable Dyadic Gesture Generation](https://mlsanity.com/paper/2602.23165/) (2026)
- [Generating Human Motion Videos using a Cascaded Text-to-Video Framework](https://mlsanity.com/paper/2510.03909/) (2025)
- [Syn4D: A Multiview Synthetic 4D Dataset](https://mlsanity.com/paper/2605.05207/) (2026)
- [PEAR: Pixel-aligned Expressive humAn mesh Recovery](https://mlsanity.com/paper/2601.22693/) (2026)
- [Zero-Shot Reconstruction of Animatable 3D Avatars with Cloth Dynamics from a Single Image](https://mlsanity.com/paper/2603.14772/) (2026)
- [ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine](https://mlsanity.com/paper/2607.28625/) (2026)
- [CoT-Pose: Chain-of-Thought Reasoning for 3D Pose Generation from Abstract Prompts](https://mlsanity.com/paper/2508.07540/) (2025)
- [Learnable SMPLify: A Neural Solution for Optimization-Free Human Pose Inverse Kinematics](https://mlsanity.com/paper/2508.13562/) (2025)
- [Real-Time Human Frontal View Synthesis from a Single Image](https://mlsanity.com/paper/2603.15433/) (2026)
- [DiffProxy: Multi-View Human Mesh Recovery via Diffusion-Generated Dense Proxies](https://mlsanity.com/paper/2601.02267/) (2026)
- [VRGaussianAvatar: Integrating 3D Gaussian Avatars into VR](https://mlsanity.com/paper/2602.01674/) (2026)
- [InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization](https://mlsanity.com/paper/2603.13375/) (2026)
- [UniCon3R: Unified Contact-aware 4D Human-Scene Reconstruction from Monocular Video](https://mlsanity.com/paper/2604.19923/) (2026)
- [DanceHMR: Hand-Aware Whole-Body Human Mesh Recovery from Monocular Videos](https://mlsanity.com/paper/2605.18102/) (2026)
- [Proprioceptive-visual correspondence enables self-other distinction in humanoid robots](https://mlsanity.com/paper/2606.13222/) (2026)
- [Patient4D: Temporally Consistent Patient Body Mesh Recovery from Monocular Operating Room Video](https://mlsanity.com/paper/2603.17178/) (2026)
- [GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates](https://mlsanity.com/paper/2603.28997/) (2026)
- [Evaluation of Pose Estimation Systems for Sign Language Translation](https://mlsanity.com/paper/2604.24609/) (2026)
- [Tamaththul3D: High-Fidelity 3D Saudi Sign Language Avatars from Monocular Video](https://mlsanity.com/paper/2605.05367/) (2026)
- [SMART: SMPLest-X Mesh Adaptation and RAFT Tracking for Soccer Pose Estimation](https://mlsanity.com/paper/2605.31551/) (2026)
- [FlexiAvatar: Unified 3D Gaussian Human Avatars Under Arbitrary Body Visibility](https://mlsanity.com/paper/2607.19100/) (2026)
- [GRAPE: Graduated Routing for Articulated Portrait mesh Estimation](https://mlsanity.com/paper/2607.23657/) (2026)

## Abstract (arXiv)

Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications. Despite encouraging progress, current state-of-the-art methods focus on training innovative architectural designs on confined datasets. In this work, we investigate the impact of scaling up EHPS towards a family of generalist foundation models. 1) For data scaling, we perform a systematic investigation on 40 EHPS datasets, encompassing a wide range of scenarios that a model trained on any single dataset cannot handle. More importantly, capitalizing on insights obtained from the extensive benchmarking process, we optimize our training scheme and select datasets that lead to a significant leap in EHPS capabilities. Ultimately, we achieve diminishing returns at 10M training instances from diverse data sources. 2) For model scaling, we take advantage of vision transformers (up to ViT-Huge as the backbone) to study the scaling law of model sizes in EHPS. To exclude the influence of algorithmic design, we base our experiments on two minimalist architectures: SMPLer-X, which consists of an intermediate step for hand and face localization, and SMPLest-X, an even simpler version that reduces the network to its bare essentials and highlights significant advances in the capture of articulated hands. With big data and the large model, the foundation models exhibit strong performance across diverse test benchmarks and excellent transferability to even unseen environments. Moreover, our finetuning strategy turns the generalist into specialist models, allowing them to achieve further performance boosts. Notably, our foundation models consistently deliver state-of-the-art results on seven benchmarks such as AGORA, UBody, EgoBody, and our proposed SynHand dataset for comprehensive hand evaluation. (Code is available at: https://github.com/wqyin/SMPLest-X).