mlsanityData to 7 Oct 2026

The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation

arXiv 2205.00508 ↗First posted 1 May 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Nonparametric based methods have recently shown promising results in reconstructing human bodies from monocular images while model-based methods can help correct these estimates and improve prediction. However, estimating model parameters from global image features may lead to noticeable misalignment between the estimated meshes and image evidence. To address this issue and leverage the best of both worlds, we propose a framework of three consecutive modules. A dense map prediction module explicitly establishes the dense UV correspondence between the image evidence and each part of the body model. The inverse kinematics module refines the key point prediction and generates a posed template mesh. Finally, a UV inpainting module relies on the corresponding feature, prediction and the posed template, and completes the predictions of occluded body shape. Our framework leverages the best of non-parametric and model-based methods and is also robust to partial occlusion. Experiments demonstrate that our framework outperforms existing 3D human estimation methods on multiple public benchmarks.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 1 May 2022.

Builds on

40 of 56 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  4. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  5. 2017End-to-end Recovery of Human Shape and Posecited by 883
  6. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  7. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  8. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  9. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  10. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  11. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  12. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
28 more
  1. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  2. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  3. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  4. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  5. 2021Mesh Graphormercited by 264
  6. 2017Integral Human Pose Regressioncited by 255
  7. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  8. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  9. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  10. 2019Semantic Graph Convolutional Networks for 3D Human Pose Regressioncited by 204
  11. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  12. 2018PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Modelcited by 133
  13. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
  14. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  15. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  16. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  17. 2018Neural Kinematic Networks for Unsupervised Motion Retargettingcited by 85
  18. 20203D Human Mesh Regression with Dense Correspondencecited by 66
  19. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  20. 2020Monocular Real-time Full Body Capture with Inter-part Correlationscited by 53
  21. 2019Learning 3D Human Shape and Pose from Dense Body Partscited by 48
  22. 2020Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimationcited by 36
  23. 2021TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimationcited by 32
  24. 2020Full-Body Awareness from Partial Observationscited by 29
  25. 2021PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videoscited by 25
  26. 2018How Robust is 3D Human Pose Estimation to Occlusion?cited by 23
  27. 2021Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videoscited by 13
  28. 2018Synthetic Occlusion Augmentation with Volumetric Heatmaps for the 2018 ECCV PoseTrack Challenge on 3D Human Pose Estimationcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 4.

8 papers.

  1. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  2. 2022PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimationcited by 27
  3. 2023High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decompositioncited by 18
  4. 2022Occluded Human Body Capture with Self-Supervised Spatial-Temporal Motion Priorcited by 15
  5. 2022Identity-Aware Hand Mesh Estimation and Personalization from RGB Imagescited by 11
  6. 2023E3Pose: Energy-Efficient Edge-assisted Multi-camera System for Multi-human 3D Pose Estimationcited by 0
  7. 2023Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimationcited by 0
  8. 2023Improved TokenPose with Sparsitycited by 0