mlsanityData to 7 Oct 2026

Collaborative Regression of Expressive Bodies using Moderation

arXiv 2105.05301 ↗First posted 11 May 2021cs.CVcited by 137

Not on the map. Why?

Abstract (arXiv)

Recovering expressive humans from images is essential for understanding human behavior. Methods that estimate 3D bodies, faces, or hands have progressed significantly, yet separately. Face methods recover accurate 3D shape and geometric details, but need a tight crop and struggle with extreme views and low resolution. Whole-body methods are robust to a wide range of poses and resolutions, but provide only a rough 3D face shape without details like wrinkles. To get the best of both worlds, we introduce PIXIE, which produces animatable, whole-body 3D avatars with realistic facial detail, from a single image. For this, PIXIE uses two key observations. First, existing work combines independent estimates from body, face, and hand experts, by trusting them equally. PIXIE introduces a novel moderator that merges the features of the experts, weighted by their confidence. All part experts can contribute to the whole, using SMPL-X's shared shape space across all body parts. Second, human shape is highly correlated with gender, but existing work ignores this. We label training images as male, female, or non-binary, and train PIXIE to infer "gendered" 3D body shapes with a novel shape loss. In addition to 3D body pose and shape parameters, PIXIE estimates expression, illumination, albedo and 3D facial surface displacements. Quantitative and qualitative evaluation shows that PIXIE estimates more accurate whole-body shape and detailed face shape than the state of the art. Models and code are available at https://pixie.is.tue.mpg.de.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). Builds on 84 of its 119 references that are in the map. First posted on arXiv 11 May 2021.

Builds on

84 of 119 references.

References · 84

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  5. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  7. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  8. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  9. 2017End-to-end Recovery of Human Shape and Posecited by 883
  10. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  11. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  12. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
38 more
  1. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  2. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  3. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  4. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  5. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  6. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  7. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  8. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  9. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  10. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  11. 2018Learning 3D Human Dynamics from Videocited by 315
  12. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  13. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  14. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  15. 2017Integral Human Pose Regressioncited by 255
  16. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  17. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  18. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  19. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  20. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  21. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  22. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  23. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  24. 2019Semantic Graph Convolutional Networks for 3D Human Pose Regressioncited by 204
  25. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 203
  26. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  27. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  28. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  29. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  30. 2019DeepHuman: 3D Human Reconstruction from a Single Imagecited by 182
  31. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  32. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  33. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  34. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  35. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  36. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  37. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 154
  38. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 19; 2023: 38; 2024: 31; 2025: 30; 2026: 16.

137 papers.

Sort:

By area:

  1. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  2. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  3. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  4. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  5. 2022Generating Holistic 3D Human Motion from Speechcited by 180
  6. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 176
  7. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  8. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  9. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 141
  10. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  11. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  12. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
38 more
  1. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  2. 20233D Human Pose Estimation via Intuitive Physicscited by 100
  3. 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 98
  4. 2023Human Motion Generation: A Surveycited by 95
  5. 2024Expressive Whole-Body 3D Gaussian Avatarcited by 89
  6. 2022Capturing and Animation of Body and Clothing from Monocular Videocited by 83
  7. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 79
  8. 2023LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learningcited by 78
  9. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  10. 2023ChatPose: Chatting about 3D Human Posecited by 71
  11. 2023X-Avatar: Expressive Human Avatarscited by 66
  12. 2024CameraHMR: Aligning People with Perspectivecited by 64
  13. 2023SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modelingcited by 54
  14. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  15. 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54
  16. 2022Accurate 3D Body Shape Regression using Metric and Semantic Attributescited by 49
  17. 2023HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recoverycited by 48
  18. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 48
  19. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 46
  20. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  21. 2023Neural Haircut: Prior-Guided Strand-Based Hair Reconstructioncited by 43
  22. 2023Global-correlated 3D-decoupling Transformer for Clothed Avatar Reconstructioncited by 42
  23. 2024AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimationcited by 41
  24. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  25. 2023ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstructioncited by 37
  26. 2024GaussianBody: Clothed Human Reconstruction via 3d Gaussian Splattingcited by 36
  27. 2025Human3R: Everyone Everywhere All at Oncecited by 35
  28. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 34
  29. 2023Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinementcited by 31
  30. 2025SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimationcited by 31
  31. 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmarkcited by 29
  32. 2024WHAC: World-grounded Humans and Camerascited by 29
  33. 2024Generalizable Human Gaussians from Single-View Imagecited by 29
  34. 2022FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstructioncited by 27
  35. 2022DART: Articulated Hand Model with Diverse Accessories and Rich Texturescited by 25
  36. 2023Garment Recovery with Shape and Deformation Priorscited by 23
  37. 2024Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRMcited by 23
  38. 2024Look Ma, no markers: holistic performance capture without the hasslecited by 23

50 most cited of 137 All 137 (JSON)