mlsanityData to 7 Oct 2026

MEGA: Masked Generative Autoencoder for Human Mesh Recovery

arXiv 2405.18839 ↗First posted 29 May 2024cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Human Mesh Recovery (HMR) from a single RGB image is a highly ambiguous problem, as an infinite set of 3D interpretations can explain the 2D observation equally well. Nevertheless, most HMR methods overlook this issue and make a single prediction without accounting for this ambiguity. A few approaches generate a distribution of human meshes, enabling the sampling of multiple predictions; however, none of them is competitive with the latest single-output model when making a single prediction. This work proposes a new approach based on masked generative modeling. By tokenizing the human pose and shape, we formulate the HMR task as generating a sequence of discrete tokens conditioned on an input image. We introduce MEGA, a MaskEd Generative Autoencoder trained to recover human meshes from images and partial human mesh token sequences. Given an image, our flexible generation scheme allows us to predict a single human mesh in deterministic mode or to generate multiple human meshes in stochastic mode. Experiments on in-the-wild benchmarks show that MEGA achieves state-of-the-art performance in deterministic and stochastic modes, outperforming single-output and multi-output approaches.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 82 of its 100 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

82 of 100 references.

References · 82

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  11. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  12. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
38 more
  1. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  3. 2017End-to-end Recovery of Human Shape and Posecited by 883
  4. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  5. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  6. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  7. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  8. 2017Convolutional Sequence to Sequence Learningcited by 550
  9. 2021Neural Fields in Visual Computing and Beyondcited by 539
  10. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  11. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  12. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  13. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  14. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  15. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  16. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  17. 2021Mesh Graphormercited by 264
  18. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  19. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  20. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 237
  21. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  22. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  23. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  24. 2021Probabilistic Modeling for Human Mesh Recoverycited by 152
  25. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  26. 20233D Human Pose Estimation via Intuitive Physicscited by 100
  27. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  28. 2017Self-supervised Learning of Motion Capturecited by 97
  29. 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
  30. 2023EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wildcited by 95
  31. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 93
  32. 2019Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Networkcited by 92
  33. 2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
  34. 2022Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformerscited by 81
  35. 2019Monocular 3D Human Pose Estimation by Generation and Ordinal Rankingcited by 76
  36. 2021Multi-Person Extreme Motion Predictioncited by 74
  37. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65
  38. 2021Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenescited by 64

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 8.

13 papers.

Sort:

By area:

  1. 2024UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editingcited by 18
  2. 2025SAM-Body4D: Training-Free 4D Human Body Mesh Recovery from Videoscited by 8
  3. 2026DuoMo: Dual Motion Diffusion for World-Space Human Reconstructioncited by 6
  4. 2026VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recoverycited by 2
  5. 2025Pose-RFT: Enhancing MLLMs for 3D Pose Generation via Hybrid Action Reinforcement Fine-Tuningcited by 0
  6. 2025LieHMR: Autoregressive Human Mesh Recovery with SO(3) Diffusioncited by 0
  7. 2025On the Use of Hierarchical Vision Foundation Models for Low-Cost Human Mesh Recovery and Pose Estimationcited by 0
  8. 2026Masked Modeling for Human Motion Recovery Under Occlusionscited by 0
  9. 2026PHAC: Promptable Human Amodal Completioncited by 0
  10. 2026MoPO: Incorporating Motion Prior for Occluded Human Mesh Recoverycited by 0
  11. 2026FactorizedHMR: A Hybrid Framework for Video Human Mesh Recoverycited by 0
  12. 2026B'ezier Degradation Modeling for LiDAR-based Human Motion Capturecited by 0
1 more
  1. 2026MHE-Former: Multi-Hypothesis Transformers via Entropy Maximization for 3D Mesh Recoverycited by 0