mlsanityData to 7 Oct 2026

REALY: Rethinking the Evaluation of 3D Face Reconstruction

arXiv 2203.09729 ↗First posted 18 Mar 2022cs.CV cs.GRcited by 32

Not on the map. Why?

Abstract (arXiv)

The evaluation of 3D face reconstruction results typically relies on a rigid shape alignment between the estimated 3D model and the ground-truth scan. We observe that aligning two shapes with different reference points can largely affect the evaluation results. This poses difficulties for precisely diagnosing and improving a 3D face reconstruction method. In this paper, we propose a novel evaluation approach with a new benchmark REALY, consists of 100 globally aligned face scans with accurate facial keypoints, high-quality region masks, and topology-consistent meshes. Our approach performs region-wise shape alignment and leads to more accurate, bidirectional correspondences during computing the shape errors. The fine-grained, region-wise evaluation results provide us detailed understandings about the performance of state-of-the-art 3D face reconstruction methods. For example, our experiments on single-image based reconstruction methods reveal that DECA performs the best on nose regions, while GANFit performs better on cheek regions. Besides, a new and high-quality 3DMM basis, HIFI3D++, is further derived using the same procedure as we construct REALY to align and retopologize several 3D face datasets. We will release REALY, HIFI3D++, and our new evaluation pipeline at https://realy3dface.com.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 52 of its 81 references that are in the map. First posted on arXiv 18 Mar 2022.

Builds on

52 of 81 references.

References · 52

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  5. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  6. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  7. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  8. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  9. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  10. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  11. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  12. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
38 more
  1. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  2. 2018Nonlinear 3D Face Morphable Modelcited by 157
  3. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  4. 2018Face Alignment in Full Pose Range: A 3D Total Solutioncited by 147
  5. 2021Pixel Codec Avatarscited by 125
  6. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  7. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  8. 2018Unsupervised Training for 3D Morphable Model Regressioncited by 99
  9. 2019Towards High-fidelity Nonlinear 3D Face Morphable Modelcited by 91
  10. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  11. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81
  12. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  13. 20163D Face Reconstruction by Learning from Synthetic Datacited by 64
  14. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  15. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  16. 2018On Learning 3D Face Morphable Model from In-the-wild Imagescited by 54
  17. 2020A Morphable Face Albedo Modelcited by 54
  18. 2020Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networkscited by 51
  19. 2019Combining 3D Morphable Models: A Large scale Face-and-Head Modelcited by 49
  20. 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
  21. 2020High-Fidelity 3D Digital Human Head Creation from RGB-D Selfiescited by 47
  22. 2019MVF-Net: Multi-View 3D Face Morphable Model Regressioncited by 45
  23. 2021Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinementcited by 36
  24. 2020Learning Complete 3D Morphable Face Models from Images and Videoscited by 35
  25. 2021Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstructioncited by 33
  26. 2021Riggable 3D Face Reconstruction via In-Network Optimizationcited by 31
  27. 2018ExpNet: Landmark-Free, Deep, 3D Facial Expressionscited by 29
  28. 2018Pose-Robust Face Recognition via Deep Residual Equivariant Mappingcited by 28
  29. 2018Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognitioncited by 27
  30. 2020Personalized Face Modeling for Improved Face Reconstruction and Motion Retargetingcited by 27
  31. 2018Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wildcited by 22
  32. 2017FacePoseNet: Making a Case for Landmark-Free Face Alignmentcited by 21
  33. 2019Self-supervised Learning of Detailed 3D Face Reconstructioncited by 21
  34. 2021Inverting Generative Adversarial Renderer for Face Reconstructioncited by 17
  35. 2021Self-Supervised 3D Face Reconstruction via Conditional Estimationcited by 17
  36. 2021MeInGame: Create a Game Character Face from a Single Portraitcited by 16
  37. 2021Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collectioncited by 16
  38. 2020Learning 3D Face Reconstruction with a Pose Guidance Networkcited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 6; 2025: 10; 2026: 4.

32 papers.

Sort:

By area:

  1. 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 171
  2. 2023A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Imagescited by 43
  3. 2022FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstructioncited by 39
  4. 20233D Face Reconstruction with the Geometric Guidance of Facial Part Segmentationcited by 38
  5. 2022Smooth Non-Rigid Shape Matching via Effective Dirichlet Energy Optimizationcited by 37
  6. 2023HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Detailscited by 22
  7. 2023Towards Realistic Generative 3D Face Modelscited by 20
  8. 20243D Face Tracking from 2D Video through Iterative Dense UV to Image Flowcited by 20
  9. 2025Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstructioncited by 20
  10. 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
  11. 2023Learning Audio-Driven Viseme Dynamics for 3D Face Animationcited by 12
  12. 2025Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Controlcited by 12
20 more
  1. 2024FacialFlowNet: Advancing Facial Optical Flow Estimation with a Diverse Dataset and a Decomposed Modelcited by 7
  2. 2025FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategycited by 7
  3. 2023A Perceptual Shape Loss for Monocular 3D Face Reconstructioncited by 6
  4. 2024Learning a Generalized Physical Face Model From Datacited by 6
  5. 2023FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Modelscited by 5
  6. 2023InstantAvatar: Efficient 3D Head Reconstruction via Surface Renderingcited by 4
  7. 2023Anatomically Constrained Implicit Face Modelscited by 4
  8. 2024SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargetingcited by 4
  9. 2025Exploiting Multiple Representations: 3D Face Biometrics Fusion with Application to Surveillancecited by 4
  10. 2025SRM-Hair: Single Image Head Mesh Reconstruction via 3D Morphable Haircited by 2
  11. 2025High-Quality Facial Albedo Generation for 3D Face Reconstruction from a Single Image using a Coarse-to-Fine Approachcited by 2
  12. 2025Electromyography-Informed Facial Expression Reconstruction for Physiological-Based Synthesis and Analysiscited by 1
  13. 20253D Face Reconstruction Error Decomposed: A Modular Benchmark for Fair and Fast Method Evaluationcited by 1
  14. 2025AvatarTex: High-Fidelity Facial Texture Reconstruction from Single-Image Stylized Avatarscited by 1
  15. 2026Monocular Avatar Reconstruction via Cascaded Diffusion Priors and UV-Space Differentiable Shadingcited by 1
  16. 2024Text-Driven Diverse Facial Texture Generation via Progressive Latent-Space Refinementcited by 0
  17. 2025Geometry-Aware Texture Generation for 3D Head Modeling with Artist-driven Controlcited by 0
  18. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generationcited by 0
  19. 2026Profile-Specific 3DMM Regression from a Single Lateral Face Imagecited by 0
  20. 2026Discovering Geometric Biases in 3D Face Reconstruction: A Curvature-Aware Spectral Framework for Fairness Evaluationcited by 0