mlsanityData to 7 Oct 2026

HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details

arXiv 2303.11225 ↗First posted 20 Mar 2023cs.CV cs.GRcited by 22

Not on the map. Why?

Abstract (arXiv)

3D Morphable Models (3DMMs) demonstrate great potential for reconstructing faithful and animatable 3D facial surfaces from a single image. The facial surface is influenced by the coarse shape, as well as the static detail (e,g., person-specific appearance) and dynamic detail (e.g., expression-driven wrinkles). Previous work struggles to decouple the static and dynamic details through image-level supervision, leading to reconstructions that are not realistic. In this paper, we aim at high-fidelity 3D face reconstruction and propose HiFace to explicitly model the static and dynamic details. Specifically, the static detail is modeled as the linear combination of a displacement basis, while the dynamic detail is modeled as the linear interpolation of two displacement maps with polarized expressions. We exploit several loss functions to jointly learn the coarse shape and fine details with both synthetic and real-world datasets, which enable HiFace to reconstruct high-fidelity 3D shapes with animatable details. Extensive quantitative and qualitative experiments demonstrate that HiFace presents state-of-the-art reconstruction quality and faithfully recovers both the static and dynamic details. Our project page can be found at https://project-hiface.github.io.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 56 of its 85 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

56 of 85 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  8. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  9. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  10. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  11. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  12. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
38 more
  1. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  2. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  3. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  4. 2018Deep Video Portraitscited by 265
  5. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 257
  6. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  7. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  8. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  9. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  10. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  11. 2022Towards Metrical Reconstruction of Human Facescited by 165
  12. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  13. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  14. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  15. 2018Unsupervised Training for 3D Morphable Model Regressioncited by 99
  16. 2022FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Datasetcited by 98
  17. 2020Learning Formation of Physically-Based Face Attributescited by 94
  18. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  19. 20223D face reconstruction with dense landmarkscited by 70
  20. 2017Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regressioncited by 66
  21. 2020A Morphable Face Albedo Modelcited by 54
  22. 2020Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networkscited by 51
  23. 2017Towards Large-Pose Face Frontalization in the Wildcited by 50
  24. 2019Combining 3D Morphable Models: A Large scale Face-and-Head Modelcited by 49
  25. 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
  26. 2020High-Fidelity 3D Digital Human Head Creation from RGB-D Selfiescited by 47
  27. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  28. 2023A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Imagescited by 43
  29. 2019Photo-Realistic Facial Details Synthesis from Single Imagecited by 40
  30. 2022FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstructioncited by 39
  31. 2021Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinementcited by 36
  32. 2022REALY: Rethinking the Evaluation of 3D Face Reconstructioncited by 32
  33. 2022DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Imagecited by 31
  34. 2018Pose-Robust Face Recognition via Deep Residual Equivariant Mappingcited by 28
  35. 20173D Face Reconstruction with Geometry Details from a Single Imagecited by 27
  36. 2021SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstructioncited by 27
  37. 2021Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometrycited by 27
  38. 2022Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsingcited by 25

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 6; 2025: 6; 2026: 4.

22 papers.

Sort:

By area:

  1. 2023GAIA: Zero-shot Talking Avatar Generationcited by 43
  2. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  3. 2024UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Modelcited by 29
  4. 20243D Face Tracking from 2D Video through Iterative Dense UV to Image Flowcited by 20
  5. 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
  6. 2023ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignmentcited by 13
  7. 2024DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Imagecited by 9
  8. 2025FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategycited by 7
  9. 2024VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstructioncited by 6
  10. 2023FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Modelscited by 5
  11. 2024S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketchcited by 5
  12. 2024SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargetingcited by 4
10 more
  1. 2025AvatarMakeup: Realistic Makeup Transfer for 3D Animatable Head Avatarscited by 3
  2. 2025GeoAvatar: Adaptive Geometrical Gaussian Splatting for 3D Head Avatarcited by 3
  3. 2025High-Quality Facial Albedo Generation for 3D Face Reconstruction from a Single Image using a Coarse-to-Fine Approachcited by 2
  4. 2023GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidancecited by 1
  5. 2026Skullptor: High Fidelity 3D Head Reconstruction in Seconds with Multi-View Normal Predictioncited by 1
  6. 2026Monocular Avatar Reconstruction via Cascaded Diffusion Priors and UV-Space Differentiable Shadingcited by 1
  7. 2025JOLT3D: Joint Learning of Talking Heads and 3DMM Parameters with Application to Lip-Synccited by 0
  8. 2025Using Gaussian Splats to Create High-Fidelity Facial Geometry and Texturecited by 0
  9. 2026SelfieAvatar: Real-time Head Avatar reenactment from a Selfie Videocited by 0
  10. 2026Diff-PC: Identity-preserving and 3D-aware Controllable Diffusion for Zero-shot Portrait Customizationcited by 0