mlsanityData to 7 Oct 2026

S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Image

arXiv 2203.07732 ↗First posted 15 Mar 2022cs.CV cs.GR cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

We present a novel face reconstruction method capable of reconstructing detailed face geometry, spatially varying face reflectance from a single monocular image. We build our work upon the recent advances of DNN-based auto-encoders with differentiable ray tracing image formation, trained in self-supervised manner. While providing the advantage of learning-based approaches and real-time reconstruction, the latter methods lacked fidelity. In this work, we achieve, for the first time, high fidelity face reconstruction using self-supervised learning only. Our novel coarse-to-fine deep architecture allows us to solve the challenging problem of decoupling face reflectance from geometry using a single image, at high computational speed. Compared to state-of-the-art methods, our method achieves more visually appealing reconstruction.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 29 of its 64 references that are in the map. First posted on arXiv 15 Mar 2022.

Builds on

29 of 64 references.

References · 29

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  2. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  3. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  4. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  5. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  6. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  7. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  8. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  9. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  10. 2018Nonlinear 3D Face Morphable Modelcited by 157
  11. 2017Morphable Face Models - An Open Frameworkcited by 138
  12. 2017SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wildcited by 110
17 more
  1. 2017Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hzcited by 91
  2. 2019Towards High-fidelity Nonlinear 3D Face Morphable Modelcited by 91
  3. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  4. 2018FML: Face Model Learning from Videoscited by 78
  5. 2016Learning Detailed Face Reconstruction from a Single Imagecited by 71
  6. 2019Towards a complete 3D morphable model of the human headcited by 63
  7. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  8. 2020A Morphable Face Albedo Modelcited by 54
  9. 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
  10. 2020Portrait Shadow Manipulationcited by 42
  11. 2019Photo-Realistic Facial Details Synthesis from Single Imagecited by 40
  12. 2021Practical Face Reconstruction via Differentiable Ray Tracingcited by 35
  13. 20173D Face Reconstruction with Geometry Details from a Single Imagecited by 27
  14. 2020Cross-modal Deep Face Normals with Deactivable Skip Connectionscited by 20
  15. 2021Efficient and Differentiable Shadow Computation for Inverse Problemscited by 11
  16. 20193D Face Reconstruction from A Single Image Assisted by 2D Face Images in the Wildcited by 7
  17. 2019Face Reflectance and Geometry Modeling via Differentiable Ray Tracingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 3; 2025: 3; 2026: 3.

17 papers.

Sort:

By area:

  1. 2022High-Res Facial Appearance Capture from Polarized Smartphone Imagescited by 21
  2. 2023Learning a 3D Morphable Face Reflectance Model from Low-cost Datacited by 20
  3. 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
  4. 2023Text-Guided 3D Face Synthesis -- From Generation to Editingcited by 8
  5. 2023A vector quantized masked autoencoder for audiovisual speech emotion recognitioncited by 6
  6. 2023DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignmentcited by 6
  7. 2023JoIN: Joint GANs Inversion for Intrinsic Image Decompositioncited by 4
  8. 2024Monocular Facial Appearance Capture in the Wildcited by 2
  9. 2024Democratizing the Creation of Animatable Facial Avatarscited by 1
  10. 2024Learning to Decouple the Lights for 3D Face Texture Modelingcited by 1
  11. 2025AvatarTex: High-Fidelity Facial Texture Reconstruction from Single-Image Stylized Avatarscited by 1
  12. 2026Skullptor: High Fidelity 3D Head Reconstruction in Seconds with Multi-View Normal Predictioncited by 1
5 more
  1. 2023HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Videocited by 0
  2. 2025Improving Facial Rig Semantics for Tracking and Retargetingcited by 0
  3. 2025Using Gaussian Splats to Create High-Fidelity Facial Geometry and Texturecited by 0
  4. 2026RelightAnyone: A Generalized Relightable 3D Gaussian Head Modelcited by 0
  5. 2026FaceSnap: Real-Time Personalized Lightstage Facial Performance Capturecited by 0