mlsanityData to 7 Oct 2026

FML: Face Model Learning from Videos

arXiv 1812.07603 ↗First posted 18 Dec 2018cs.CVcited by 78

Not on the map. Why?

Abstract (arXiv)

Monocular image-based 3D reconstruction of faces is a long-standing problem in computer vision. Since image data is a 2D projection of a 3D face, the resulting depth ambiguity makes the problem ill-posed. Most existing methods rely on data-driven priors that are built from limited 3D face scans. In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces. Our face model is learned using only corpora of in-the-wild video clips collected from the Internet. This virtually endless source of training data enables learning of a highly general 3D face model. In order to achieve this, we propose a novel multi-frame consistency loss that ensures consistent shape and appearance across multiple frames of a subject's face, thus minimizing depth ambiguity. At test time we can use an arbitrary number of frames, so that we can perform both monocular as well as multi-frame reconstruction.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 18; 2023: 17; 2024: 10; 2025: 6; 2026: 4.

78 papers.

Sort:

By area:

  1. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  2. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  3. 2022Towards Metrical Reconstruction of Human Facescited by 165
  4. 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
  5. 2024SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesiscited by 81
  6. 20223D face reconstruction with dense landmarkscited by 70
  7. 2023NOFA: NeRF-based One-shot Facial Avatar Reconstructioncited by 53
  8. 2021AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANscited by 49
  9. 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
  10. 2023Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videoscited by 44
  11. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  12. 2022FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editingcited by 38
38 more
  1. 2021A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyondcited by 37
  2. 2022Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Videocited by 37
  3. 2021Practical Face Reconstruction via Differentiable Ray Tracingcited by 35
  4. 2022Text and Image Guided 3D Avatar Generation and Manipulationcited by 35
  5. 2023Instant Multi-View Head Capture through Learnable Registrationcited by 34
  6. 2021Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstructioncited by 33
  7. 2022Towards Racially Unbiased Skin Tone Estimation via Scene Disambiguationcited by 32
  8. 2021Riggable 3D Face Reconstruction via In-Network Optimizationcited by 31
  9. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  10. 2021SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstructioncited by 27
  11. 2021Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometrycited by 27
  12. 2022Facial Geometric Detail Recovery via Implicit Representationcited by 27
  13. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  14. 2022PV3D: A 3D Generative Model for Portrait Video Generationcited by 23
  15. 2021Learning Compositional Representation for 4D Captures with Neural ODEcited by 22
  16. 20223DFaceShop: Explicitly Controllable 3D-Aware Portrait Generationcited by 22
  17. 2022SUPR: A Sparse Unified Part-Based Human Representationcited by 21
  18. 2023Learning a 3D Morphable Face Reflectance Model from Low-cost Datacited by 20
  19. 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
  20. 2021Self-Supervised 3D Face Reconstruction via Conditional Estimationcited by 17
  21. 2022S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Imagecited by 17
  22. 2023GANHead: Towards Generative Animatable Neural Head Avatarscited by 17
  23. 2021Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collectioncited by 16
  24. 2023Single-Shot Implicit Morphable Faces with Consistent Texture Parameterizationcited by 16
  25. 2023Implicit Neural Head Synthesis via Controllable Local Deformation Fieldscited by 14
  26. 2021High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptationcited by 13
  27. 20223DMM-RF: Convolutional Radiance Fields for 3D Face Modelingcited by 10
  28. 2021SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recoverycited by 9
  29. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  30. 2021Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Realitycited by 8
  31. 2022GAN2X: Non-Lambertian Inverse Rendering of Image GANscited by 8
  32. 2024Makeup Prior Models for 3D Facial Makeup Estimation and Applicationscited by 7
  33. 2025FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategycited by 7
  34. 2021Unsupervised High-Fidelity Facial Texture Generation and Reconstructioncited by 6
  35. 2024High-Quality Mesh Blendshape Generation from Face Videos via Neural Inverse Renderingcited by 6
  36. 2024SPARK: Self-supervised Personalized Real-time Monocular Face Capturecited by 6
  37. 2023FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Modelscited by 5
  38. 2024Monocular Identity-Conditioned Facial Reflectance Reconstructioncited by 5

50 most cited of 78 All 78 (JSON)