mlsanityData to 14 Oct 2026

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

arXiv 2106.07852 ↗First posted 15 Jun 2021cs.CV cs.GRcited by 16

Not on the map. Why?

Abstract (arXiv)

Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. While plausible facial details are predicted, the models tend to over-depend on local color appearance and suffer from ambiguous noise. To address such problem, this paper presents a novel Learning to Aggregate and Personalize (LAP) framework for unsupervised robust 3D face modeling. Instead of using controlled environment, the proposed method implicitly disentangles ID-consistent and scene-specific face from unconstrained photo set. Specifically, to learn ID-consistent face, LAP adaptively aggregates intrinsic face factors of an identity based on a novel curriculum learning approach with relaxed consistency loss. To adapt the face for a personalized scene, we propose a novel attribute-refining network to modify ID-consistent face with target attribute and details. Based on the proposed method, we make unsupervised 3D face modeling benefit from meaningful image facial structure and possibly higher resolutions. Extensive experiments on benchmarks show LAP recovers superior or competitive face shape and texture, compared with state-of-the-art (SOTA) methods with or without prior and supervision.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

49 of 74 references.

References · 49

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  5. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  6. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  7. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  8. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  9. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  10. 2014Learning Face Representation from Scratchcited by 453
  11. 2017Neural 3D Mesh Renderercited by 437
  12. 2020CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognitioncited by 321
37 more
  1. 20193D Morphable Face Models -- Past, Present and Futurecited by 270
  2. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  3. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  4. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  5. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  6. 2019Probabilistic Face Embeddingscited by 173
  7. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  8. 2018GANimation: Anatomically-aware Facial Animation from a Single Imagecited by 161
  9. 2018Nonlinear 3D Face Morphable Modelcited by 157
  10. 2018Face Alignment in Full Pose Range: A 3D Total Solutioncited by 148
  11. 2017SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wildcited by 110
  12. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  13. 2018Unsupervised Training for 3D Morphable Model Regressioncited by 100
  14. 2017Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hzcited by 91
  15. 2019Towards High-fidelity Nonlinear 3D Face Morphable Modelcited by 91
  16. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  17. 2018FML: Face Model Learning from Videoscited by 78
  18. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  19. 2016Learning Detailed Face Reconstruction from a Single Imagecited by 71
  20. 2017Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regressioncited by 66
  21. 20163D Face Reconstruction by Learning from Synthetic Datacited by 64
  22. 2018Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearancecited by 58
  23. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  24. 2020Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networkscited by 51
  25. 2019Disentangled Representation Learning for 3D Face Shapecited by 48
  26. 2017End-to-end 3D face reconstruction with deep neural networkscited by 47
  27. 2019MVF-Net: Multi-View 3D Face Morphable Model Regressioncited by 45
  28. 2017Quality Aware Network for Set to Set Recognitioncited by 36
  29. 2017Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervisioncited by 27
  30. 2020Personalized Face Modeling for Improved Face Reconstruction and Motion Retargetingcited by 27
  31. 2019Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motioncited by 23
  32. 2019Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoderscited by 22
  33. 2017Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognitioncited by 21
  34. 2020Cross-modal Deep Face Normals with Deactivable Skip Connectionscited by 20
  35. 2020LEED: Label-Free Expression Editing via Disentanglementcited by 16
  36. 2019Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Trackingcited by 12
  37. 2018Unsupervised Depth Estimation, 3D Face Rotation and Replacementcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 8; 2024: 1; 2025: 2; 2026: 3.

16 papers.

Sort:

By area:

  1. 2023GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivationcited by 46
  2. 2023A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Imagescited by 43
  3. 2021FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstructioncited by 36
  4. 2022REALY: Rethinking the Evaluation of 3D Face Reconstructioncited by 33
  5. 2023Creative Birds: Self-Supervised Single-View 3D Style Transfercited by 9
  6. 20233D Face Reconstruction: the Road to Forensicscited by 7
  7. 2023RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairscited by 6
  8. 2025VGGTFace: Topologically Consistent Facial Geometry Reconstruction in the Wildcited by 4
  9. 2023Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Imagescited by 3
  10. 2023FLAME-based Multi-View 3D Face Reconstructioncited by 3
  11. 2023High-Quality 3D Face Reconstruction with Affine Convolutional Networkscited by 3
  12. 2024Personalized Generative Low-light Image Denoising and Enhancementcited by 1
4 more
  1. 2025LH2Face: Loss function for Hard High-quality Facecited by 1
  2. 2026Face Normal Estimation from Rags to Richescited by 0
  3. 2026M3D-Net: Multi-Modal 3D Facial Feature Reconstruction Network for Deepfake Detectioncited by 0
  4. 2026UVFaceFusion: Fast Multi-view Topologically Consistent Face Reconstruction in the Wild via UV-space Neural Fusioncited by 0