mlsanityData to 14 Oct 2026

INFAMOUS-NeRF: ImproviNg FAce MOdeling Using Semantically-Aligned Hypernetworks with Neural Radiance Fields

arXiv 2312.16197 ↗First posted 23 Dec 2023cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We propose INFAMOUS-NeRF, an implicit morphable face model that introduces hypernetworks to NeRF to improve the representation power in the presence of many training subjects. At the same time, INFAMOUS-NeRF resolves the classic hypernetwork tradeoff of representation power and editability by learning semantically-aligned latent spaces despite the subject-specific models, all without requiring a large pretrained model. INFAMOUS-NeRF further introduces a novel constraint to improve NeRF rendering along the face boundary. Our constraint can leverage photometric surface rendering and multi-view supervision to guide surface color prediction and improve rendering near the surface. Finally, we introduce a novel, loss-guided adaptive sampling method for more effective NeRF training by reducing the sampling redundancy. We show quantitatively and qualitatively that our method achieves higher representation power than prior face modeling methods in both controlled and in-the-wild settings. Code and models will be released upon publication.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 54 of its 67 references that are in the map. First posted on arXiv 23 Dec 2023.

Builds on

54 of 67 references.

References · 54

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  5. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  8. 2016HyperNetworkscited by 1,027
  9. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  10. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  11. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  12. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
38 more
  1. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  2. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  3. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  4. 2022AdaFace: Quality Adaptive Margin for Face Recognitioncited by 391
  5. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  6. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  7. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  8. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  9. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  10. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  11. 2018Nonlinear 3D Face Morphable Modelcited by 157
  12. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  13. 2022FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Datasetcited by 98
  14. 2021MoFaNeRF: Morphable Facial Neural Radiance Fieldcited by 94
  15. 2023PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circcited by 93
  16. 2019Towards High-fidelity Nonlinear 3D Face Morphable Modelcited by 91
  17. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  18. 2021HyperInverter: Improving StyleGAN Inversion via Hypernetworkcited by 85
  19. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  20. 2020Learning Compositional Radiance Fields of Dynamic Human Headscited by 78
  21. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 75
  22. 2020HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentationcited by 66
  23. 2020Portrait Neural Radiance Fields from a Single Imagecited by 61
  24. 2018On Learning 3D Face Morphable Model from In-the-wild Imagescited by 54
  25. 2019Deep Meta Functionals for Shape Representationcited by 53
  26. 2020Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networkscited by 51
  27. 2019Disentangled Representation Learning for 3D Face Shapecited by 48
  28. 2022Face Relighting with Geometrically Consistent Shadowscited by 43
  29. 2022FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editingcited by 38
  30. 2021Riggable 3D Face Reconstruction via In-Network Optimizationcited by 31
  31. 2018Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognitioncited by 27
  32. 20193D Face Modeling From Diverse Raw Scan Datacited by 25
  33. 2022Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversioncited by 24
  34. 2022NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animationcited by 23
  35. 2017Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognitioncited by 21
  36. 2020Monocular Reconstruction of Neural Face Reflectance Fieldscited by 20
  37. 2022Improving 3D-aware Image Synthesis with A Geometry-aware Discriminatorcited by 16
  38. 2023Learning 3D-aware Image Synthesis with Unknown Pose Distributioncited by 14

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenescited by 16
  2. 2024Boost Your NeRF: A Model-Agnostic Mixture of Experts Framework for High Quality and Efficient Renderingcited by 5
  3. 2025FFaceNeRF: Few-shot Face Editing in Neural Radiance Fieldscited by 1