mlsanityData to 7 Oct 2026

VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting

arXiv 2201.04873 ↗First posted 13 Jan 2022cs.CVcited by 37

Not on the map. Why?

Abstract (arXiv)

We propose VoLux-GAN, a generative framework to synthesize 3D-aware faces with convincing relighting. Our main contribution is a volumetric HDRI relighting method that can efficiently accumulate albedo, diffuse and specular lighting contributions along each 3D ray for any desired HDR environmental map. Additionally, we show the importance of supervising the image decomposition process using multiple discriminators. In particular, we propose a data augmentation technique that leverages recent advances in single image portrait relighting to enforce consistent geometry, albedo, diffuse and specular components. Multiple experiments and comparisons with other generative frameworks show how our model is a step forward towards photorealistic relightable 3D generative models.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 52 of its 80 references that are in the map. First posted on arXiv 13 Jan 2022.

Builds on

52 of 80 references.

References · 52

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  9. 2018Demystifying MMD GANscited by 1,268
  10. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  11. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  12. 2021Alias-Free Generative Adversarial Networkscited by 1,181
38 more
  1. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  2. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  3. 2018Self-Attention Generative Adversarial Networkscited by 809
  4. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  7. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  8. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  9. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  10. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  11. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  12. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  13. 2020State of the Art on Neural Renderingcited by 276
  14. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  15. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  16. 2020DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detectioncited by 231
  17. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  18. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  19. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 204
  20. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  21. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  22. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  23. 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
  24. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  25. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  26. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  27. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  28. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  29. 2016Generative Multi-Adversarial Networkscited by 60
  30. 2019Unsupervised Generative 3D Shape Learning from Natural Imagescited by 57
  31. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  32. 2018LIME: Live Intrinsic Material Estimationcited by 33
  33. 2021VariTex: Variational Neural Face Texturescited by 30
  34. 2021Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptationcited by 21
  35. 2020SimPose: Effectively Learning DensePose and Surface Normals of People from Simulated Datacited by 17
  36. 2018Geometric Image Synthesiscited by 16
  37. 2019Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Imagescited by 16
  38. 2018Dropout-GAN: Learning from a Dynamic Ensemble of Discriminatorscited by 15

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 15; 2024: 9; 2025: 4; 2026: 4.

37 papers.

Sort:

By area:

  1. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  2. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 74
  3. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  4. 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
  5. 2024URAvatar: Universal Relightable Gaussian Codec Avatarscited by 43
  6. 20223D GAN Inversion with Facial Symmetry Priorcited by 39
  7. 2023Preface: A Data-driven Volumetric Prior for Few-shot Ultra High-resolution Face Synthesiscited by 37
  8. 2024Cafca: High-quality Novel View Synthesis of Expressive Faces from Casual Few-shot Capturescited by 34
  9. 2024Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Imagecited by 32
  10. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatarscited by 29
  11. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  12. 2023UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fieldscited by 26
25 more
  1. 2025SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Facescited by 24
  2. 2023LPFF: A Portrait Dataset for Face Generators Across Large Posescited by 22
  3. 2023Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editingcited by 22
  4. 2023FaceLit: Neural 3D Relightable Facescited by 20
  5. 2023LumiGAN: Unconditional Generation of Relightable 3D Human Facescited by 14
  6. 2024Isotropic3D: Image-to-3D Generation Based on a Single CLIP Embeddingcited by 13
  7. 2024GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Imagecited by 12
  8. 2023Controllable Light Diffusion for Portraitscited by 11
  9. 2023Relightable Neural Actor with Intrinsic Decomposition and Pose Controlcited by 11
  10. 2023Makeup Extraction of 3D Representation via Illumination-Aware Image Decompositioncited by 8
  11. 20233DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Posescited by 8
  12. 2024Learning with Unreliability: Fast Few-shot Voxel Radiance Fields with Relative Geometric Consistencycited by 6
  13. 2024COMPOSE: Comprehensive Portrait Shadow Editingcited by 6
  14. 2025Parametric Shadow Control for Portrait Generation in Text-to-Image Diffusion Modelscited by 5
  15. 2026GR3EN: Generative Relighting for 3D Environmentscited by 5
  16. 2022StyLitGAN: Prompting StyleGAN to Produce New Illumination Conditionscited by 4
  17. 2022NARRATE: A Normal Assisted Free-View Portrait Stylizercited by 3
  18. 2023Towards Flexible, Scalable, and Adaptive Multi-Modal Conditioned Face Synthesiscited by 3
  19. 2022Deep Generative Models on 3D Representations: A Surveycited by 1
  20. 2024Analyzing and Improving the Skin Tone Consistency and Bias in Implicit 3D Relightable Face Generatorscited by 1
  21. 2025POLAR: A Portrait OLAT Dataset and Generative Framework for Illumination-Aware Face Modelingcited by 1
  22. 2024Text-Driven Diverse Facial Texture Generation via Progressive Latent-Space Refinementcited by 0
  23. 2026HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Capturescited by 0
  24. 2026RelightAnyone: A Generalized Relightable 3D Gaussian Head Modelcited by 0
  25. 2026Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproductioncited by 0