# FaceLit: Neural 3D Relightable Faces

arXiv: https://arxiv.org/abs/2303.15437 · HTML: https://mlsanity.com/paper/2303.15437/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids](https://mlsanity.com/paper/2206.07695/) (2022)
- [SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections](https://mlsanity.com/paper/2205.15768/) (2022)
- [LOLNeRF: Learn from One Look](https://mlsanity.com/paper/2111.09996/) (2021)
- [A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis](https://mlsanity.com/paper/2110.15678/) (2021)
- [Lifting 2D StyleGAN for 3D-Aware Face Generation](https://mlsanity.com/paper/2011.13126/) (2020)
- [Generative Multiplane Images: Making a 2D GAN 3D-Aware](https://mlsanity.com/paper/2207.10642/) (2022)
- [GIF: Generative Interpretable Faces](https://mlsanity.com/paper/2009.00149/) (2020)
- [VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting](https://mlsanity.com/paper/2201.04873/) (2022)
- [MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias](https://mlsanity.com/paper/2012.05225/) (2020)

## Built on by (most cited first, 20 of 20)

- [HUGS: Human Gaussian Splats](https://mlsanity.com/paper/2311.17910/) (2023)
- [Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2312.03704/) (2023)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark](https://mlsanity.com/paper/2310.16044/) (2023)
- [MonoNPHM: Dynamic Head Reconstruction from Monocular Videos](https://mlsanity.com/paper/2312.06740/) (2023)
- [Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image](https://mlsanity.com/paper/2403.09632/) (2024)
- [AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars](https://mlsanity.com/paper/2306.00547/) (2023)
- [Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Dataset](https://mlsanity.com/paper/2503.14485/) (2025)
- [Real-time 3D-aware Portrait Video Relighting](https://mlsanity.com/paper/2410.18355/) (2024)
- [GenLit: Reformulating Single-Image Relighting as Video Generation](https://mlsanity.com/paper/2412.11224/) (2024)
- [DiFaReli++: Diffusion Face Relighting with Consistent Cast Shadows](https://mlsanity.com/paper/2304.09479/) (2023)
- [LumiGAN: Unconditional Generation of Relightable 3D Human Faces](https://mlsanity.com/paper/2304.13153/) (2023)
- [Gaussian Shadow Casting for Neural Characters](https://mlsanity.com/paper/2401.06116/) (2024)
- [Relightable Neural Actor with Intrinsic Decomposition and Pose Control](https://mlsanity.com/paper/2312.11587/) (2023)
- [COMPOSE: Comprehensive Portrait Shadow Editing](https://mlsanity.com/paper/2408.13922/) (2024)
- [Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting](https://mlsanity.com/paper/2505.20582/) (2025)
- [Analyzing and Improving the Skin Tone Consistency and Bias in Implicit 3D Relightable Face Generators](https://mlsanity.com/paper/2411.12002/) (2024)
- [HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Captures](https://mlsanity.com/paper/2601.02103/) (2026)
- [RelightAnyone: A Generalized Relightable 3D Gaussian Head Model](https://mlsanity.com/paper/2601.03357/) (2026)
- [Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation](https://mlsanity.com/paper/2603.04307/) (2026)

## Abstract (arXiv)

We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D images in-the-wild without any manual annotation. Unlike existing works that require careful capture setup or human labor, we rely on off-the-shelf pose and illumination estimators. With these estimates, we incorporate the Phong reflectance model in the neural volume rendering framework. Our model learns to generate shape and material properties of a face such that, when rendered according to the natural statistics of pose and illumination, produces photorealistic face images with multiview 3D and illumination consistency. Our method enables photorealistic generation of faces with explicit illumination and view controls on multiple datasets - FFHQ, MetFaces and CelebA-HQ. We show state-of-the-art photorealism among 3D aware GANs on FFHQ dataset achieving an FID score of 3.5.