# Lifting 2D StyleGAN for 3D-Aware Face Generation

arXiv: https://arxiv.org/abs/2011.13126 · HTML: https://mlsanity.com/paper/2011.13126/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 30 of 64)

- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation](https://mlsanity.com/paper/2112.08867/) (2021)
- [VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids](https://mlsanity.com/paper/2206.07695/) (2022)
- [EpiGRAF: Rethinking training of 3D GANs](https://mlsanity.com/paper/2206.10535/) (2022)
- [PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circ](https://mlsanity.com/paper/2303.13071/) (2023)
- [GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifolds](https://mlsanity.com/paper/2206.07255/) (2022)
- [Real-Time Radiance Fields for Single-Image Portrait View Synthesis](https://mlsanity.com/paper/2305.02310/) (2023)
- [GAN-Supervised Dense Visual Alignment](https://mlsanity.com/paper/2112.05143/) (2021)
- [Generative Multiplane Images: Making a 2D GAN 3D-Aware](https://mlsanity.com/paper/2207.10642/) (2022)
- [Learning Locally Editable Virtual Humans](https://mlsanity.com/paper/2305.00121/) (2023)
- [3D GAN Inversion for Controllable Portrait Image Animation](https://mlsanity.com/paper/2203.13441/) (2022)
- [TriPlaneNet: An Encoder for EG3D Inversion](https://mlsanity.com/paper/2303.13497/) (2023)
- [3DAvatarGAN: Bridging Domains for Personalized Editable Avatars](https://mlsanity.com/paper/2301.02700/) (2023)
- [Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields](https://mlsanity.com/paper/2206.08361/) (2022)
- [Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding](https://mlsanity.com/paper/2303.12326/) (2023)
- [OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis](https://mlsanity.com/paper/2303.15539/) (2023)
- [DATID-3D: Diversity-Preserved Domain Adaptation Using Text-to-Image Diffusion for 3D Generative Model](https://mlsanity.com/paper/2211.16374/) (2022)
- [VQ3D: Learning a 3D-Aware Generative Model on ImageNet](https://mlsanity.com/paper/2302.06833/) (2023)
- [One-Shot Adaptation of GAN in Just One CLIP](https://mlsanity.com/paper/2203.09301/) (2022)
- [SphereHead: Stable 3D Full-head Synthesis with Spherical Tri-plane Representation](https://mlsanity.com/paper/2404.05680/) (2024)
- [PV3D: A 3D Generative Model for Portrait Video Generation](https://mlsanity.com/paper/2212.06384/) (2022)
- [3DFaceShop: Explicitly Controllable 3D-Aware Portrait Generation](https://mlsanity.com/paper/2209.05434/) (2022)
- [FaceLit: Neural 3D Relightable Faces](https://mlsanity.com/paper/2303.15437/) (2023)
- [Towards Realistic Generative 3D Face Models](https://mlsanity.com/paper/2304.12483/) (2023)
- [CHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Images](https://mlsanity.com/paper/2308.12288/) (2023)
- [Mimic3D: Thriving 3D-Aware GANs via 3D-to-2D Imitation](https://mlsanity.com/paper/2303.09036/) (2023)
- [3D-FM GAN: Towards 3D-Controllable Face Manipulation](https://mlsanity.com/paper/2208.11257/) (2022)
- [XAGen: 3D Expressive Human Avatars Generation](https://mlsanity.com/paper/2311.13574/) (2023)
- [A Survey on Deep Generative 3D-aware Image Synthesis](https://mlsanity.com/paper/2210.14267/) (2022)
- [AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections](https://mlsanity.com/paper/2309.02186/) (2023)

## Abstract (arXiv)

We propose a framework, called LiftedGAN, that disentangles and lifts a pre-trained StyleGAN2 for 3D-aware face generation. Our model is "3D-aware" in the sense that it is able to (1) disentangle the latent space of StyleGAN2 into texture, shape, viewpoint, lighting and (2) generate 3D components for rendering synthetic images. Unlike most previous methods, our method is completely self-supervised, i.e. it neither requires any manual annotation nor 3DMM model for training. Instead, it learns to generate images as well as their 3D components by distilling the prior knowledge in StyleGAN2 with a differentiable renderer. The proposed model is able to output both the 3D shape and texture, allowing explicit pose and lighting control over generated images. Qualitative and quantitative results show the superiority of our approach over existing methods on 3D-controllable GANs in content controllability while generating realistic high quality images.