# Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars

arXiv: https://arxiv.org/abs/2211.11208 · HTML: https://mlsanity.com/paper/2211.11208/

## Key facts

- Cited by 124 papers in the map (data through 14 Oct 2026).
- Builds on 72 of its 88 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 72)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)
- [GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation](https://mlsanity.com/paper/2112.08867/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)

## Built on by (most cited first, 30 of 124)

- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior](https://mlsanity.com/paper/2310.16818/) (2023)
- [Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis](https://mlsanity.com/paper/2401.08503/) (2024)
- [Generalizable and Animatable Gaussian Head Avatar](https://mlsanity.com/paper/2410.07971/) (2024)
- [X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention](https://mlsanity.com/paper/2403.15931/) (2024)
- [GPAvatar: Generalizable and Precise Head Avatar from Image(s)](https://mlsanity.com/paper/2401.10215/) (2024)
- [AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels](https://mlsanity.com/paper/2211.13206/) (2022)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Gaussian Shell Maps for Efficient 3D Human Generation](https://mlsanity.com/paper/2311.17857/) (2023)
- [MIME: Human-Aware 3D Scene Generation](https://mlsanity.com/paper/2212.04360/) (2022)
- [Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data](https://mlsanity.com/paper/2311.18729/) (2023)
- [NOFA: NeRF-based One-shot Facial Avatar Reconstruction](https://mlsanity.com/paper/2307.03441/) (2023)
- [Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars](https://mlsanity.com/paper/2502.20220/) (2025)
- [StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video](https://mlsanity.com/paper/2305.00942/) (2023)
- [LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar](https://mlsanity.com/paper/2305.01190/) (2023)
- [HeadSculpt: Crafting 3D Head Avatars with Text](https://mlsanity.com/paper/2306.03038/) (2023)
- [RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model](https://mlsanity.com/paper/2309.00810/) (2023)
- [HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors](https://mlsanity.com/paper/2408.06019/) (2024)
- [LAM: Large Avatar Model for One-shot Animatable Gaussian Head](https://mlsanity.com/paper/2502.17796/) (2025)
- [Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2509.16630/) (2025)
- [DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars](https://mlsanity.com/paper/2311.18635/) (2023)
- [DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance](https://mlsanity.com/paper/2504.01724/) (2025)
- [HHAvatar: Gaussian Head Avatar with Dynamic Hairs](https://mlsanity.com/paper/2312.03029/) (2023)
- [FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video](https://mlsanity.com/paper/2411.15604/) (2024)
- [Face Generation and Editing with StyleGAN: A Survey](https://mlsanity.com/paper/2212.09102/) (2022)
- [Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Images](https://mlsanity.com/paper/2304.06700/) (2023)
- [Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation](https://mlsanity.com/paper/2401.04728/) (2024)
- [Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing](https://mlsanity.com/paper/2312.03763/) (2023)
- [Generalizable One-shot Neural Head Avatar](https://mlsanity.com/paper/2306.08768/) (2023)
- [TextToon: Real-Time Text Toonify Head Avatar from Single Video](https://mlsanity.com/paper/2410.07160/) (2024)

## Abstract (arXiv)

3D-aware generative adversarial networks (GANs) synthesize high-fidelity and multi-view-consistent facial images using only collections of single-view 2D imagery. Towards fine-grained control over facial attributes, recent efforts incorporate 3D Morphable Face Model (3DMM) to describe deformation in generative radiance fields either explicitly or implicitly. Explicit methods provide fine-grained expression control but cannot handle topological changes caused by hair and accessories, while implicit ones can model varied topologies but have limited generalization caused by the unconstrained deformation fields. We propose a novel 3D GAN framework for unsupervised learning of generative, high-quality and 3D-consistent facial avatars from unstructured 2D images. To achieve both deformation accuracy and topological flexibility, we propose a 3D representation called Generative Texture-Rasterized Tri-planes. The proposed representation learns Generative Neural Textures on top of parametric mesh templates and then projects them into three orthogonal-viewed feature planes through rasterization, forming a tri-plane feature representation for volume rendering. In this way, we combine both fine-grained expression control of mesh-guided explicit deformation and the flexibility of implicit volumetric representation. We further propose specific modules for modeling mouth interior which is not taken into account by 3DMM. Our method demonstrates state-of-the-art 3D-aware synthesis quality and animation ability through extensive experiments. Furthermore, serving as 3D prior, our animatable 3D representation boosts multiple applications including one-shot facial avatars and 3D-aware stylization.