# GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar

arXiv: https://arxiv.org/abs/2311.13655 · HTML: https://mlsanity.com/paper/2311.13655/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 66 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 30 of 44)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [Learning to Reconstruct People in Clothing from a Single RGB Camera](https://mlsanity.com/paper/1903.05885/) (2019)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [Pixel Codec Avatars](https://mlsanity.com/paper/2104.04638/) (2021)
- [Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars](https://mlsanity.com/paper/2211.11208/) (2022)
- [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020)
- [CHORE: Contact, Human and Object REconstruction from a single RGB image](https://mlsanity.com/paper/2204.02445/) (2022)
- [PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circ](https://mlsanity.com/paper/2303.13071/) (2023)

## Built on by (most cited first, 14 of 14)

- [Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars](https://mlsanity.com/paper/2502.20220/) (2025)
- [Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models](https://mlsanity.com/paper/2406.08475/) (2024)
- [Gaussian Eigen Models for Human Heads](https://mlsanity.com/paper/2407.04545/) (2024)
- [Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy](https://mlsanity.com/paper/2412.06698/) (2024)
- [3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing](https://mlsanity.com/paper/2312.00870/) (2023)
- [Synthetic Prior for Few-Shot Drivable Head Avatar Inversion](https://mlsanity.com/paper/2501.06903/) (2025)
- [Gaussian Deja-vu: Creating Controllable 3D Gaussian Head-Avatars with Enhanced Generalization and Personalization Abilities](https://mlsanity.com/paper/2409.16147/) (2024)
- [3DiFACE: Synthesizing and Editing Holistic 3D Facial Animation](https://mlsanity.com/paper/2509.26233/) (2025)
- [PhysHead: Simulation-Ready Gaussian Head Avatars](https://mlsanity.com/paper/2604.06467/) (2026)
- [StyleMorpheus: A Style-Based 3D-Aware Morphable Face Model](https://mlsanity.com/paper/2503.11792/) (2025)
- [TimeWalker: Personalized Neural Space for Lifelong Head Avatars](https://mlsanity.com/paper/2412.02421/) (2024)
- [Gaussian Pixel Codec Avatars: A Hybrid Representation for Efficient Rendering](https://mlsanity.com/paper/2512.15711/) (2025)
- [3DRealHead: Few-Shot Detailed Head Avatar](https://mlsanity.com/paper/2604.13171/) (2026)
- [Head Avatars with Dynamic Explicit Hair](https://mlsanity.com/paper/2607.23861/) (2026)

## Abstract (arXiv)

Digital humans and, especially, 3D facial avatars have raised a lot of attention in the past years, as they are the backbone of several applications like immersive telepresence in AR or VR. Despite the progress, facial avatars reconstructed from commodity hardware are incomplete and miss out on parts of the side and back of the head, severely limiting the usability of the avatar. This limitation in prior work stems from their requirement of face tracking, which fails for profile and back views. To address this issue, we propose to learn person-specific animatable avatars from images without assuming to have access to precise facial expression tracking. At the core of our method, we leverage a 3D-aware generative model that is trained to reproduce the distribution of facial expressions from the training data. To train this appearance model, we only assume to have a collection of 2D images with the corresponding camera parameters. For controlling the model, we learn a mapping from 3DMM facial expression parameters to the latent space of the generative model. This mapping can be learned by sampling the latent space of the appearance model and reconstructing the facial parameters from a normalized frontal view, where facial expression estimation performs well. With this scheme, we decouple 3D appearance reconstruction and animation control to achieve high fidelity in image synthesis. In a series of experiments, we compare our proposed technique to state-of-the-art monocular methods and show superior quality while not requiring expression tracking of the training data.