# VariTex: Variational Neural Face Textures

arXiv: https://arxiv.org/abs/2104.05988 · HTML: https://mlsanity.com/paper/2104.05988/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis](https://mlsanity.com/paper/2103.11078/) (2021)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Neural Voice Puppetry: Audio-driven Facial Reenactment](https://mlsanity.com/paper/1912.05566/) (2019)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)
- [ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation](https://mlsanity.com/paper/2007.15837/) (2020)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [GAN-Control: Explicitly Controllable GANs](https://mlsanity.com/paper/2101.02477/) (2021)
- [SofGAN: A Portrait Image Generator with Dynamic Styling](https://mlsanity.com/paper/2007.03780/) (2020)
- [UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition](https://mlsanity.com/paper/1712.04695/) (2017)
- [CONFIG: Controllable Neural Face Image Generation](https://mlsanity.com/paper/2005.02671/) (2020)
- [ANR: Articulated Neural Rendering for Virtual Avatars](https://mlsanity.com/paper/2012.12890/) (2020)
- [GIF: Generative Interpretable Faces](https://mlsanity.com/paper/2009.00149/) (2020)

## Built on by (most cited first, 30 of 30)

- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints](https://mlsanity.com/paper/2205.04992/) (2022)
- [AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels](https://mlsanity.com/paper/2211.13206/) (2022)
- [AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars](https://mlsanity.com/paper/2210.06465/) (2022)
- [VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting](https://mlsanity.com/paper/2201.04873/) (2022)
- [FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video](https://mlsanity.com/paper/2411.15604/) (2024)
- [HARP: Personalized Hand Reconstruction from a Monocular RGB Video](https://mlsanity.com/paper/2212.09530/) (2022)
- [MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation](https://mlsanity.com/paper/2111.01048/) (2021)
- [Towards Robust and Unconstrained Full Range of Rotation Head Pose Estimation](https://mlsanity.com/paper/2309.07654/) (2023)
- [DINER: Depth-aware Image-based NEural Radiance fields](https://mlsanity.com/paper/2211.16630/) (2022)
- [3D-FM GAN: Towards 3D-Controllable Face Manipulation](https://mlsanity.com/paper/2208.11257/) (2022)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)
- [A Survey on Deep Generative 3D-aware Image Synthesis](https://mlsanity.com/paper/2210.14267/) (2022)
- [3DPR: Single Image 3D Portrait Relight using Generative Priors](https://mlsanity.com/paper/2510.15846/) (2025)
- [3D-Aware Talking-Head Video Motion Transfer](https://mlsanity.com/paper/2311.02549/) (2023)
- [SelfSwapper: Self-Supervised Face Swapping via Shape Agnostic Masked AutoEncoder](https://mlsanity.com/paper/2402.07370/) (2024)
- [Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping](https://mlsanity.com/paper/2405.12069/) (2024)
- [Dream, Lift, Animate: From Single Images to Animatable Gaussian Avatars](https://mlsanity.com/paper/2507.15979/) (2025)
- [HyperGaussians: High-Dimensional Gaussian Splatting for High-Fidelity Animatable Face Avatars](https://mlsanity.com/paper/2507.02803/) (2025)
- [Causal Representation Learning for Context-Aware Face Transfer](https://mlsanity.com/paper/2110.01571/) (2021)
- [PAV: Personalized Head Avatar from Unstructured Video Collection](https://mlsanity.com/paper/2407.21047/) (2024)
- [FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces](https://mlsanity.com/paper/2404.13807/) (2024)
- [Instruction-Driven 3D Facial Expression Generation and Transition](https://mlsanity.com/paper/2601.08179/) (2026)
- [GAT-NeRF: Geometry-Aware-Transformer Enhanced Neural Radiance Fields for High-Fidelity 4D Facial Avatars](https://mlsanity.com/paper/2601.14875/) (2026)
- [NBAvatar: Neural Billboards Avatars with Realistic Hand-Face Interaction](https://mlsanity.com/paper/2603.12063/) (2026)
- [LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space](https://mlsanity.com/paper/2203.07881/) (2022)
- [Giving Faces Their Feelings Back: Explicit Emotion Control for Feedforward Single-Image 3D Head Avatars](https://mlsanity.com/paper/2604.14541/) (2026)

## Abstract (arXiv)

Deep generative models can synthesize photorealistic images of human faces with novel identities. However, a key challenge to the wide applicability of such techniques is to provide independent control over semantically meaningful parameters: appearance, head pose, face shape, and facial expressions. In this paper, we propose VariTex - to the best of our knowledge the first method that learns a variational latent feature space of neural face textures, which allows sampling of novel identities. We combine this generative model with a parametric face model and gain explicit control over head pose and facial expressions. To generate complete images of human heads, we propose an additive decoder that adds plausible details such as hair. A novel training scheme enforces a pose-independent latent space and in consequence, allows learning a one-to-many mapping between latent codes and pose-conditioned exterior regions. The resulting method can generate geometrically consistent images of novel identities under fine-grained control over head pose, face shape, and facial expressions. This facilitates a broad range of downstream tasks, like sampling novel identities, changing the head pose, expression transfer, and more. Code and models are available for research on https://mcbuehler.github.io/VariTex.