# Controllable 3D Generative Adversarial Face Model via Disentangling Shape and Appearance

arXiv: https://arxiv.org/abs/2208.14263 · HTML: https://mlsanity.com/paper/2208.14263/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Builds on (most cited first, 27 of 27)

- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Learning Formation of Physically-Based Face Attributes](https://mlsanity.com/paper/2004.03458/) (2020)
- [AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"](https://mlsanity.com/paper/2003.13845/) (2020)
- [Variational Autoencoders for Deforming 3D Mesh Models](https://mlsanity.com/paper/1709.04307/) (2017)
- [3D Face Reconstruction by Learning from Synthetic Data](https://mlsanity.com/paper/1609.04387/) (2016)
- [UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition](https://mlsanity.com/paper/1712.04695/) (2017)
- [Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation](https://mlsanity.com/paper/1703.10131/) (2017)
- [Disentangled Representation Learning for 3D Face Shape](https://mlsanity.com/paper/1902.09887/) (2019)
- [Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks](https://mlsanity.com/paper/1909.02215/) (2019)
- [Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model](https://mlsanity.com/paper/1804.03675/) (2018)
- [3DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translation](https://mlsanity.com/paper/1905.00307/) (2019)
- [MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation](https://mlsanity.com/paper/2111.01048/) (2021)
- [High Quality Facial Surface and Texture Synthesis via Generative Adversarial Networks](https://mlsanity.com/paper/1808.08281/) (2018)
- [Synthesizing facial photometries and corresponding geometries using generative adversarial networks](https://mlsanity.com/paper/1901.06551/) (2019)
- [FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer Using Neural Generative Adversarial Networks](https://mlsanity.com/paper/2112.00532/) (2021)
- [A Decoupled 3D Facial Shape Model by Adversarial Training](https://mlsanity.com/paper/1902.03619/) (2019)

## Built on by (most cited first, 8 of 8)

- [Towards Realistic Generative 3D Face Models](https://mlsanity.com/paper/2304.12483/) (2023)
- [FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation](https://mlsanity.com/paper/2412.02690/) (2024)
- [UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping](https://mlsanity.com/paper/2502.01846/) (2025)
- [FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Models](https://mlsanity.com/paper/2312.04465/) (2023)
- [Synthesizing Iris Images using Generative Adversarial Networks: Survey and Comparative Analysis](https://mlsanity.com/paper/2404.17105/) (2024)
- [3D-Aware Talking-Head Video Motion Transfer](https://mlsanity.com/paper/2311.02549/) (2023)
- [Spherical Geometry Diffusion: Generating High-quality 3D Face Geometry via Sphere-anchored Representations](https://mlsanity.com/paper/2601.13371/) (2026)
- [MUNCH: Modelling Unique 'N Controllable Heads](https://mlsanity.com/paper/2310.02753/) (2023)

## Abstract (arXiv)

3D face modeling has been an active area of research in computer vision and computer graphics, fueling applications ranging from facial expression transfer in virtual avatars to synthetic data generation. Existing 3D deep learning generative models (e.g., VAE, GANs) allow generating compact face representations (both shape and texture) that can model non-linearities in the shape and appearance space (e.g., scatter effects, specularities, etc.). However, they lack the capability to control the generation of subtle expressions. This paper proposes a new 3D face generative model that can decouple identity and expression and provides granular control over expressions. In particular, we propose using a pair of supervised auto-encoder and generative adversarial networks to produce high-quality 3D faces, both in terms of appearance and shape. Experimental results in the generation of 3D faces learned with holistic expression labels, or Action Unit labels, show how we can decouple identity and expression; gaining fine-control over expressions while preserving identity.