# MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation

arXiv: https://arxiv.org/abs/2111.01048 · HTML: https://mlsanity.com/paper/2111.01048/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 78 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 30 of 55)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [NIPS 2016 Tutorial: Generative Adversarial Networks](https://mlsanity.com/paper/1701.00160/) (2016)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)
- [Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision](https://mlsanity.com/paper/1905.06817/) (2019)
- [BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images](https://mlsanity.com/paper/2002.08988/) (2020)
- [Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images](https://mlsanity.com/paper/1901.11153/) (2019)
- [MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction](https://mlsanity.com/paper/1703.10580/) (2017)
- [Fine-Grained Head Pose Estimation Without Keypoints](https://mlsanity.com/paper/1710.00925/) (2017)

## Built on by (most cited first, 24 of 24)

- [DCFace: Synthetic Face Generation with Dual Condition Diffusion Model](https://mlsanity.com/paper/2304.07060/) (2023)
- [Proactive Image Manipulation Detection](https://mlsanity.com/paper/2203.15880/) (2022)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [MaLP: Manipulation Localization Using a Proactive Scheme](https://mlsanity.com/paper/2303.16976/) (2023)
- [Controllable and Guided Face Synthesis for Unconstrained Face Recognition](https://mlsanity.com/paper/2207.10180/) (2022)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)
- [Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion](https://mlsanity.com/paper/2504.00430/) (2025)
- [2D GANs Meet Unsupervised Single-view 3D Reconstruction](https://mlsanity.com/paper/2207.10183/) (2022)
- [Controllable 3D Generative Adversarial Face Model via Disentangling Shape and Appearance](https://mlsanity.com/paper/2208.14263/) (2022)
- [Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Network](https://mlsanity.com/paper/2312.02224/) (2023)
- [LatentSwap3D: Semantic Edits on 3D Image GANs](https://mlsanity.com/paper/2212.01381/) (2022)
- [Fooling State-of-the-Art Deepfake Detection with High-Quality Deepfakes](https://mlsanity.com/paper/2305.05282/) (2023)
- [MyPortrait: Morphable Prior-Guided Personalized Portrait Generation](https://mlsanity.com/paper/2312.02703/) (2023)
- [Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection](https://mlsanity.com/paper/2212.05667/) (2022)
- [Pose Impact Estimation on Face Recognition using 3D-Aware Synthetic Data with Application to Quality Assessment](https://mlsanity.com/paper/2303.00491/) (2023)
- [Facing Asymmetry -- Uncovering the Causal Link between Facial Symmetry and Expression Classifiers using Synthetic Interventions](https://mlsanity.com/paper/2409.15927/) (2024)
- [FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces](https://mlsanity.com/paper/2404.13807/) (2024)
- [Electromyography-Informed Facial Expression Reconstruction for Physiological-Based Synthesis and Analysis](https://mlsanity.com/paper/2503.09556/) (2025)
- [Controllable Face Manipulation and UV Map Generation by Self-supervised Learning](https://mlsanity.com/paper/2209.12050/) (2022)
- [Survey on Controlable Image Synthesis with Deep Learning](https://mlsanity.com/paper/2307.10275/) (2023)
- [UP-FacE: User-predictable Fine-grained Face Shape Editing](https://mlsanity.com/paper/2403.13972/) (2024)
- [HAIFAI: Human-AI Interaction for Mental Face Reconstruction](https://mlsanity.com/paper/2412.06323/) (2024)
- [SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation](https://mlsanity.com/paper/2605.30894/) (2026)
- [MirrorPPR: Exemplar-Based Portrait Photo Retouching](https://mlsanity.com/paper/2606.29308/) (2026)

## Abstract (arXiv)

Recent advances in generative adversarial networks (GANs) have led to remarkable achievements in face image synthesis. While methods that use style-based GANs can generate strikingly photorealistic face images, it is often difficult to control the characteristics of the generated faces in a meaningful and disentangled way. Prior approaches aim to achieve such semantic control and disentanglement within the latent space of a previously trained GAN. In contrast, we propose a framework that a priori models physical attributes of the face such as 3D shape, albedo, pose, and lighting explicitly, thus providing disentanglement by design. Our method, MOST-GAN, integrates the expressive power and photorealism of style-based GANs with the physical disentanglement and flexibility of nonlinear 3D morphable models, which we couple with a state-of-the-art 2D hair manipulation network. MOST-GAN achieves photorealistic manipulation of portrait images with fully disentangled 3D control over their physical attributes, enabling extreme manipulation of lighting, facial expression, and pose variations up to full profile view.