# GAN-Control: Explicitly Controllable GANs

arXiv: https://arxiv.org/abs/2101.02477 · HTML: https://mlsanity.com/paper/2101.02477/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 62 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [cGANs with Projection Discriminator](https://mlsanity.com/paper/1802.05637/) (2018)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)

## Built on by (most cited first, 30 of 91)

- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Human Motion Generation: A Survey](https://mlsanity.com/paper/2307.10894/) (2023)
- [IDiff-Face: Synthetic-based Face Recognition through Fizzy Identity-Conditioned Diffusion Models](https://mlsanity.com/paper/2308.04995/) (2023)
- [OBJECT 3DIT: Language-guided 3D-aware Image Editing](https://mlsanity.com/paper/2307.11073/) (2023)
- [SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing](https://mlsanity.com/paper/2112.02236/) (2021)
- [Third Time's the Charm? Image and Video Editing with StyleGAN3](https://mlsanity.com/paper/2201.13433/) (2022)
- [Style and Pose Control for Image Synthesis of Humans from a Single Monocular View](https://mlsanity.com/paper/2102.11263/) (2021)
- [StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video](https://mlsanity.com/paper/2305.00942/) (2023)
- [RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model](https://mlsanity.com/paper/2309.00810/) (2023)
- [Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN](https://mlsanity.com/paper/2109.06166/) (2021)
- [StyleGAN knows Normal, Depth, Albedo, and More](https://mlsanity.com/paper/2306.00987/) (2023)
- [Synthetic Data for Face Recognition: Current State and Future Prospects](https://mlsanity.com/paper/2305.01021/) (2023)
- [Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields](https://mlsanity.com/paper/2206.08361/) (2022)
- [OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis](https://mlsanity.com/paper/2303.15539/) (2023)
- [VariTex: Variational Neural Face Textures](https://mlsanity.com/paper/2104.05988/) (2021)
- [PrintsGAN: Synthetic Fingerprint Generator](https://mlsanity.com/paper/2201.03674/) (2022)
- [Finding Directions in GAN's Latent Space for Neural Face Reenactment](https://mlsanity.com/paper/2202.00046/) (2022)
- [Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models](https://mlsanity.com/paper/2209.06970/) (2022)
- [One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations](https://mlsanity.com/paper/2210.07883/) (2022)
- [3DFaceShop: Explicitly Controllable 3D-Aware Portrait Generation](https://mlsanity.com/paper/2209.05434/) (2022)
- [ExFaceGAN: Exploring Identity Directions in GAN's Learned Latent Space for Synthetic Identity Generation](https://mlsanity.com/paper/2307.05151/) (2023)
- [Controllable Data Generation by Deep Learning: A Review](https://mlsanity.com/paper/2207.09542/) (2022)
- [ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentation](https://mlsanity.com/paper/2312.10998/) (2023)
- [3D-Aware Video Generation](https://mlsanity.com/paper/2206.14797/) (2022)
- [3D-FM GAN: Towards 3D-Controllable Face Manipulation](https://mlsanity.com/paper/2208.11257/) (2022)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)

## Abstract (arXiv)

We present a framework for training GANs with explicit control over generated images. We are able to control the generated image by settings exact attributes such as age, pose, expression, etc. Most approaches for editing GAN-generated images achieve partial control by leveraging the latent space disentanglement properties, obtained implicitly after standard GAN training. Such methods are able to change the relative intensity of certain attributes, but not explicitly set their values. Recently proposed methods, designed for explicit control over human faces, harness morphable 3D face models to allow fine-grained control capabilities in GANs. Unlike these methods, our control is not constrained to morphable 3D face model parameters and is extendable beyond the domain of human faces. Using contrastive learning, we obtain GANs with an explicitly disentangled latent space. This disentanglement is utilized to train control-encoders mapping human-interpretable inputs to suitable latent vectors, thus allowing explicit control. In the domain of human faces we demonstrate control over identity, age, pose, expression, hair color and illumination. We also demonstrate control capabilities of our framework in the domains of painted portraits and dog image generation. We demonstrate that our approach achieves state-of-the-art performance both qualitatively and quantitatively.