GAN-Control: Explicitly Controllable GANs
Not on the map. Why?
Abstract (arXiv)
We present a framework for training GANs with explicit control over generated images. We are able to control the generated image by settings exact attributes such as age, pose, expression, etc. Most approaches for editing GAN-generated images achieve partial control by leveraging the latent space disentanglement properties, obtained implicitly after standard GAN training. Such methods are able to change the relative intensity of certain attributes, but not explicitly set their values. Recently proposed methods, designed for explicit control over human faces, harness morphable 3D face models to allow fine-grained control capabilities in GANs. Unlike these methods, our control is not constrained to morphable 3D face model parameters and is extendable beyond the domain of human faces. Using contrastive learning, we obtain GANs with an explicitly disentangled latent space. This disentanglement is utilized to train control-encoders mapping human-interpretable inputs to suitable latent vectors, thus allowing explicit control. In the domain of human faces we demonstrate control over identity, age, pose, expression, hair color and illumination. We also demonstrate control capabilities of our framework in the domains of painted portraits and dog image generation. We demonstrate that our approach achieves state-of-the-art performance both qualitatively and quantitatively.
Key facts. Cited by 91 papers in the map (data through 7 Oct 2026). Builds on 43 of its 62 references that are in the map. First posted on arXiv 7 Jan 2021.
Builds on
43 of 62 references.
References · 43
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2016Pyramid Scene Parsing Networkcited by 2,472
- 2014Conditional Generative Adversarial Netscited by 2,152
- 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
- 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
31 more
- 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
- 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
- 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
- 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
- 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
- 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
- 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
- 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
- 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
- 2020In-Domain GAN Inversion for Real Image Editingcited by 420
- 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
- 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
- 20193D Morphable Face Models -- Past, Present and Futurecited by 269
- 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
- 2018cGANs with Projection Discriminatorcited by 253
- 2019On the "steerability" of generative adversarial networkscited by 247
- 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
- 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 219
- 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
- 2017Fine-Grained Head Pose Estimation Without Keypointscited by 163
- 2019Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesiscited by 106
- 2018Towards Open-Set Identity Preserving Face Synthesiscited by 74
- 2020Towards causal benchmarking of bias in face analysis algorithmscited by 64
- 2020CONFIG: Controllable Neural Face Image Generationcited by 61
- 2019HoloGAN: Unsupervised learning of 3D representations from natural imagescited by 59
- 2020GAN "Steerability" without optimizationcited by 48
- 2019Breaking the cycle -- Colleagues are all you needcited by 40
- 2020Efficient Facial Feature Learning with Wide Ensemble-based Convolutional Neural Networkscited by 24
- 2020High Resolution Face Age Editingcited by 23
- 2017Semantically Decomposing the Latent Spaces of Generative Adversarial Networkscited by 19
- 2018FaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face Synthesiscited by 14
Built on by
Citing papers per year: 2021: 14; 2022: 26; 2023: 28; 2024: 11; 2025: 8; 2026: 4.
91 papers.
Sort:
By area:
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
- 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
- 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
- 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
- 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
- 2023Human Motion Generation: A Surveycited by 95
- 2023IDiff-Face: Synthetic-based Face Recognition through Fizzy Identity-Conditioned Diffusion Modelscited by 72
- 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
- 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
- 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
- 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
- 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
38 more
- 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
- 2021Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGANcited by 45
- 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
- 2023Synthetic Data for Face Recognition: Current State and Future Prospectscited by 38
- 2022Controllable 3D Face Synthesis with Conditional Generative Occupancy Fieldscited by 37
- 2023OmniAvatar: Geometry-Guided Controllable 3D Head Synthesiscited by 34
- 2021VariTex: Variational Neural Face Texturescited by 30
- 2022PrintsGAN: Synthetic Fingerprint Generatorcited by 30
- 2022Finding Directions in GAN's Latent Space for Neural Face Reenactmentcited by 27
- 2022Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Modelscited by 27
- 2022One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulationscited by 24
- 20223DFaceShop: Explicitly Controllable 3D-Aware Portrait Generationcited by 22
- 2023ExFaceGAN: Exploring Identity Directions in GAN's Learned Latent Space for Synthetic Identity Generationcited by 22
- 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
- 2023ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentationcited by 20
- 20223D-Aware Video Generationcited by 17
- 20223D-FM GAN: Towards 3D-Controllable Face Manipulationcited by 17
- 2022GAN-based Facial Attribute Manipulationcited by 16
- 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
- 2023LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesiscited by 15
- 2021StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGANcited by 14
- 2024Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Datacited by 13
- 2023Painterly Image Harmonization using Diffusion Modelcited by 12
- 2024Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusioncited by 12
- 2022Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesiscited by 11
- 2025Flux Already Knows -- Activating Subject-Driven Image Generation without Trainingcited by 11
- 2024Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Surveycited by 9
- 2022IA-FaceS: A Bidirectional Method for Semantic Face Editingcited by 8
- 2023MyStyle++: A Controllable Personalized Generative Priorcited by 8
- 2021DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editingcited by 7
- 2021A Multi-attribute Controllable Generative Model for Histopathology Image Synthesiscited by 7
- 2023BallGAN: 3D-aware Image Synthesis with a Spherical Backgroundcited by 7
- 2023NAISR: A 3D Neural Additive Model for Interpretable Shape Representationcited by 7
- 2021Transforming the Latent Space of StyleGAN for Real Face Editingcited by 6
- 20233D Generative Model Latent Disentanglement via Local Eigenprojectioncited by 6
- 2023FPGAN-Control: A Controllable Fingerprint Generator for Training with Synthetic Datacited by 6
- 2024One-shot Neural Face Reenactment via Finding Directions in GAN's Latent Spacecited by 6
- 2025CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesiscited by 6
50 most cited of 91 All 91 (JSON)
- 2026EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment21 Aug 2026
- 2026Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shaping11 May 2026
- 2026Scalable Secure Biometric Authentication without Auxiliary Identifiers27 Apr 2026
- 2026HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Captures5 Jan 2026
- 2025ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer13 Oct 2025
- 2025Latent Diffusion : Multi-Dimension Stable Diffusion Latent Space Explorer26 Sep 2025
- 2025Attributes Shape the Embedding Space of Face Recognition Models15 Jul 2025
- 2025CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesis23 May 2025
- 2025PCDiff: Proactive Control for Ownership Protection in Diffusion Models with Watermark Compatibility16 Apr 2025
- 2025Flux Already Knows -- Activating Subject-Driven Image Generation without Training12 Apr 2025
- 2025Stochastic Trajectory Prediction under Unstructured Constraints18 Mar 2025
- 2025LucidAtlas: Learning Uncertainty-Aware, Covariate-Disentangled, Individualized Atlas Representations12 Feb 2025
38 more
- 2024Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data2 Dec 2024
- 2024Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey5 Nov 2024
- 2024The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition4 Sep 2024
- 2024SAT3D: Image-driven Semantic Attribute Transfer in 3D3 Aug 2024
- 2024Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations29 May 2024
- 2024Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusion30 Apr 2024
- 2024PANet: A Physics-guided Parametric Augmentation Net for Image Dehazing by Hazing14 Apr 2024
- 2024Identification of Fine-grained Systematic Errors via Controlled Scene Generation10 Apr 2024
- 2024If It's Not Enough, Make It So: Reducing Authentic Data Demand in Face Recognition through Synthetic Faces4 Apr 2024
- 2024Disentangling Racial Phenotypes: Fine-Grained Control of Race-related Facial Phenotype Characteristics29 Mar 2024
- 2024One-shot Neural Face Reenactment via Finding Directions in GAN's Latent Space5 Feb 2024
- 2023EyePreserve: Identity-Preserving Iris Synthesis19 Dec 2023
- 2023ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentation18 Dec 2023
- 2023Multi-view Inversion for 3D-aware Generative Adversarial Networks8 Dec 2023
- 2023VerA: Versatile Anonymization Applicable to Clinical Facial Photographs4 Dec 2023
- 2023Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification15 Nov 2023
- 2023FPGAN-Control: A Controllable Fingerprint Generator for Training with Synthetic Data29 Oct 2023
- 2023Customize StyleGAN with One Hand Sketch29 Oct 2023
- 2023Subjective Face Transform using Human First Impressions27 Sep 2023
- 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model2 Sep 2023
- 2023IDiff-Face: Synthetic-based Face Recognition through Fizzy Identity-Conditioned Diffusion Models9 Aug 2023
- 2023Painterly Image Harmonization using Diffusion Model4 Aug 2023
- 2023OBJECT 3DIT: Language-guided 3D-aware Image Editing20 Jul 2023
- 2023Human Motion Generation: A Survey20 Jul 2023
- 2023ExFaceGAN: Exploring Identity Directions in GAN's Learned Latent Space for Synthetic Identity Generation11 Jul 2023
- 2023MyStyle++: A Controllable Personalized Generative Prior8 Jun 2023
- 2023StyleGAN knows Normal, Depth, Albedo, and More1 Jun 2023
- 2023Urban-StyleGAN: Learning to Generate and Manipulate Images of Urban Scenes16 May 2023
- 2023Synthetic Data for Face Recognition: Current State and Future Prospects1 May 2023
- 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video1 May 2023
- 2023MetaHead: An Engine to Create Realistic Digital Head3 Apr 2023
- 2023PrefGen: Preference Guided Image Generation with Relative Attributes1 Apr 2023
- 2023OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis27 Mar 2023
- 2023NeRF-GAN Distillation for Efficient 3D-Aware Generation with Convolutions22 Mar 2023
- 2023NAISR: A 3D Neural Additive Model for Interpretable Shape Representation16 Mar 2023
- 2023Self-Supervised One-Shot Learning for Automatic Segmentation of StyleGAN Images10 Mar 2023
- 20233D Generative Model Latent Disentanglement via Local Eigenprojection24 Feb 2023
- 2023BallGAN: 3D-aware Image Synthesis with a Spherical Background22 Jan 2023
50 newest of 91 All 91 (JSON)
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
- 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
- 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
- 2023IDiff-Face: Synthetic-based Face Recognition through Fizzy Identity-Conditioned Diffusion Modelscited by 72
- 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
- 2022Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Modelscited by 27
- 2023Painterly Image Harmonization using Diffusion Modelcited by 12
- 2024Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolationscited by 5
- 2025PCDiff: Proactive Control for Ownership Protection in Diffusion Models with Watermark Compatibilitycited by 1
- 2025Latent Diffusion : Multi-Dimension Stable Diffusion Latent Space Explorercited by 0
- 2025Latent Diffusion : Multi-Dimension Stable Diffusion Latent Space Explorer26 Sep 2025
- 2025PCDiff: Proactive Control for Ownership Protection in Diffusion Models with Watermark Compatibility16 Apr 2025
- 2024Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations29 May 2024
- 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model2 Sep 2023
- 2023IDiff-Face: Synthetic-based Face Recognition through Fizzy Identity-Conditioned Diffusion Models9 Aug 2023
- 2023Painterly Image Harmonization using Diffusion Model4 Aug 2023
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct 2022
- 2022Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models14 Sep 2022
- 2022Compositional Visual Generation with Composable Diffusion Models3 Jun 2022
- 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance10 Dec 2021
- 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
- 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
- 2023Human Motion Generation: A Surveycited by 95
- 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
- 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
- 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
- 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
- 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
- 2021Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGANcited by 45
- 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
- 2023Synthetic Data for Face Recognition: Current State and Future Prospectscited by 38
- 2022Controllable 3D Face Synthesis with Conditional Generative Occupancy Fieldscited by 37
12 most cited of 80 All 80 (JSON)
- 2026EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment21 Aug 2026
- 2026Scalable Secure Biometric Authentication without Auxiliary Identifiers27 Apr 2026
- 2026HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Captures5 Jan 2026
- 2025ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer13 Oct 2025
- 2025Attributes Shape the Embedding Space of Face Recognition Models15 Jul 2025
- 2025CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesis23 May 2025
- 2025Flux Already Knows -- Activating Subject-Driven Image Generation without Training12 Apr 2025
- 2025Stochastic Trajectory Prediction under Unstructured Constraints18 Mar 2025
- 2025LucidAtlas: Learning Uncertainty-Aware, Covariate-Disentangled, Individualized Atlas Representations12 Feb 2025
- 2024Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data2 Dec 2024
- 2024Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey5 Nov 2024
- 2024The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition4 Sep 2024
12 newest of 80 All 80 (JSON)