# Linear Semantics in Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2104.00487 · HTML: https://mlsanity.com/paper/2104.00487/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 45 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks](https://mlsanity.com/paper/1506.05751/) (2015)

## Built on by (most cited first, 25 of 25)

- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [EditGAN: High-Precision Semantic Image Editing](https://mlsanity.com/paper/2111.03186/) (2021)
- [BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations](https://mlsanity.com/paper/2201.04684/) (2022)
- [EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models](https://mlsanity.com/paper/2401.11739/) (2024)
- [Exploiting Diffusion Prior for Generalizable Dense Prediction](https://mlsanity.com/paper/2311.18832/) (2023)
- [State-of-the-Art in the Architecture, Methods and Applications of StyleGAN](https://mlsanity.com/paper/2202.14020/) (2022)
- [CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs](https://mlsanity.com/paper/2203.16521/) (2022)
- [Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention](https://mlsanity.com/paper/2312.03556/) (2023)
- [Generator Knows What Discriminator Should Learn in Unconditional GANs](https://mlsanity.com/paper/2207.13320/) (2022)
- [Line of Sight: On Linear Representations in VLLMs](https://mlsanity.com/paper/2506.04706/) (2025)
- [Optimizing Latent Space Directions For GAN-based Local Image Editing](https://mlsanity.com/paper/2111.12583/) (2021)
- [Extracting Semantic Knowledge from GANs with Unsupervised Learning](https://mlsanity.com/paper/2211.16710/) (2022)
- [Householder Projector for Unsupervised Latent Semantics Discovery](https://mlsanity.com/paper/2307.08012/) (2023)
- [Towards Full-to-Empty Room Generation with Structure-Aware Feature Encoding and Soft Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/2112.05396/) (2021)
- [One-Shot Synthesis of Images and Segmentation Masks](https://mlsanity.com/paper/2209.07547/) (2022)
- [SeaLion: Semantic Part-Aware Latent Point Diffusion Models for 3D Generation](https://mlsanity.com/paper/2505.17721/) (2025)
- [Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps](https://mlsanity.com/paper/2206.02903/) (2022)
- [Semantic Image Synthesis with Unconditional Generator](https://mlsanity.com/paper/2402.14395/) (2024)
- [InvSeg: Test-Time Prompt Inversion for Semantic Segmentation](https://mlsanity.com/paper/2410.11473/) (2024)
- [Interpreting Class Conditional GANs with Channel Awareness](https://mlsanity.com/paper/2203.11173/) (2022)
- [Enhancing Semantic Communication with Deep Generative Models -- An ICASSP Special Session Overview](https://mlsanity.com/paper/2309.02478/) (2023)
- [Latent Space Imaging](https://mlsanity.com/paper/2407.07052/) (2024)
- [Label-Efficient 3D Brain Segmentation via Complementary 2D Diffusion Models with Orthogonal Views](https://mlsanity.com/paper/2407.12329/) (2024)
- [Rethinking the editing of generative adversarial networks: a method to estimate editing vectors based on dimension reduction](https://mlsanity.com/paper/2305.09454/) (2023)
- [A Training-Free Approach for Multi-ID Customization via Attention Adjustment and Spatial Control](https://mlsanity.com/paper/2511.20401/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are able to generate high-quality images, but it remains difficult to explicitly specify the semantics of synthesized images. In this work, we aim to better understand the semantic representation of GANs, and thereby enable semantic control in GAN's generation process. Interestingly, we find that a well-trained GAN encodes image semantics in its internal feature maps in a surprisingly simple way: a linear transformation of feature maps suffices to extract the generated image semantics. To verify this simplicity, we conduct extensive experiments on various GANs and datasets; and thanks to this simplicity, we are able to learn a semantic segmentation model for a trained GAN from a small number (e.g., 8) of labeled images. Last but not least, leveraging our findings, we propose two few-shot image editing approaches, namely Semantic-Conditional Sampling and Semantic Image Editing. Given a trained GAN and as few as eight semantic annotations, the user is able to generate diverse images subject to a user-provided semantic layout, and control the synthesized image semantics. We have made the code publicly available.