# On Self Modulation for Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1810.01365 · HTML: https://mlsanity.com/paper/1810.01365/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2018.

## Built on by (most cited first, 30 of 34)

- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning](https://mlsanity.com/paper/2210.08823/) (2022)
- [Cross-Modal Contrastive Learning for Text-to-Image Generation](https://mlsanity.com/paper/2101.04702/) (2021)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [MAT: Mask-Aware Transformer for Large Hole Image Inpainting](https://mlsanity.com/paper/2203.15270/) (2022)
- [TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up](https://mlsanity.com/paper/2102.07074/) (2021)
- [Pretrained Transformers as Universal Computation Engines](https://mlsanity.com/paper/2103.05247/) (2021)
- [LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks](https://mlsanity.com/paper/2206.06565/) (2022)
- [ViTGAN: Training GANs with Vision Transformers](https://mlsanity.com/paper/2107.04589/) (2021)
- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [Improved Transformer for High-Resolution GANs](https://mlsanity.com/paper/2106.07631/) (2021)
- [Conditional Deformable Image Registration with Convolutional Neural Network](https://mlsanity.com/paper/2106.12673/) (2021)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models](https://mlsanity.com/paper/2403.12952/) (2024)
- [Dual Contrastive Loss and Attention for GANs](https://mlsanity.com/paper/2103.16748/) (2021)
- [Adaptive Denoising via GainTuning](https://mlsanity.com/paper/2107.12815/) (2021)
- [ME-GAN: Learning Panoptic Electrocardio Representations for Multi-view ECG Synthesis Conditioned on Heart Diseases](https://mlsanity.com/paper/2207.10670/) (2022)
- [CoPE: Conditional image generation using Polynomial Expansions](https://mlsanity.com/paper/2104.05077/) (2021)
- [Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generation](https://mlsanity.com/paper/2306.00964/) (2023)
- [Self-Supervised GANs with Label Augmentation](https://mlsanity.com/paper/2106.08601/) (2021)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [Location-aware Adaptive Normalization: A Deep Learning Approach For Wildfire Danger Forecasting](https://mlsanity.com/paper/2212.08208/) (2022)
- [Enhance Images as You Like with Unpaired Learning](https://mlsanity.com/paper/2110.01161/) (2021)
- [Coordinate-Aware Modulation for Neural Fields](https://mlsanity.com/paper/2311.14993/) (2023)
- [Unveiling the Latent Space Geometry of Push-Forward Generative Models](https://mlsanity.com/paper/2207.10541/) (2022)
- [Diverse Shape Completion via Style Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2311.11184/) (2023)
- [Efficient generative adversarial networks using linear additive-attention Transformers](https://mlsanity.com/paper/2401.09596/) (2024)
- [Self Sparse Generative Adversarial Networks](https://mlsanity.com/paper/2101.10556/) (2021)
- [End-to-end Training for Text-to-Image Synthesis using Dual-Text Embeddings](https://mlsanity.com/paper/2502.01507/) (2025)

## Abstract (arXiv)

Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings. Intuitively, self-modulation allows the intermediate feature maps of a generator to change as a function of the input noise vector. While reminiscent of other conditioning techniques, it requires no labeled data. In a large-scale empirical study we observe a relative decrease of 5%-35% in FID. Furthermore, all else being equal, adding this modification to the generator leads to improved performance in 124/144 (86%) of the studied settings. Self-modulation is a simple architectural change that requires no additional parameter tuning, which suggests that it can be applied readily to any GAN.