# Improved Consistency Regularization for GANs

arXiv: https://arxiv.org/abs/2002.04724 · HTML: https://mlsanity.com/paper/2002.04724/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 30 of 69)

- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [StyleSwin: Transformer-based GAN for High-resolution Image Generation](https://mlsanity.com/paper/2112.10762/) (2021)
- [ViTGAN: Training GANs with Vision Transformers](https://mlsanity.com/paper/2107.04589/) (2021)
- [Regularizing Generative Adversarial Networks under Limited Data](https://mlsanity.com/paper/2104.03310/) (2021)
- [Improved Transformer for High-Resolution GANs](https://mlsanity.com/paper/2106.07631/) (2021)
- [Data-Efficient Instance Generation from Instance Discrimination](https://mlsanity.com/paper/2106.04566/) (2021)
- [Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images](https://mlsanity.com/paper/2106.07873/) (2021)
- [Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training](https://mlsanity.com/paper/2111.01118/) (2021)
- [Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds](https://mlsanity.com/paper/2301.12485/) (2023)
- [Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data](https://mlsanity.com/paper/2111.06849/) (2021)
- [Training GANs with Stronger Augmentations via Contrastive Discriminator](https://mlsanity.com/paper/2103.09742/) (2021)
- [One-Shot Generative Domain Adaptation](https://mlsanity.com/paper/2111.09876/) (2021)
- [StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis](https://mlsanity.com/paper/2206.09479/) (2022)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [Dual Contrastive Loss and Attention for GANs](https://mlsanity.com/paper/2103.16748/) (2021)
- [Zero-shot Generative Model Adaptation via Image-specific Prompt Learning](https://mlsanity.com/paper/2304.03119/) (2023)
- [Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective](https://mlsanity.com/paper/2103.00397/) (2021)
- [Conditional GANs with Auxiliary Discriminative Classifier](https://mlsanity.com/paper/2107.10060/) (2021)
- [Improving GANs with A Dynamic Discriminator](https://mlsanity.com/paper/2209.09897/) (2022)
- [A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot](https://mlsanity.com/paper/2307.14397/) (2023)
- [Domain Re-Modulation for Few-Shot Generative Domain Adaptation](https://mlsanity.com/paper/2302.02550/) (2023)
- [Efficient Image Generation with Variadic Attention Heads](https://mlsanity.com/paper/2211.05770/) (2022)
- [Manifold Learning Benefits GANs](https://mlsanity.com/paper/2112.12618/) (2021)
- [Improving GANs for Long-Tailed Data through Group Spectral Regularization](https://mlsanity.com/paper/2208.09932/) (2022)
- [MOGAN: Morphologic-structure-aware Generative Learning from a Single Image](https://mlsanity.com/paper/2103.02997/) (2021)
- [Self-Supervised GANs with Label Augmentation](https://mlsanity.com/paper/2106.08601/) (2021)
- [Feature Statistics Mixing Regularization for Generative Adversarial Networks](https://mlsanity.com/paper/2112.04120/) (2021)
- [A Regularized Conditional GAN for Posterior Sampling in Image Recovery Problems](https://mlsanity.com/paper/2210.13389/) (2022)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey](https://mlsanity.com/paper/2411.05027/) (2024)

## Abstract (arXiv)

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then propose several modifications to the consistency regularization procedure designed to improve its performance. We carry out extensive experiments quantifying the benefit of our improvements. For unconditional image synthesis on CIFAR-10 and CelebA, our modifications yield the best known FID scores on various GAN architectures. For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21. Finally, on ImageNet-2012, we apply our technique to the original BigGAN model and improve the FID from 6.66 to 5.38, which is the best score at that model size.