mlsanityData to 14 Oct 2026

Consistency Regularization for Generative Adversarial Networks

arXiv 1910.12027 ↗First posted 26 Oct 2019cs.LG cs.CV stat.MLcited by 136

Not on the map. Why?

Abstract (arXiv)

Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization. In this work, we propose a simple, effective training stabilizer based on the notion of consistency regularization---a popular technique in the semi-supervised learning literature. In particular, we augment data passing into the GAN discriminator and penalize the sensitivity of the discriminator to these augmentations. We conduct a series of experiments to demonstrate that consistency regularization works effectively with spectral normalization and various GAN architectures, loss functions and optimizer settings. Our method achieves the best FID scores for unconditional image generation compared to other regularization methods on CIFAR-10 and CelebA. Moreover, Our consistency regularized GAN (CR-GAN) improves state-of-the-art FID scores for conditional generation from 14.73 to 11.48 on CIFAR-10 and from 8.73 to 6.66 on ImageNet-2012.

Key facts. Cited by 136 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 52; 2022: 25; 2023: 23; 2024: 15; 2025: 8; 2026: 12.

136 papers.

Sort:

By area:

  1. 2023Consistency Modelscited by 1,822
  2. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  3. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  4. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  5. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  6. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  7. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 107
  8. 2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
  9. 2021Ensembling Off-the-shelf Models for GAN Trainingcited by 86
  10. 2021Is Attention Better Than Matrix Decomposition?cited by 77
  11. 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
  12. 2022The ArtBench Dataset: Benchmarking Generative Models with Artworkscited by 64
38 more
  1. 2021Improved Transformer for High-Resolution GANscited by 59
  2. 2021Data-Efficient Instance Generation from Instance Discriminationcited by 58
  3. 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
  4. 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
  5. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 56
  6. 2022Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignmentcited by 54
  7. 2021Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Datacited by 53
  8. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  9. 2021Consistency Regularization for Variational Auto-Encoderscited by 51
  10. 2022A Closer Look at Few-shot Image Generationcited by 49
  11. 2021Training GANs with Stronger Augmentations via Contrastive Discriminatorcited by 45
  12. 2021Consistency Regularization for Adversarial Robustnesscited by 42
  13. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  14. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  15. 2021Dual Contrastive Loss and Attention for GANscited by 30
  16. 2022vCLIMB: A Novel Video Class Incremental Learning Benchmarkcited by 28
  17. 2023Zero-shot Generative Model Adaptation via Image-specific Prompt Learningcited by 27
  18. 2021Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspectivecited by 25
  19. 2021Gradient Normalization for Generative Adversarial Networkscited by 25
  20. 2021Conditional GANs with Auxiliary Discriminative Classifiercited by 20
  21. 2022Improving GANs with A Dynamic Discriminatorcited by 20
  22. 2023A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shotcited by 19
  23. 2021Quaternion Generative Adversarial Networkscited by 17
  24. 2021Class Balancing GAN with a Classifier in the Loopcited by 17
  25. 2021GenCo: Generative Co-training for Generative Adversarial Networks with Limited Datacited by 14
  26. 2021Manifold Learning Benefits GANscited by 14
  27. 2022Improving GANs for Long-Tailed Data through Group Spectral Regularizationcited by 14
  28. 2023SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layercited by 14
  29. 2024PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularizationcited by 14
  30. 2023Phasic Content Fusing Diffusion Model with Directional Distribution Consistency for Few-Shot Model Adaptioncited by 11
  31. 2021ReMix: Towards Image-to-Image Translation with Limited Datacited by 9
  32. 2021Partition-Guided GANscited by 9
  33. 2021Self-Supervised GANs with Label Augmentationcited by 9
  34. 2021Feature Statistics Mixing Regularization for Generative Adversarial Networkscited by 9
  35. 2022Adaptive Domain Generalization via Online Disagreement Minimizationcited by 9
  36. 2022Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Datacited by 9
  37. 2023Image Synthesis under Limited Data: A Survey and Taxonomycited by 9
  38. 2024Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Surveycited by 9

50 most cited of 136 All 136 (JSON)