mlsanityData to 7 Oct 2026

NoisyTwins: Class-Consistent and Diverse Image Generation through StyleGANs

arXiv 2304.05866 ↗First posted 12 Apr 2023cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

StyleGANs are at the forefront of controllable image generation as they produce a latent space that is semantically disentangled, making it suitable for image editing and manipulation. However, the performance of StyleGANs severely degrades when trained via class-conditioning on large-scale long-tailed datasets. We find that one reason for degradation is the collapse of latents for each class in the W latent space. With NoisyTwins, we first introduce an effective and inexpensive augmentation strategy for class embeddings, which then decorrelates the latents based on self-supervision in the W space. This decorrelation mitigates collapse, ensuring that our method preserves intra-class diversity with class-consistency in image generation. We show the effectiveness of our approach on large-scale real-world long-tailed datasets of ImageNet-LT and iNaturalist 2019, where our method outperforms other methods by ~ 19% on FID, establishing a new state-of-the-art.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 46 of its 54 references that are in the map. First posted on arXiv 12 Apr 2023.

Builds on

46 of 54 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
34 more
  1. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  2. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  5. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  6. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  7. 2018Which Training Methods for GANs do actually Converge?cited by 672
  8. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  9. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  10. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  11. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 393
  12. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  13. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 331
  14. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  15. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  16. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  17. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  18. 2021Projected GANs Converge Fastercited by 204
  19. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  20. 2021Instance-Conditioned GANcited by 119
  21. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  22. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  23. 2021Data-Efficient Instance Generation from Instance Discriminationcited by 58
  24. 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
  25. 2021Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Datacited by 53
  26. 2021Training GANs with Stronger Augmentations via Contrastive Discriminatorcited by 45
  27. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  28. 2022Finding Directions in GAN's Latent Space for Neural Face Reenactmentcited by 27
  29. 2022Collapse by Conditioning: Training Class-conditional GANs with Limited Datacited by 24
  30. 2022Everything is There in Latent Space: Attribute Editing and Attribute Style Manipulation by StyleGAN Latent Space Explorationcited by 21
  31. 2022Improving GANs with A Dynamic Discriminatorcited by 20
  32. 2021Class Balancing GAN with a Classifier in the Loopcited by 17
  33. 2022Improving GANs for Long-Tailed Data through Group Spectral Regularizationcited by 14
  34. 20222D GANs Meet Unsupervised Single-view 3D Reconstructioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 3; 2026: 1.

12 papers.

  1. 2024EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Modelscited by 35
  2. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  3. 2023Learning Robust Deep Visual Representations from EEG Brain Recordingscited by 26
  4. 2024DeiT-LT Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasetscited by 14
  5. 2024Taming the Tail in Class-Conditional GANs: Knowledge Sharing via Unconditional Training at Lower Resolutionscited by 8
  6. 2025Phi-GAN: Physics-Inspired GAN for Generating SAR Images Under Limited Datacited by 4
  7. 2024DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Modelscited by 3
  8. 2025Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Modelcited by 1
  9. 2023Fair GANs through model rebalancing for extremely imbalanced class distributionscited by 0
  10. 2024Learning from Limited and Imperfect Datacited by 0
  11. 2025Learning from Limited and Imperfect Datacited by 0
  12. 2026HP-GAN: Harnessing pretrained networks for GAN improvement with FakeTwins and discriminator consistencycited by 0