mlsanityData to 14 Oct 2026

Ensembling Off-the-shelf Models for GAN Training

arXiv 2112.09130 ↗First posted 16 Dec 2021cs.CV cs.GR cs.LGcited by 86

Not on the map. Why?

Cited by 27 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The advent of large-scale training has produced a cornucopia of powerful visual recognition models. However, generative models, such as GANs, have traditionally been trained from scratch in an unsupervised manner. Can the collective "knowledge" from a large bank of pretrained vision models be leveraged to improve GAN training? If so, with so many models to choose from, which one(s) should be selected, and in what manner are they most effective? We find that pretrained computer vision models can significantly improve performance when used in an ensemble of discriminators. Notably, the particular subset of selected models greatly affects performance. We propose an effective selection mechanism, by probing the linear separability between real and fake samples in pretrained model embeddings, choosing the most accurate model, and progressively adding it to the discriminator ensemble. Interestingly, our method can improve GAN training in both limited data and large-scale settings. Given only 10k training samples, our FID on LSUN Cat matches the StyleGAN2 trained on 1.6M images. On the full dataset, our method improves FID by 1.5x to 2x on cat, church, and horse categories of LSUN.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Builds on 95 of its 113 references that are in the map. First posted on arXiv 16 Dec 2021.

Builds on

95 of 113 references.

References · 95

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  8. 2016Improved Techniques for Training GANscited by 2,970
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  13. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  14. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  15. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  16. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  17. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  18. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  19. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  20. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  21. 2018Demystifying MMD GANscited by 1,293
  22. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  23. 2014How transferable are features in deep neural networks?cited by 1,225
  24. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  25. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  26. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  27. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  28. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  29. 2016Least Squares Generative Adversarial Networkscited by 1,001
  30. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  31. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  32. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  33. 2019A Closer Look at Few-shot Classificationcited by 684
  34. 2018Which Training Methods for GANs do actually Converge?cited by 679
  35. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  36. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  37. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  38. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577

50 most cited of 95 All 95 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 18; 2023: 19; 2024: 19; 2025: 18; 2026: 12.

86 papers.

Sort:

By area:

  1. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  2. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  3. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  4. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 213
  5. 2024One-Step Image Translation with Text-to-Image Modelscited by 116
  6. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  7. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 92
  8. 2024Degradation-Guided One-Step Image Super-Resolution with Diffusion Priorscited by 69
  9. 20233D generation on ImageNetcited by 47
  10. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  11. 2025Harnessing Diffusion-Yielded Score Priors for Image Restorationcited by 36
  12. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
38 more
  1. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  2. 2025StableCodec: Taming One-Step Diffusion for Extreme Image Compressioncited by 33
  3. 2022Fair Generative Models via Transfer Learningcited by 27
  4. 2022Rewriting Geometric Rules of a GANcited by 25
  5. 2022A Comprehensive Survey on Data-Efficient GANs in Image Generationcited by 20
  6. 2022Improving GANs with A Dynamic Discriminatorcited by 20
  7. 2023A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shotcited by 19
  8. 2022FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANscited by 17
  9. 2022FreGAN: Exploiting Frequency Components for Training GANs under Limited Datacited by 16
  10. 2023MagicFusion: Boosting Text-to-Image Generation Performance by Fusing Diffusion Modelscited by 16
  11. 2023KD-DLGAN: Data Limited Image Generation via Knowledge Distillationcited by 16
  12. 2025Guiding a Diffusion Transformer with the Internal Dynamics of Itselfcited by 16
  13. 2025OMGSR: You Only Need One Mid-timestep Guidance for Real-World Image Super-Resolutioncited by 15
  14. 2023Domain Expansion of Image Generatorscited by 13
  15. 2024Ensembling Diffusion Models via Adaptive Feature Aggregationcited by 12
  16. 2026Representation Distribution Matching for One-Step Visual Generationcited by 11
  17. 2024VIMs: Virtual Immunohistochemistry Multiplex staining via Text-to-Stain Diffusion Trained on Uniplex Stainscited by 10
  18. 2023Image Synthesis under Limited Data: A Survey and Taxonomycited by 9
  19. 2022Hierarchical Semantic Regularization of Latent Spaces in StyleGANscited by 8
  20. 2023BlobGAN-3D: A Spatially-Disentangled 3D-Aware Generative Model for Indoor Scenescited by 8
  21. 2023Controllable Visual-Tactile Synthesiscited by 8
  22. 2024Taming the Tail in Class-Conditional GANs: Knowledge Sharing via Unconditional Training at Lower Resolutionscited by 8
  23. 2025When Schr"odinger Bridge Meets Real-World Image Dehazing with Unpaired Trainingcited by 8
  24. 2022Augmentation-Aware Self-Supervision for Data-Efficient GAN Trainingcited by 7
  25. 2022Generator Knows What Discriminator Should Learn in Unconditional GANscited by 7
  26. 2022Content-Based Search for Deep Generative Modelscited by 7
  27. 2022DigGAN: Discriminator gradIent Gap Regularization for GAN Training with Limited Datacited by 7
  28. 2023Revisiting the Evaluation of Image Synthesis with GANscited by 7
  29. 2025Ultra-Low Bitrate Perceptual Image Compression with Shallow Encodercited by 7
  30. 2023Unlocking Pre-trained Image Backbones for Semantic Image Synthesiscited by 6
  31. 2025Efficient Generative Model Training via Embedded Representation Warmupcited by 6
  32. 2026One-Shot Refiner: Boosting Feed-forward Novel View Synthesis via One-Step Diffusioncited by 6
  33. 2025Scalable GANs with Transformerscited by 5
  34. 2022GLeaD: Improving GANs with A Generator-Leading Taskcited by 4
  35. 2023Adversarially-Guided Portrait Mattingcited by 4
  36. 2023On the Biometric Capacity of Generative Face Modelscited by 4
  37. 2023How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Surveycited by 4
  38. 2024SAM-I2I: Unleash the Power of Segment Anything Model for Medical Image Translationcited by 4

50 most cited of 86 All 86 (JSON)