mlsanityData to 7 Oct 2026

Ensembling with Deep Generative Views

arXiv 2104.14551 ↗First posted 29 Apr 2021cs.CV cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Recent generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose, simply by learning from unlabeled image collections. Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification. Using a pretrained generator, we first find the latent code corresponding to a given real input image. Applying perturbations to the code creates natural variations of the image, which can then be ensembled together at test-time. We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars. Critically, we find that several design decisions are required towards making this process work; the perturbation procedure, weighting between the augmentations and original image, and training the classifier on synthesized images can all impact the result. Currently, we find that while test-time ensembling with GAN-based augmentations can offer some small improvements, the remaining bottlenecks are the efficiency and accuracy of the GAN reconstructions, coupled with classifier sensitivities to artifacts in GAN-generated images.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 67 of its 77 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

67 of 77 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Mask R-CNNcited by 7,259
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  12. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
38 more
  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  2. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  3. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  4. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  5. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  6. 2018Self-Attention Generative Adversarial Networkscited by 809
  7. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  8. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  9. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  10. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  11. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  12. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  13. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  14. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  15. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  16. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  17. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  18. 2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293
  19. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
  20. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  21. 2020What makes fake images detectable? Understanding properties that generalizecited by 272
  22. 2019On the "steerability" of generative adversarial networkscited by 247
  23. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  24. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  25. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  26. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  27. 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
  28. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  29. 2019Seeing What a GAN Cannot Generatecited by 148
  30. 2018Synthetic Data Augmentation using GAN for Improved Liver Lesion Classificationcited by 136
  31. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  32. 2021Counterfactual Generative Networkscited by 111
  33. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  34. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  35. 2020Controlling generative models with continuous factors of variationscited by 93
  36. 2017Learning to Compose Domain-Specific Transformations for Data Augmentationcited by 91
  37. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  38. 2020StyleGAN2 Distillation for Feed-forward Image Manipulationcited by 78

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 20; 2023: 17; 2024: 7; 2025: 4.

53 papers.

Sort:

By area:

  1. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  2. 2022Diffusion Models for Adversarial Purificationcited by 469
  3. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  4. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 143
  5. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 125
  6. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  7. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  8. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  9. 2023Training on Thin Air: Improve Image Classification with Generated Datacited by 50
  10. 2022A Closer Look at Few-shot Image Generationcited by 49
  11. 2022Spatially-Adaptive Multilayer Selection for GAN Inversion and Editingcited by 39
  12. 2022TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editingcited by 36
38 more
  1. 2023Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encodingcited by 35
  2. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  3. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  4. 2022Few-shot Image Generation via Adaptation-Aware Kernel Modulationcited by 33
  5. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  6. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  7. 2022Learning to Generalize across Domains on Single Test Samplescited by 24
  8. 2022Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustnesscited by 23
  9. 2022Improving GANs with A Dynamic Discriminatorcited by 20
  10. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  11. 2023Exploring Incompatible Knowledge Transfer in Few-shot Image Generationcited by 19
  12. 2023CHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Imagescited by 19
  13. 2023StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generationcited by 19
  14. 2021LARGE: Latent-Based Regression through GAN Semanticscited by 17
  15. 2022Intra-Source Style Augmentation for Improved Domain Generalizationcited by 17
  16. 2022Feature-Style Encoder for Style-Based GAN Inversioncited by 15
  17. 2022Near Perfect GAN Inversioncited by 15
  18. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  19. 2021Semantically Distributed Robust Optimization for Vision-and-Language Inferencecited by 9
  20. 2023Make It So: Steering StyleGAN for Any Image Inversion and Editingcited by 9
  21. 2023Controllable 3D Face Generation with Conditional Style Code Diffusioncited by 9
  22. 2022Compressed Vision for Efficient Video Understandingcited by 8
  23. 2024Controlled Training Data Generation with Diffusion Modelscited by 8
  24. 2022Deep Generative Views to Mitigate Gender Classification Bias Across Gender-Race Groupscited by 7
  25. 2023Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalizationcited by 7
  26. 2024LoRID: Low-Rank Iterative Diffusion for Adversarial Purificationcited by 7
  27. 2023Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensationcited by 6
  28. 2024A Data Augmentation Pipeline to Generate Synthetic Labeled Datasets of 3D Echocardiography Images using a GANcited by 6
  29. 2024Diffusion Models as Data Mining Toolscited by 5
  30. 2024Gradient-free Decoder Inversion in Latent Diffusion Modelscited by 5
  31. 2023JoIN: Joint GANs Inversion for Intrinsic Image Decompositioncited by 4
  32. 2023Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolationcited by 4
  33. 2024Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Languagecited by 4
  34. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Modelscited by 4
  35. 2025The Role of Video Generation in Enhancing Data-Limited Action Understandingcited by 2
  36. 2025TAIGen: Training-Free Adversarial Image Generation via Diffusion Modelscited by 2
  37. 2022A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restorationcited by 1
  38. 2023AdAM: Few-Shot Image Generation via Adaptation-Aware Kernel Modulationcited by 1

50 most cited of 53 All 53 (JSON)