mlsanityData to 7 Oct 2026

Synthesizing Informative Training Samples with GAN

arXiv 2204.07513 ↗First posted 15 Apr 2022cs.LG cs.CVcited by 77

Not on the map. Why?

Abstract (arXiv)

Remarkable progress has been achieved in synthesizing photo-realistic images with generative adversarial networks (GANs). Recently, GANs are utilized as the training sample generator when obtaining or storing real training data is expensive even infeasible. However, traditional GANs generated images are not as informative as the real training samples when being used to train deep neural networks. In this paper, we propose a novel method to synthesize Informative Training samples with GAN (IT-GAN). Specifically, we freeze a pre-trained GAN model and learn the informative latent vectors that correspond to informative training samples. The synthesized images are required to preserve information for training deep neural networks rather than visual reality or fidelity. Experiments verify that the deep neural networks can learn faster and achieve better performance when being trained with our IT-GAN generated images. We also show that our method is a promising solution to dataset condensation problem.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Builds on 54 of its 66 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

54 of 66 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  8. 2016Wide Residual Networkscited by 3,057
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2017Improved Training of Wasserstein GANscited by 2,223
38 more
  1. 2014Conditional Generative Adversarial Netscited by 2,152
  2. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  3. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  5. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  6. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  8. 2017Continual Learning with Deep Generative Replaycited by 1,152
  9. 2018Self-Attention Generative Adversarial Networkscited by 809
  10. 2016Generative Adversarial Text to Image Synthesiscited by 735
  11. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  12. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  13. 2020Dataset Condensation with Gradient Matchingcited by 510
  14. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  15. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  16. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  17. 2021GAN Inversion: A Surveycited by 366
  18. 2021Dataset Condensation with Distribution Matchingcited by 362
  19. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  20. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  21. 2017Data Augmentation Generative Adversarial Networkscited by 278
  22. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  23. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
  24. 2018Differentially Private Generative Adversarial Networkcited by 235
  25. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  26. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  27. 2018Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networkscited by 146
  28. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  29. 2017Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networkscited by 107
  30. 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
  31. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 106
  32. 2019MineGAN: effective knowledge transfer from GANs to target domains with few imagescited by 106
  33. 2021SynFace: Face Recognition with Synthetic Datacited by 106
  34. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 103
  35. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  36. 2020Flexible Dataset Distillation: Learn Labels Instead of Imagescited by 91
  37. 2019Data-Free Adversarial Distillationcited by 90
  38. 2020GAN Memory with No Forgettingcited by 84

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 28; 2024: 19; 2025: 18; 2026: 10.

77 papers.

Sort:

By area:

  1. 2023Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspectivecited by 136
  2. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  3. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 125
  4. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  5. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  6. 2023InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruningcited by 90
  7. 2023Data Distillation: A Surveycited by 86
  8. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 81
  9. 2023Large-scale Dataset Pruning with Dynamic Uncertaintycited by 63
  10. 2022Expanding Small-Scale Datasets with Guided Imaginationcited by 61
  11. 2023Training on Thin Air: Improve Image Classification with Generated Datacited by 50
  12. 2023DiM: Distilling Dataset into Generative Modelcited by 45
38 more
  1. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 35
  2. 2023A Survey on Dataset Distillation: Approaches, Applications and Future Directionscited by 32
  3. 2023Dataset Distillation via the Wasserstein Metriccited by 31
  4. 2024Exploiting Inter-sample and Inter-feature Relations in Dataset Distillationcited by 31
  5. 2024Dataset Distillation by Automatic Training Trajectoriescited by 26
  6. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 24
  7. 2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
  8. 2023You Only Condense Once: Two Rules for Pruning Condensed Datasetscited by 20
  9. 2024Distribution-Aware Data Expansion with Diffusion Modelscited by 20
  10. 2024DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillationcited by 20
  11. 2024Multisize Dataset Condensationcited by 19
  12. 2024Self-Guided Generation of Minority Samples Using Diffusion Modelscited by 19
  13. 2024Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarioscited by 18
  14. 2023Data Distillation Can Be Like Vodka: Distilling More Times For Better Qualitycited by 17
  15. 2025Dataset Distillation via Vision-Language Category Prototypecited by 17
  16. 2024Low-Rank Similarity Mining for Multimodal Dataset Distillationcited by 16
  17. 2023Dataset Condensation via Generative Modelcited by 15
  18. 2024Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillationcited by 13
  19. 2024Generative Dataset Distillation Based on Diffusion Modelcited by 13
  20. 2025Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memorycited by 10
  21. 2025Information-Guided Diffusion Sampling for Dataset Distillationcited by 10
  22. 2023A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairnesscited by 9
  23. 2024GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Costcited by 9
  24. 2025DD-Ranking: Rethinking the Evaluation of Dataset Distillationcited by 8
  25. 2025Task-Specific Generative Dataset Distillation with Difficulty-Guided Samplingcited by 8
  26. 2023Instance-Conditioned GAN Data Augmentation for Representation Learningcited by 7
  27. 2024Improve Cross-Architecture Generalization on Dataset Distillationcited by 7
  28. 2024Distributional Dataset Distillation with Subtask Decompositioncited by 7
  29. 2022FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoderscited by 6
  30. 2023Free-ATM: Exploring Unsupervised Learning on Diffusion-Generated Images with Free Attention Maskscited by 6
  31. 2023Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensationcited by 6
  32. 2023DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matchingcited by 6
  33. 2023Data Optimization in Deep Learning: A Surveycited by 6
  34. 2024A Study in Dataset Pruning for Image Super-Resolutioncited by 6
  35. 2024Color-Oriented Redundancy Reduction in Dataset Distillationcited by 6
  36. 2025Unifying Dataset Pruning and Distillation for Efficient Large-scale Compressioncited by 6
  37. 2025MGD3: Mode-Guided Dataset Distillation using Diffusion Modelscited by 6
  38. 2025ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detectioncited by 6

50 most cited of 77 All 77 (JSON)