mlsanityData to 14 Oct 2026

Generalizing Dataset Distillation via Deep Generative Prior

arXiv 2305.01649 ↗First posted 2 May 2023cs.CV cs.AI cs.LGcited by 126

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Dataset Distillation aims to distill an entire dataset's knowledge into a few synthetic images. The idea is to synthesize a small number of synthetic data points that, when given to a learning algorithm as training data, result in a model approximating one trained on the original data. Despite recent progress in the field, existing dataset distillation methods fail to generalize to new architectures and scale to high-resolution datasets. To overcome the above issues, we propose to use the learned prior from pre-trained deep generative models to synthesize the distilled data. To achieve this, we present a new optimization algorithm that distills a large number of images into a few intermediate feature vectors in the generative model's latent space. Our method augments existing techniques, significantly improving cross-architecture generalization in all settings.

Key facts. Cited by 126 papers in the map (data through 14 Oct 2026). Builds on 64 of its 82 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

64 of 82 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  12. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
38 more
  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  5. 2017Deep Image Priorcited by 1,098
  6. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  7. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  10. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  11. 2020Dataset Condensation with Gradient Matchingcited by 516
  12. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  13. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  14. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  15. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  16. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  17. 2021Dataset Condensation with Distribution Matchingcited by 365
  18. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 363
  19. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  20. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  21. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 332
  22. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  23. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  24. 2018Dataset Distillationcited by 230
  25. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  26. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  27. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  28. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  29. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 192
  30. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 188
  31. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 181
  32. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  33. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  34. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 120
  35. 2022Dataset Distillation via Factorizationcited by 115
  36. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  37. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
  38. 2020Rewriting a Deep Generative Modelcited by 107

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 20; 2024: 34; 2025: 46; 2026: 26.

126 papers.

Sort:

By area:

  1. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 109
  2. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 83
  3. 2024D4M: Dataset Distillation via Disentangled Diffusion Modelcited by 77
  4. 2023CaT: Balanced Continual Graph Learning with Graph Condensationcited by 52
  5. 2023M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancycited by 49
  6. 2023Does Graph Distillation See Like Vision Dataset Counterpart?cited by 45
  7. 2023Sequential Subset Matching for Dataset Distillationcited by 40
  8. 2023Dataset Distillation via the Wasserstein Metriccited by 31
  9. 2024A Label is Worth a Thousand Images in Dataset Distillationcited by 31
  10. 2023Frequency Domain-based Dataset Distillationcited by 28
  11. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  12. 2023Dataset Distillation via Curriculum Data Synthesis in Large Data Eracited by 26
38 more
  1. 2024Dataset Distillation by Automatic Training Trajectoriescited by 26
  2. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
  3. 2024Unlocking Dataset Distillation with Diffusion Modelscited by 23
  4. 2024DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillationcited by 20
  5. 2024Multisize Dataset Condensationcited by 19
  6. 2025Dataset Distillation via Vision-Language Category Prototypecited by 19
  7. 2024Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarioscited by 18
  8. 2023Data Distillation Can Be Like Vodka: Distilling More Times For Better Qualitycited by 17
  9. 2024Low-Rank Similarity Mining for Multimodal Dataset Distillationcited by 17
  10. 2023Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latentscited by 16
  11. 2024DANCE: Dual-View Distribution Alignment for Dataset Condensationcited by 15
  12. 2024Neural Spectral Decomposition for Dataset Distillationcited by 14
  13. 2025CaO2: Rectifying Inconsistencies in Diffusion-Based Dataset Distillationcited by 14
  14. 2024Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillationcited by 13
  15. 2024Generative Dataset Distillation Based on Diffusion Modelcited by 13
  16. 2023Can pre-trained models assist in dataset distillation?cited by 12
  17. 2024Group Distributionally Robust Dataset Distillation with Risk Minimizationcited by 10
  18. 2025DD-Ranking: Rethinking the Evaluation of Dataset Distillationcited by 10
  19. 2025Dataset distillation for memorized data: Soft labels can leak held-out teacher knowledgecited by 10
  20. 2025Information-Guided Diffusion Sampling for Dataset Distillationcited by 10
  21. 2023Dataset Distillation via Adversarial Prediction Matchingcited by 9
  22. 2023PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensationcited by 9
  23. 2024GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Costcited by 9
  24. 2024Adaptive Dataset Quantizationcited by 9
  25. 2023Dataset Distillation in Latent Spacecited by 8
  26. 2025Task-Specific Generative Dataset Distillation with Difficulty-Guided Samplingcited by 8
  27. 2024Distributional Dataset Distillation with Subtask Decompositioncited by 7
  28. 2025MGD3: Mode-Guided Dataset Distillation using Diffusion Modelscited by 7
  29. 2023Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensationcited by 6
  30. 2024One Category One Prompt: Dataset Distillation using Diffusion Modelscited by 6
  31. 2024A Study in Dataset Pruning for Image Super-Resolutioncited by 6
  32. 2024Image Distillation for Safe Data Sharing in Histopathologycited by 6
  33. 2024Color-Oriented Redundancy Reduction in Dataset Distillationcited by 6
  34. 2024A Large-Scale Study on Video Action Dataset Condensationcited by 6
  35. 2026When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generatorscited by 6
  36. 2023Embarassingly Simple Dataset Distillationcited by 5
  37. 2024Contributing Dimension Structure of Deep Feature for Coreset Selectioncited by 5
  38. 2024Label-Augmented Dataset Distillationcited by 5

50 most cited of 126 All 126 (JSON)