mlsanityData to 14 Oct 2026

Dataset Condensation via Efficient Synthetic-Data Parameterization

arXiv 2205.14959 ↗First posted 30 May 2022cs.LGcited by 188

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The great success of machine learning with massive amounts of data comes at a price of huge computation costs and storage for training and tuning. Recent studies on dataset condensation attempt to reduce the dependence on such massive data by synthesizing a compact training dataset. However, the existing approaches have fundamental limitations in optimization due to the limited representability of synthetic datasets without considering any data regularity characteristics. To this end, we propose a novel condensation framework that generates multiple synthetic data with a limited storage budget via efficient parameterization considering data regularity. We further analyze the shortcomings of the existing gradient matching-based condensation methods and develop an effective optimization technique for improving the condensation of training data information. We propose a unified algorithm that drastically improves the quality of condensed data against the current state-of-the-art on CIFAR-10, ImageNet, and Speech Commands.

Key facts. Cited by 188 papers in the map (data through 14 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

30 of 41 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  7. 2016Learning without Forgettingcited by 2,496
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  9. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  12. 2019Contrastive Multiview Codingcited by 1,301
18 more
  1. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  2. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  3. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  4. 2020Dataset Condensation with Gradient Matchingcited by 516
  5. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  6. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 394
  7. 2021Dataset Condensation with Distribution Matchingcited by 365
  8. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
  9. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  10. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  11. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 261
  12. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  13. 2021Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversitycited by 128
  14. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
  15. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  16. 2018On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Lengthcited by 88
  17. 2016Coresets and Sketchescited by 63
  18. 2017Towards End-to-End Speech Recognition with Deep Convolutional Neural Networkscited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 43; 2024: 61; 2025: 45; 2026: 34.

188 papers.

Sort:

By area:

  1. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 181
  2. 2023Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspectivecited by 138
  3. 2023Dataset Distillation: A Comprehensive Reviewcited by 127
  4. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 126
  5. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  6. 2023Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matchingcited by 118
  7. 2022Dataset Distillation via Factorizationcited by 115
  8. 2023A Comprehensive Survey of Dataset Distillationcited by 115
  9. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 109
  10. 2022Condensing Graphs via One-Step Gradient Matchingcited by 100
  11. 2023DREAM: Efficient Dataset Distillation by Representative Matchingcited by 94
  12. 2023Data Distillation: A Surveycited by 87
38 more
  1. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 83
  2. 2023Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matchingcited by 69
  3. 2023Data Pruning via Moving-one-Sample-outcited by 67
  4. 2022Accelerating Dataset Distillation via Model Augmentationcited by 62
  5. 2023M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancycited by 49
  6. 2023Dataset Quantizationcited by 48
  7. 2023DiM: Distilling Dataset into Generative Modelcited by 46
  8. 2023Does Graph Distillation See Like Vision Dataset Counterpart?cited by 45
  9. 2025Dataset Distillation with Neural Characteristic Function: A Minmax Perspectivecited by 45
  10. 2023Sequential Subset Matching for Dataset Distillationcited by 40
  11. 2024Elucidating the Design Space of Dataset Condensationcited by 40
  12. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  13. 2022Dataset Condensation with Latent Space Knowledge Factorization and Sharingcited by 34
  14. 2023Loss-Curvature Matching for Dataset Selection and Condensationcited by 33
  15. 2024Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustmentcited by 33
  16. 2023A Survey on Dataset Distillation: Approaches, Applications and Future Directionscited by 32
  17. 2024A Label is Worth a Thousand Images in Dataset Distillationcited by 31
  18. 2023Frequency Domain-based Dataset Distillationcited by 28
  19. 2023Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learningcited by 26
  20. 2023Dataset Distillation via Curriculum Data Synthesis in Large Data Eracited by 26
  21. 2024Dataset Distillation by Automatic Training Trajectoriescited by 26
  22. 2023GPT-FL: Generative Pre-trained Model-Assisted Federated Learningcited by 24
  23. 2024Unlocking Dataset Distillation with Diffusion Modelscited by 23
  24. 2023Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglementcited by 21
  25. 2023Neural Relation Graph: A Unified Framework for Identifying Label Noise and Outlier Datacited by 20
  26. 2023You Only Condense Once: Two Rules for Pruning Condensed Datasetscited by 20
  27. 2024DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillationcited by 20
  28. 2023Summarizing Stream Data for Memory-Constrained Online Continual Learningcited by 19
  29. 2024Multisize Dataset Condensationcited by 19
  30. 2024Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation?cited by 19
  31. 2025Dataset Distillation via Vision-Language Category Prototypecited by 19
  32. 2024Graph Data Condensation via Self-expressive Graph Structure Reconstructioncited by 18
  33. 2024Generative Dataset Distillation: Balancing Global Structure and Local Detailscited by 18
  34. 2023Data Distillation Can Be Like Vodka: Distilling More Times For Better Qualitycited by 17
  35. 2024Low-Rank Similarity Mining for Multimodal Dataset Distillationcited by 17
  36. 2024DANCE: Dual-View Distribution Alignment for Dataset Condensationcited by 15
  37. 2024Not All Samples Should Be Utilized Equally: Towards Understanding and Improving Dataset Distillationcited by 14
  38. 2024Neural Spectral Decomposition for Dataset Distillationcited by 14

50 most cited of 188 All 188 (JSON)