mlsanityData to 7 Oct 2026

Dataset Condensation with Gradient Matching

arXiv 2006.05929 ↗First posted 10 Jun 2020cs.CV cs.LGcited by 510

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As the state-of-the-art machine learning methods in many fields rely on larger datasets, storing datasets and training models on them become significantly more expensive. This paper proposes a training set synthesis technique for data-efficient learning, called Dataset Condensation, that learns to condense large dataset into a small set of informative synthetic samples for training deep neural networks from scratch. We formulate this goal as a gradient matching problem between the gradients of deep neural network weights that are trained on the original and our synthetic data. We rigorously evaluate its performance in several computer vision benchmarks and demonstrate that it significantly outperforms the state-of-the-art methods. Finally we explore the use of our method in continual learning and neural architecture search and report promising gains when limited memory and computations are available.

Key facts. Cited by 510 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 58; 2023: 97; 2024: 138; 2025: 133; 2026: 72.

510 papers.

Sort:

By area:

  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  2. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  3. 2021Dataset Condensation with Distribution Matchingcited by 362
  4. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  5. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  6. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 187
  7. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 180
  8. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  9. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  10. 2021Graph Condensation for Graph Neural Networkscited by 150
  11. 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 146
  12. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 146
38 more
  1. 2022Minimizing the Accumulated Trajectory Error to Improve Dataset Distillationcited by 142
  2. 2023Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspectivecited by 136
  3. 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 132
  4. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  5. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 125
  6. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  7. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 119
  8. 2023Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matchingcited by 118
  9. 2022Dataset Distillation via Factorizationcited by 115
  10. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  11. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  12. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 107
  13. 2022Dataset Condensation with Contrastive Signalscited by 105
  14. 2022Remember the Past: Distilling Datasets into Addressable Memories for Neural Networkscited by 104
  15. 2022Condensing Graphs via One-Step Gradient Matchingcited by 100
  16. 2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratchcited by 98
  17. 2023DREAM: Efficient Dataset Distillation by Representative Matchingcited by 93
  18. 2023InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruningcited by 90
  19. 2023Data Distillation: A Surveycited by 86
  20. 2023DataDAM: Efficient Dataset Distillation with Attention Matchingcited by 83
  21. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 81
  22. 2022Synthesizing Informative Training Samples with GANcited by 77
  23. 2024D4M: Dataset Distillation via Disentangled Diffusion Modelcited by 76
  24. 2023Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Datacited by 71
  25. 2023Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matchingcited by 69
  26. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  27. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  28. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  29. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  30. 2023Large-scale Dataset Pruning with Dynamic Uncertaintycited by 63
  31. 2022Accelerating Dataset Distillation via Model Augmentationcited by 62
  32. 2022Expanding Small-Scale Datasets with Guided Imaginationcited by 61
  33. 2022Graph Condensation via Receptive Field Distribution Matchingcited by 60
  34. 2024A Comprehensive Survey on Graph Reduction: Sparsification, Coarsening, and Condensationcited by 57
  35. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  36. 2023CaT: Balanced Continual Graph Learning with Graph Condensationcited by 50
  37. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
  38. 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49

50 most cited of 510 All 510 (JSON)