mlsanityData to 14 Oct 2026

A Discrepancy-Based Perspective on Dataset Condensation

arXiv 2509.10367 ↗First posted 12 Sep 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Given a dataset of finitely many elements T = xii = 1N, the goal of dataset condensation (DC) is to construct a synthetic dataset S = tildexjj = 1M which is significantly smaller (M ll N) such that a model trained from scratch on S achieves comparable or even superior generalization performance to a model trained on T. Recent advances in DC reveal a close connection to the problem of approximating the data distribution represented by T with a reduced set of points. In this work, we present a unified framework that encompasses existing DC methods and extend the task-specific notion of DC to a more general and formal definition using notions of discrepancy, which quantify the distance between probability distribution in different regimes. Our framework broadens the objective of DC beyond generalization, accommodating additional objectives such as robustness, privacy, and other desirable properties.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 69 of its 98 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

69 of 98 references.

References · 69

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  10. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  11. 2014Conditional Generative Adversarial Netscited by 2,162
  12. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
38 more
  1. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  2. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  3. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  4. 2019A Closer Look at Few-shot Classificationcited by 684
  5. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  6. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  7. 2020Dataset Condensation with Gradient Matchingcited by 516
  8. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  9. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  10. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  11. 2021Dataset Condensation with Distribution Matchingcited by 365
  12. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  13. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  14. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 284
  15. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 254
  16. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  17. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  18. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 188
  19. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  20. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 169
  21. 2023Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspectivecited by 138
  22. 2023Dataset Distillation: A Comprehensive Reviewcited by 127
  23. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 126
  24. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  25. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  26. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 120
  27. 2022Dataset Distillation via Factorizationcited by 115
  28. 2023A Comprehensive Survey of Dataset Distillationcited by 115
  29. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  30. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 109
  31. 2022Dataset Condensation with Contrastive Signalscited by 105
  32. 2023DREAM: Efficient Dataset Distillation by Representative Matchingcited by 94
  33. 2023Data Distillation: A Surveycited by 87
  34. 2023DataDAM: Efficient Dataset Distillation with Attention Matchingcited by 84
  35. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 83
  36. 2020DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generationcited by 78
  37. 2022Synthesizing Informative Training Samples with GANcited by 77
  38. 2023M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancycited by 49

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AIcited by 0