mlsanityData to 14 Oct 2026

A Survey on Dataset Distillation: Approaches, Applications and Future Directions

arXiv 2305.01975 ↗First posted 3 May 2023cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Dataset distillation is attracting more attention in machine learning as training sets continue to grow and the cost of training state-of-the-art models becomes increasingly high. By synthesizing datasets with high information density, dataset distillation offers a range of potential applications, including support for continual learning, neural architecture search, and privacy protection. Despite recent advances, we lack a holistic understanding of the approaches and applications. Our survey aims to bridge this gap by first proposing a taxonomy of dataset distillation, characterizing existing approaches, and then systematically reviewing the data modalities, and related applications. In addition, we summarize the challenges and discuss future directions for this field of research.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 41 of its 61 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

41 of 61 references.

References · 41

  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  5. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  6. 2023Extracting Training Data from Diffusion Modelscited by 732
  7. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  8. 2016Learning to learn by gradient descent by gradient descentcited by 620
  9. 2018End-to-End Incremental Learningcited by 597
  10. 2020Dataset Condensation with Gradient Matchingcited by 516
  11. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  12. 2021Dataset Condensation with Distribution Matchingcited by 365
29 more
  1. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
  2. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  3. 2017Automated Curriculum Learning for Neural Networkscited by 257
  4. 2018Dataset Distillationcited by 230
  5. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  6. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  7. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 188
  8. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  9. 2021Graph Condensation for Graph Neural Networkscited by 150
  10. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 120
  11. 2022Dataset Distillation via Factorizationcited by 115
  12. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  13. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
  14. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  15. 2022Remember the Past: Distilling Datasets into Addressable Memories for Neural Networkscited by 104
  16. 2017Learning Active Learning from Datacited by 101
  17. 2022Condensing Graphs via One-Step Gradient Matchingcited by 100
  18. 2022Synthesizing Informative Training Samples with GANcited by 77
  19. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  20. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 66
  21. 2022Graph Condensation via Receptive Field Distribution Matchingcited by 60
  22. 2021Towards General Deep Leakage in Federated Learningcited by 40
  23. 2022Dataset Condensation with Latent Space Knowledge Factorization and Sharingcited by 34
  24. 2022Private Set Generation with Discriminative Informationcited by 34
  25. 2022FedSynth: Gradient Compression via Synthetic Data in Federated Learningcited by 31
  26. 2022Dataset Distillation for Medical Dataset Sharingcited by 25
  27. 2023Backdoor Attacks Against Dataset Distillationcited by 21
  28. 2022Can we achieve robustness from data alone?cited by 16
  29. 2022Towards Robust Dataset Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 11; 2025: 10; 2026: 6.

32 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey on Graph Reduction: Sparsification, Coarsening, and Condensationcited by 57
  2. 2023Does Graph Distillation See Like Vision Dataset Counterpart?cited by 45
  3. 2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 27
  4. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
  5. 2024Graph Condensation: A Surveycited by 23
  6. 2024DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillationcited by 20
  7. 2025Dataset Distillation via Vision-Language Category Prototypecited by 19
  8. 2024Towards Adversarially Robust Dataset Distillation by Curvature Regularizationcited by 17
  9. 2023Graph Distillation with Eigenbasis Matchingcited by 16
  10. 2024DANCE: Dual-View Distribution Alignment for Dataset Condensationcited by 15
  11. 2024Generative Dataset Distillation Based on Diffusion Modelcited by 13
  12. 2023Dataset Distillation in Latent Spacecited by 8
20 more
  1. 2025UniDetox: Universal Detoxification of Large Language Models via Dataset Distillationcited by 8
  2. 2023DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matchingcited by 6
  3. 2023Embarassingly Simple Dataset Distillationcited by 5
  4. 2024Progressive trajectory matching for medical dataset distillationcited by 5
  5. 2025Dataset Condensation with Color Compensationcited by 5
  6. 2025Beyond Random: Automatic Inner-loop Optimization in Dataset Distillationcited by 5
  7. 2024Dataset Distillation from First Principles: Integrating Core Information Extraction and Purposeful Learningcited by 3
  8. 2025Understanding Dataset Distillation via Spectral Filteringcited by 2
  9. 2025A Discrepancy-Based Perspective on Dataset Condensationcited by 1
  10. 2026Geometry-Aware Dataset Condensation for Diffusion Model Trainingcited by 1
  11. 2024Brain-Inspired Continual Learning-Robust Feature Distillation and Re-Consolidation for Class Incremental Learningcited by 0
  12. 2024Distributionally Robust Safe Sample Elimination under Covariate Shiftcited by 0
  13. 2025scDD: Latent Codes Based scRNA-seq Dataset Distillation with Foundation Model Knowledgecited by 0
  14. 2025Model-to-Model Knowledge Transmission (M2KT): A Data-Free Framework for Cross-Model Understanding Transfercited by 0
  15. 2025Technical Report on Text Dataset Distillationcited by 0
  16. 2026Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Taskscited by 0
  17. 2026Beyond Confidence: The Rhythms of Reasoning in Generative Modelscited by 0
  18. 2026IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillationcited by 0
  19. 2026Omnimodal Dataset Distillation via High-order Proxy Alignmentcited by 0
  20. 2026D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentationcited by 0