mlsanityData to 14 Oct 2026

Generative Dataset Distillation: Balancing Global Structure and Local Details

arXiv 2404.17732 ↗First posted 26 Apr 2024cs.CV cs.AI cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a new dataset distillation method that considers balancing global structure and local details when distilling the information from a large dataset into a generative model. Dataset distillation has been proposed to reduce the size of the required dataset when training models. The conventional dataset distillation methods face the problem of long redeployment time and poor cross-architecture performance. Moreover, previous methods focused too much on the high-level semantic attributes between the synthetic dataset and the original dataset while ignoring the local features such as texture and shape. Based on the above understanding, we propose a new method for distilling the original image dataset into a generative model. Our method involves using a conditional generative adversarial network to generate the distilled dataset. Subsequently, we ensure balancing global structure and local details in the distillation process, continuously optimizing the generator for more information-dense dataset generation.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

33 of 42 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2014Conditional Generative Adversarial Netscited by 2,162
  4. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  5. 2020Dataset Condensation with Gradient Matchingcited by 516
  6. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  7. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  8. 2021Dataset Condensation with Distribution Matchingcited by 365
  9. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 304
  10. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  11. 2018Dataset Distillationcited by 230
  12. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
21 more
  1. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  2. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 188
  3. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  4. 2018Fair Clustering Through Fairletscited by 176
  5. 2023Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matchingcited by 118
  6. 2023A Comprehensive Survey of Dataset Distillationcited by 115
  7. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
  8. 2022Dataset Condensation with Contrastive Signalscited by 105
  9. 2020Flexible Dataset Distillation: Learn Labels Instead of Imagescited by 91
  10. 2023DataDAM: Efficient Dataset Distillation with Attention Matchingcited by 84
  11. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 60
  12. 2021Soft-Label Anonymous Gastric X-ray Image Distillationcited by 51
  13. 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49
  14. 2023DiM: Distilling Dataset into Generative Modelcited by 46
  15. 2023Data-centric AI: Perspectives and Challengescited by 43
  16. 2022Delving into Effective Gradient Matching for Dataset Condensationcited by 40
  17. 2022Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharingcited by 35
  18. 2022Dataset Distillation Using Parameter Pruningcited by 27
  19. 2019A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trendscited by 26
  20. 2022Dataset Distillation for Medical Dataset Sharingcited by 25
  21. 2024Importance-Aware Adaptive Dataset Distillationcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 9; 2026: 8.

18 papers.

Sort:

By area:

  1. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
  2. 2025Hyperbolic Dataset Distillationcited by 15
  3. 2024Generative Dataset Distillation Based on Diffusion Modelcited by 13
  4. 2025Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memorycited by 10
  5. 2025Information-Guided Diffusion Sampling for Dataset Distillationcited by 10
  6. 2025Task-Specific Generative Dataset Distillation with Difficulty-Guided Samplingcited by 8
  7. 2025Decoupled Audio-Visual Dataset Distillationcited by 7
  8. 2025Generative Dataset Distillation Based on Self-knowledge Distillationcited by 5
  9. 2026FD2: A Dedicated Framework for Fine-Grained Dataset Distillationcited by 3
  10. 2026Learnability-Guided Diffusion for Dataset Distillationcited by 3
  11. 2026Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Modelscited by 3
  12. 2025Latent Video Dataset Distillationcited by 1
6 more
  1. 2026Efficient Video Dataset Distillation via Cluster-Guided Prototype Blendingcited by 1
  2. 2025Secure and Explainable Fraud Detection in Finance via Hierarchical Multi-source Dataset Distillationcited by 0
  3. 2026Foreground-Aware Dataset Distillation via Dynamic Patch Selectioncited by 0
  4. 2026Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Taskscited by 0
  5. 2026Soft Label Pruning and Quantization for Large-Scale Dataset Distillationcited by 0
  6. 2026SAS: Semantic-aware Sampling for Generative Dataset Distillationcited by 0