# Dataset Condensation via Generative Model arXiv: https://arxiv.org/abs/2309.07698 ยท HTML: https://mlsanity.com/paper/2309.07698/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 56 of its 71 references that are in the map. - First posted on arXiv 14 Sep 2023. ## Builds on (most cited first, 30 of 56) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022) - [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017) - [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020) - [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018) - [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014) - [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021) - [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016) - [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021) - [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020) - [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017) - [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021) - [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021) - [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016) - [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018) - [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018) - [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019) - [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016) - [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021) - [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020) - [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022) - [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022) - [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021) - [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016) - [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021) - [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021) - [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021) - [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017) ## Built on by (most cited first, 15 of 15) - [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025) - [Multisize Dataset Condensation](https://mlsanity.com/paper/2403.06075/) (2024) - [Generative Dataset Distillation Based on Diffusion Model](https://mlsanity.com/paper/2408.08610/) (2024) - [MGD3: Mode-Guided Dataset Distillation using Diffusion Models](https://mlsanity.com/paper/2505.18963/) (2025) - [Mitigating Bias in Dataset Distillation](https://mlsanity.com/paper/2406.06609/) (2024) - [Condensing Action Segmentation Datasets via Generative Network Inversion](https://mlsanity.com/paper/2503.14112/) (2025) - [ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation](https://mlsanity.com/paper/2602.23295/) (2026) - [EVLF: Early Vision-Language Fusion for Generative Dataset Distillation](https://mlsanity.com/paper/2603.07476/) (2026) - [Decomposed Distribution Matching in Dataset Condensation](https://mlsanity.com/paper/2412.04748/) (2024) - [Learnability-Guided Diffusion for Dataset Distillation](https://mlsanity.com/paper/2604.00519/) (2026) - [A Discrepancy-Based Perspective on Dataset Condensation](https://mlsanity.com/paper/2509.10367/) (2025) - [Dynamic-Aware Video Distillation: Optimizing Temporal Resolution Based on Video Semantics](https://mlsanity.com/paper/2506.02021/) (2025) - [Path-Guided Flow Matching for Dataset Distillation](https://mlsanity.com/paper/2602.05616/) (2026) - [Multimodal Dataset Distillation Made Simple by Prototype-Guided Data Synthesis](https://mlsanity.com/paper/2602.19756/) (2026) - [Dataset Distillation by Influence Matching](https://mlsanity.com/paper/2607.16859/) (2026) ## Abstract (arXiv) Dataset condensation aims to condense a large dataset with a lot of training samples into a small set. Previous methods usually condense the dataset into the pixels format. However, it suffers from slow optimization speed and large number of parameters to be optimized. When increasing image resolutions and classes, the number of learnable parameters grows accordingly, prohibiting condensation methods from scaling up to large datasets with diverse classes. Moreover, the relations among condensed samples have been neglected and hence the feature distribution of condensed samples is often not diverse. To solve these problems, we propose to condense the dataset into another format, a generative model. Such a novel format allows for the condensation of large datasets because the size of the generative model remains relatively stable as the number of classes or image resolution increases. Furthermore, an intra-class and an inter-class loss are proposed to model the relation of condensed samples. Intra-class loss aims to create more diverse samples for each class by pushing each sample away from the others of the same class. Meanwhile, inter-class loss increases the discriminability of samples by widening the gap between the centers of different classes. Extensive comparisons with state-of-the-art methods and our ablation studies confirm the effectiveness of our method and its individual component. To our best knowledge, we are the first to successfully conduct condensation on ImageNet-1k.