# Generative Dataset Distillation Based on Diffusion Model arXiv: https://arxiv.org/abs/2408.08610 ยท HTML: https://mlsanity.com/paper/2408.08610/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 55 of its 63 references that are in the map. - First posted on arXiv 16 Aug 2024. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 30 of 55) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022) - [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018) - [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021) - [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021) - [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021) - [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022) - [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023) - [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018) - [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020) - [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022) - [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015) - [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021) - [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022) - [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018) - [Dataset Meta-Learning from Kernel Ridge-Regression](https://mlsanity.com/paper/2011.00050/) (2020) - [Dataset Distillation with Infinitely Wide Convolutional Networks](https://mlsanity.com/paper/2107.13034/) (2021) - [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022) - [Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory](https://mlsanity.com/paper/2211.10586/) (2022) - [Dataset Distillation using Neural Feature Regression](https://mlsanity.com/paper/2206.00719/) (2022) - [Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective](https://mlsanity.com/paper/2306.13092/) (2023) - [Generalizing Dataset Distillation via Deep Generative Prior](https://mlsanity.com/paper/2305.01649/) (2023) - [Improved Distribution Matching for Dataset Condensation](https://mlsanity.com/paper/2307.09742/) (2023) - [A Comprehensive Survey of Dataset Distillation](https://mlsanity.com/paper/2301.05603/) (2023) ## Built on by (most cited first, 13 of 13) - [Visual Large Language Models for Generalized and Specialized Applications](https://mlsanity.com/paper/2501.02765/) (2025) - [Hyperbolic Dataset Distillation](https://mlsanity.com/paper/2505.24623/) (2025) - [Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory](https://mlsanity.com/paper/2505.19469/) (2025) - [Information-Guided Diffusion Sampling for Dataset Distillation](https://mlsanity.com/paper/2507.04619/) (2025) - [Task-Specific Generative Dataset Distillation with Difficulty-Guided Sampling](https://mlsanity.com/paper/2507.03331/) (2025) - [Generative Dataset Distillation Based on Self-knowledge Distillation](https://mlsanity.com/paper/2501.04202/) (2025) - [FD2: A Dedicated Framework for Fine-Grained Dataset Distillation](https://mlsanity.com/paper/2603.25144/) (2026) - [Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Models](https://mlsanity.com/paper/2605.07194/) (2026) - [scDD: Latent Codes Based scRNA-seq Dataset Distillation with Foundation Model Knowledge](https://mlsanity.com/paper/2503.04357/) (2025) - [Dataset Distillation for Super-Resolution without Class Labels and Pre-trained Models](https://mlsanity.com/paper/2509.14777/) (2025) - [Foreground-Aware Dataset Distillation via Dynamic Patch Selection](https://mlsanity.com/paper/2601.02727/) (2026) - [Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Tasks](https://mlsanity.com/paper/2601.10090/) (2026) - [Combining Adam and its Inverse Counterpart to Enhance Generalization of Deep Learning Optimizers](https://mlsanity.com/paper/2603.07122/) (2026) ## Abstract (arXiv) This paper presents our method for the generative track of The First Dataset Distillation Challenge at ECCV 2024. Since the diffusion model has become the mainstay of generative models because of its high-quality generative effects, we focus on distillation methods based on the diffusion model. Considering that the track can only generate a fixed number of images in 10 minutes using a generative model for CIFAR-100 and Tiny-ImageNet datasets, we need to use a generative model that can generate images at high speed. In this study, we proposed a novel generative dataset distillation method based on Stable Diffusion. Specifically, we use the SDXL-Turbo model which can generate images at high speed and quality. Compared to other diffusion models that can only generate images per class (IPC) = 1, our method can achieve an IPC = 10 for Tiny-ImageNet and an IPC = 20 for CIFAR-100, respectively. Additionally, to generate high-quality distilled datasets for CIFAR-100 and Tiny-ImageNet, we use the class information as text prompts and post data augmentation for the SDXL-Turbo model. Experimental results show the effectiveness of the proposed method, and we achieved third place in the generative track of the ECCV 2024 DD Challenge. Codes are available at https://github.com/Guang000/BANKO.