# Curriculum Dataset Distillation

arXiv: https://arxiv.org/abs/2405.09150 · HTML: https://mlsanity.com/paper/2405.09150/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2024.

## Built on by (most cited first, 9 of 9)

- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [Neural Spectral Decomposition for Dataset Distillation](https://mlsanity.com/paper/2408.16236/) (2024)
- [Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensator](https://mlsanity.com/paper/2408.06927/) (2024)
- [Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation](https://mlsanity.com/paper/2510.04838/) (2025)
- [Learnability-Guided Diffusion for Dataset Distillation](https://mlsanity.com/paper/2604.00519/) (2026)
- [FARTrack: Fast Autoregressive Visual Tracking with High Performance](https://mlsanity.com/paper/2602.03214/) (2026)
- [Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation](https://mlsanity.com/paper/2606.20196/) (2026)
- [Memory-efficient Continual Learning with Prototypical Exemplar Condensation](https://mlsanity.com/paper/2603.13804/) (2026)
- [Dataset Distillation Based on Saliency-Driven Prototype Alignment](https://mlsanity.com/paper/2607.25318/) (2026)

## Abstract (arXiv)

Most dataset distillation methods struggle to accommodate large-scale datasets due to their substantial computational and memory requirements. Recent research has begun to explore scalable disentanglement methods. However, there are still performance bottlenecks and room for optimization in this direction. In this paper, we present a curriculum-based dataset distillation framework aiming to harmonize performance and scalability. This framework strategically distills synthetic images, adhering to a curriculum that transitions from simple to complex. By incorporating curriculum evaluation, we address the issue of previous methods generating images that tend to be homogeneous and simplistic, doing so at a manageable computational cost. Furthermore, we introduce adversarial optimization towards synthetic images to further improve their representativeness and safeguard against their overfitting to the neural network involved in distilling. This enhances the generalization capability of the distilled images across various neural network architectures and also increases their robustness to noise. Extensive experiments demonstrate that our framework sets new benchmarks in large-scale dataset distillation, achieving substantial improvements of 11.1% on Tiny-ImageNet, 9.0% on ImageNet-1K, and 7.3% on ImageNet-21K. Our distilled datasets and code are available at https://github.com/MIV-XJTU/CUDD.