# CaO2: Rectifying Inconsistencies in Diffusion-Based Dataset Distillation

arXiv: https://arxiv.org/abs/2506.22637 · HTML: https://mlsanity.com/paper/2506.22637/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 30)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)
- [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022)
- [Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory](https://mlsanity.com/paper/2211.10586/) (2022)
- [Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation](https://mlsanity.com/paper/2211.11004/) (2022)
- [Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective](https://mlsanity.com/paper/2306.13092/) (2023)
- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [Generalizing Dataset Distillation via Deep Generative Prior](https://mlsanity.com/paper/2305.01649/) (2023)
- [Improved Distribution Matching for Dataset Condensation](https://mlsanity.com/paper/2307.09742/) (2023)
- [Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching](https://mlsanity.com/paper/2310.05773/) (2023)
- [On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm](https://mlsanity.com/paper/2312.03526/) (2023)
- [DREAM: Efficient Dataset Distillation by Representative Matching](https://mlsanity.com/paper/2302.14416/) (2023)
- [Efficient Dataset Distillation via Minimax Diffusion](https://mlsanity.com/paper/2311.15529/) (2023)
- [Robust Classification via a Single Diffusion Model](https://mlsanity.com/paper/2305.15241/) (2023)
- [Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching](https://mlsanity.com/paper/2311.17950/) (2023)
- [Dataset Distillation with Neural Characteristic Function: A Minmax Perspective](https://mlsanity.com/paper/2502.20653/) (2025)
- [Elucidating the Design Space of Dataset Condensation](https://mlsanity.com/paper/2404.13733/) (2024)
- [Reducing catastrophic forgetting with learning on synthetic data](https://mlsanity.com/paper/2004.14046/) (2020)
- [You Only Condense Once: Two Rules for Pruning Condensed Datasets](https://mlsanity.com/paper/2310.14019/) (2023)
- [Self-supervised Dataset Distillation: A Good Compression Is All You Need](https://mlsanity.com/paper/2404.07976/) (2024)
- [Dataset Quantization with Active Learning based Adaptive Sampling](https://mlsanity.com/paper/2407.07268/) (2024)
- [DD-Ranking: Rethinking the Evaluation of Dataset Distillation](https://mlsanity.com/paper/2505.13300/) (2025)

## Built on by (most cited first, 14 of 14)

- [HIERAMP: Coarse-to-Fine Autoregressive Amplification for Generative Dataset Distillation](https://mlsanity.com/paper/2603.06932/) (2026)
- [Efficient Multimodal Dataset Distillation via Generative Models](https://mlsanity.com/paper/2509.15472/) (2025)
- [Diffusion Models as Dataset Distillation Priors](https://mlsanity.com/paper/2510.17421/) (2025)
- [Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift](https://mlsanity.com/paper/2512.15647/) (2025)
- [Towards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion](https://mlsanity.com/paper/2601.15829/) (2026)
- [Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Space](https://mlsanity.com/paper/2606.21705/) (2026)
- [Distill Video Datasets into Images](https://mlsanity.com/paper/2512.14621/) (2025)
- [Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Tasks](https://mlsanity.com/paper/2601.10090/) (2026)
- [Multimodal Dataset Distillation Made Simple by Prototype-Guided Data Synthesis](https://mlsanity.com/paper/2602.19756/) (2026)
- [IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation](https://mlsanity.com/paper/2603.13960/) (2026)
- [SAS: Semantic-aware Sampling for Generative Dataset Distillation](https://mlsanity.com/paper/2605.18012/) (2026)
- [D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentation](https://mlsanity.com/paper/2605.25022/) (2026)
- [Pool-Select-Refine for Allocation-Aware Generative Dataset Distillation](https://mlsanity.com/paper/2606.01920/) (2026)
- [MGPO: Manifold-Guided Diffusion Alignment for Task-Aware Dataset Distillation](https://mlsanity.com/paper/2610.05252/) (2026)

## Abstract (arXiv)

The recent introduction of diffusion models in dataset distillation has shown promising potential in creating compact surrogate datasets for large, high-resolution target datasets, offering improved efficiency and performance over traditional bi-level/uni-level optimization methods. However, current diffusion-based dataset distillation approaches overlook the evaluation process and exhibit two critical inconsistencies in the distillation process: (1) Objective Inconsistency, where the distillation process diverges from the evaluation objective, and (2) Condition Inconsistency, leading to mismatches between generated images and their corresponding conditions. To resolve these issues, we introduce Condition-aware Optimization with Objective-guided Sampling (CaO2), a two-stage diffusion-based framework that aligns the distillation process with the evaluation objective. The first stage employs a probability-informed sample selection pipeline, while the second stage refines the corresponding latent representations to improve conditional likelihood. CaO2 achieves state-of-the-art performance on ImageNet and its subsets, surpassing the best-performing baselines by an average of 2.3% accuracy.