# Self-Supervised Representation-Guided Generative Dataset Distillation

arXiv: https://arxiv.org/abs/2608.03218 · HTML: https://mlsanity.com/paper/2608.03218/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 55 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 6 of 6)

- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [Decoupled Audio-Visual Dataset Distillation](https://mlsanity.com/paper/2511.17890/) (2025)
- [Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Models](https://mlsanity.com/paper/2605.07194/) (2026)

## Abstract (arXiv)

Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative dataset distillation (SRG), a framework that translates the SSL geometry into diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype alignment, inter-class discrimination, and intra-class assignment. During diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for dataset distillation with pretrained SSL models.