# Dataset Condensation with Latent Space Knowledge Factorization and Sharing

arXiv: https://arxiv.org/abs/2208.10494 · HTML: https://mlsanity.com/paper/2208.10494/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 21 Aug 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Forward and Reverse Gradient-Based Hyperparameter Optimization](https://mlsanity.com/paper/1703.01785/) (2017)
- [Dataset Meta-Learning from Kernel Ridge-Regression](https://mlsanity.com/paper/2011.00050/) (2020)
- [Dataset Distillation with Infinitely Wide Convolutional Networks](https://mlsanity.com/paper/2107.13034/) (2021)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022)
- [Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data](https://mlsanity.com/paper/1912.07768/) (2019)
- [Preventing Posterior Collapse with delta-VAEs](https://mlsanity.com/paper/1901.03416/) (2019)
- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Coresets and Sketches](https://mlsanity.com/paper/1601.00617/) (2016)

## Built on by (most cited first, 30 of 34)

- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [A Comprehensive Survey of Dataset Distillation](https://mlsanity.com/paper/2301.05603/) (2023)
- [Data Distillation: A Survey](https://mlsanity.com/paper/2301.04272/) (2023)
- [Sequential Subset Matching for Dataset Distillation](https://mlsanity.com/paper/2311.01570/) (2023)
- [A Survey on Dataset Distillation: Approaches, Applications and Future Directions](https://mlsanity.com/paper/2305.01975/) (2023)
- [Dataset Distillation via the Wasserstein Metric](https://mlsanity.com/paper/2311.18531/) (2023)
- [Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation](https://mlsanity.com/paper/2404.00563/) (2024)
- [A Label is Worth a Thousand Images in Dataset Distillation](https://mlsanity.com/paper/2406.10485/) (2024)
- [Dataset Distillation by Automatic Training Trajectories](https://mlsanity.com/paper/2407.14245/) (2024)
- [Multisize Dataset Condensation](https://mlsanity.com/paper/2403.06075/) (2024)
- [Graph Data Condensation via Self-expressive Graph Structure Reconstruction](https://mlsanity.com/paper/2403.07294/) (2024)
- [Low-Rank Similarity Mining for Multimodal Dataset Distillation](https://mlsanity.com/paper/2406.03793/) (2024)
- [Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensator](https://mlsanity.com/paper/2408.06927/) (2024)
- [Generative Dataset Distillation Based on Diffusion Model](https://mlsanity.com/paper/2408.08610/) (2024)
- [Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matching](https://mlsanity.com/paper/2410.07579/) (2024)
- [FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation](https://mlsanity.com/paper/2506.24125/) (2025)
- [Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspective](https://mlsanity.com/paper/2311.16646/) (2023)
- [A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairness](https://mlsanity.com/paper/2305.03355/) (2023)
- [Dataset Condensation for Time Series Classification via Dual Domain Matching](https://mlsanity.com/paper/2403.07245/) (2024)
- [Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation](https://mlsanity.com/paper/2305.18381/) (2023)
- [Dataset Distillation in Latent Space](https://mlsanity.com/paper/2311.15547/) (2023)
- [Distributional Dataset Distillation with Subtask Decomposition](https://mlsanity.com/paper/2403.00999/) (2024)
- [Color-Oriented Redundancy Reduction in Dataset Distillation](https://mlsanity.com/paper/2411.11329/) (2024)
- [Unifying Dataset Pruning and Distillation for Efficient Large-scale Compression](https://mlsanity.com/paper/2502.06434/) (2025)
- [Multi-Source Domain Adaptation meets Dataset Distillation through Dataset Dictionary Learning](https://mlsanity.com/paper/2309.07666/) (2023)
- [Embarassingly Simple Dataset Distillation](https://mlsanity.com/paper/2311.07025/) (2023)
- [Distilling Dataset into Neural Field](https://mlsanity.com/paper/2503.04835/) (2025)
- [Mitigating Bias in Dataset Distillation](https://mlsanity.com/paper/2406.06609/) (2024)
- [Boosting the Cross-Architecture Generalization of Dataset Distillation through an Empirical Study](https://mlsanity.com/paper/2312.05598/) (2023)
- [Distilling Datasets Into Less Than One Image](https://mlsanity.com/paper/2403.12040/) (2024)

## Abstract (arXiv)

In this paper, we introduce a novel approach for systematically solving dataset condensation problem in an efficient manner by exploiting the regularity in a given dataset. Instead of condensing the dataset directly in the original input space, we assume a generative process of the dataset with a set of learnable codes defined in a compact latent space followed by a set of tiny decoders which maps them differently to the original input space. By combining different codes and decoders interchangeably, we can dramatically increase the number of synthetic examples with essentially the same parameter count, because the latent space is much lower dimensional and since we can assume as many decoders as necessary to capture different styles represented in the dataset with negligible cost. Such knowledge factorization allows efficient sharing of information between synthetic examples in a systematic way, providing far better trade-off between compression ratio and quality of the generated examples. We experimentally show that our method achieves new state-of-the-art records by significant margins on various benchmark datasets such as SVHN, CIFAR10, CIFAR100, and TinyImageNet.