# Dataset Distillation Using Parameter Pruning

arXiv: https://arxiv.org/abs/2209.14609 · HTML: https://mlsanity.com/paper/2209.14609/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [Dataset Distillation with Infinitely Wide Convolutional Networks](https://mlsanity.com/paper/2107.13034/) (2021)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Privacy for Free: How does Dataset Condensation Help Privacy?](https://mlsanity.com/paper/2206.00240/) (2022)
- [Soft-Label Dataset Distillation and Text Dataset Distillation](https://mlsanity.com/paper/1910.02551/) (2019)
- [Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data](https://mlsanity.com/paper/1912.07768/) (2019)
- [Flexible Dataset Distillation: Learn Labels Instead of Images](https://mlsanity.com/paper/2006.08572/) (2020)
- [Practical Coreset Constructions for Machine Learning](https://mlsanity.com/paper/1703.06476/) (2017)
- [Soft-Label Anonymous Gastric X-ray Image Distillation](https://mlsanity.com/paper/2104.02857/) (2021)
- [Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments](https://mlsanity.com/paper/2208.11311/) (2022)
- [Sample Condensation in Online Continual Learning](https://mlsanity.com/paper/2206.11849/) (2022)
- [Condensed Composite Memory Continual Learning](https://mlsanity.com/paper/2102.09890/) (2021)
- [Self-Knowledge Distillation based Self-Supervised Learning for Covid-19 Detection from Chest X-Ray Images](https://mlsanity.com/paper/2206.03009/) (2022)
- [TriBYOL: Triplet BYOL for Self-Supervised Representation Learning](https://mlsanity.com/paper/2206.03012/) (2022)

## Built on by (most cited first, 27 of 27)

- [Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation](https://mlsanity.com/paper/2211.11004/) (2022)
- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [Dataset Distillation by Automatic Training Trajectories](https://mlsanity.com/paper/2407.14245/) (2024)
- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [Multi-modal Vision Pre-training for Medical Image Analysis](https://mlsanity.com/paper/2410.10604/) (2024)
- [Towards Stable and Storage-efficient Dataset Distillation: Matching Convexified Trajectory](https://mlsanity.com/paper/2406.19827/) (2024)
- [Multisize Dataset Condensation](https://mlsanity.com/paper/2403.06075/) (2024)
- [Importance-Aware Adaptive Dataset Distillation](https://mlsanity.com/paper/2401.15863/) (2024)
- [Generative Dataset Distillation: Balancing Global Structure and Local Details](https://mlsanity.com/paper/2404.17732/) (2024)
- [Hyperbolic Dataset Distillation](https://mlsanity.com/paper/2505.24623/) (2025)
- [Dataset Quantization with Active Learning based Adaptive Sampling](https://mlsanity.com/paper/2407.07268/) (2024)
- [Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory](https://mlsanity.com/paper/2505.19469/) (2025)
- [Dataset Distillation in Latent Space](https://mlsanity.com/paper/2311.15547/) (2023)
- [Task-Specific Generative Dataset Distillation with Difficulty-Guided Sampling](https://mlsanity.com/paper/2507.03331/) (2025)
- [Efficient Multimodal Dataset Distillation via Generative Models](https://mlsanity.com/paper/2509.15472/) (2025)
- [Decoupled Audio-Visual Dataset Distillation](https://mlsanity.com/paper/2511.17890/) (2025)
- [CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecasting](https://mlsanity.com/paper/2406.02131/) (2024)
- [A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design](https://mlsanity.com/paper/2607.09084/) (2026)
- [Bayesian Pseudo-Coresets via Contrastive Divergence](https://mlsanity.com/paper/2303.11278/) (2023)
- [Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling](https://mlsanity.com/paper/2212.09281/) (2022)
- [Tackling Data Heterogeneity in Federated Time Series Forecasting](https://mlsanity.com/paper/2411.15716/) (2024)
- [FD2: A Dedicated Framework for Fine-Grained Dataset Distillation](https://mlsanity.com/paper/2603.25144/) (2026)
- [Distill Video Datasets into Images](https://mlsanity.com/paper/2512.14621/) (2025)
- [Federated Distillation for Whole Slide Image via Gaussian-Mixture Feature Alignment and Curriculum Integration](https://mlsanity.com/paper/2605.00578/) (2026)
- [SAS: Semantic-aware Sampling for Generative Dataset Distillation](https://mlsanity.com/paper/2605.18012/) (2026)
- [DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation](https://mlsanity.com/paper/2608.25480/) (2026)
- [PSM: Dataset Distillation Based on Precise Statistical Matching by Difficulty](https://mlsanity.com/paper/2609.34299/) (2026)

## Abstract (arXiv)

In this study, we propose a novel dataset distillation method based on parameter pruning. The proposed method can synthesize more robust distilled datasets and improve distillation performance by pruning difficult-to-match parameters during the distillation process. Experimental results on two benchmark datasets show the superiority of the proposed method.