# Dataset Distillation via Committee Voting arXiv: https://arxiv.org/abs/2501.07575 ยท HTML: https://mlsanity.com/paper/2501.07575/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 42 of its 52 references that are in the map. - First posted on arXiv 13 Jan 2025. ## Builds on (most cited first, 30 of 42) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018) - [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017) - [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017) - [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021) - [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019) - [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020) - [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020) - [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022) - [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017) - [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021) - [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021) - [Dataset Distillation with Infinitely Wide Convolutional Networks](https://mlsanity.com/paper/2107.13034/) (2021) - [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022) - [Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory](https://mlsanity.com/paper/2211.10586/) (2022) - [Dataset Distillation using Neural Feature Regression](https://mlsanity.com/paper/2206.00719/) (2022) - [Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation](https://mlsanity.com/paper/2211.11004/) (2022) - [Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective](https://mlsanity.com/paper/2306.13092/) (2023) - [Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching](https://mlsanity.com/paper/2310.05773/) (2023) - [Dataset Distillation via Factorization](https://mlsanity.com/paper/2210.16774/) (2022) - [Efficient Dataset Distillation Using Random Feature Approximation](https://mlsanity.com/paper/2210.12067/) (2022) - [On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm](https://mlsanity.com/paper/2312.03526/) (2023) - [Dataset Condensation with Contrastive Signals](https://mlsanity.com/paper/2202.02916/) (2022) - [Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks](https://mlsanity.com/paper/2206.02916/) (2022) - [DataDAM: Efficient Dataset Distillation with Attention Matching](https://mlsanity.com/paper/2310.00093/) (2023) - [Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching](https://mlsanity.com/paper/2311.17950/) (2023) - [A Fast Knowledge Distillation Framework for Visual Recognition](https://mlsanity.com/paper/2112.01528/) (2021) - [LLM Voting: Human Choices and AI Collective Decision Making](https://mlsanity.com/paper/2402.01766/) (2024) ## Built on by (most cited first, 14 of 14) - [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025) - [FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation](https://mlsanity.com/paper/2506.24125/) (2025) - [DD-Ranking: Rethinking the Evaluation of Dataset Distillation](https://mlsanity.com/paper/2505.13300/) (2025) - [Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation](https://mlsanity.com/paper/2512.00308/) (2025) - [FD2: A Dedicated Framework for Fine-Grained Dataset Distillation](https://mlsanity.com/paper/2603.25144/) (2026) - [Rectified Decoupled Dataset Distillation: A Closer Look for Fair and Comprehensive Evaluation](https://mlsanity.com/paper/2509.19743/) (2025) - [Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift](https://mlsanity.com/paper/2512.15647/) (2025) - [Geometry-Aware Dataset Condensation for Diffusion Model Training](https://mlsanity.com/paper/2606.05883/) (2026) - [Condensing Large-Scale Datasets Directly with Minimal Information Loss](https://mlsanity.com/paper/2607.00916/) (2026) - [PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors](https://mlsanity.com/paper/2511.09905/) (2025) - [Beyond Dataset Distillation: Lossless Dataset Concentration via Diffusion-Assisted Distribution Alignment](https://mlsanity.com/paper/2603.27987/) (2026) - [Soft Label Pruning and Quantization for Large-Scale Dataset Distillation](https://mlsanity.com/paper/2604.18135/) (2026) - [DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation](https://mlsanity.com/paper/2608.25480/) (2026) - [PSM: Dataset Distillation Based on Precise Statistical Matching by Difficulty](https://mlsanity.com/paper/2609.34299/) (2026) ## Abstract (arXiv) Dataset distillation aims to synthesize a compact yet representative dataset that preserves the essential characteristics of the original data for efficient model training. Existing methods mainly focus on improving data-synthetic alignment or scaling distillation to large datasets. In this work, we propose Committee Voting for Dataset Distillation (CV-DD), an orthogonal approach that leverages the collective knowledge of multiple models to produce higher-quality distilled data. We first establish a strong baseline that achieves state-of-the-art performance through modern architectural and optimization choices. By integrating distributions and predictions from multiple models and generating high-quality soft labels, our method captures a broader range of data characteristics, reduces model-specific bias and the impact of distribution shifts, and significantly improves generalization. This voting-based strategy enhances diversity and robustness, alleviates overfitting, and improves post-evaluation performance. Extensive experiments across multiple datasets and IPC settings demonstrate that CV-DD consistently outperforms single- and multi-model distillation methods and generalizes well to non-training-based frameworks and challenging synthetic-to-real transfer tasks. Code is available at: https://github.com/Jiacheng8/CV-DD.