mlsanityData to 14 Oct 2026

Dataset Distillation via the Wasserstein Metric

arXiv 2311.18531 ↗First posted 30 Nov 2023cs.CV cs.AI cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Dataset Distillation (DD) aims to generate a compact synthetic dataset that enables models to achieve performance comparable to training on the full large dataset, significantly reducing computational costs. Drawing from optimal transport theory, we introduce WMDD (Wasserstein Metric-based Dataset Distillation), a straightforward yet powerful method that employs the Wasserstein metric to enhance distribution matching. We compute the Wasserstein barycenter of features from a pretrained classifier to capture essential characteristics of the original data distribution. By optimizing synthetic data to align with this barycenter in feature space and leveraging per-class BatchNorm statistics to preserve intra-class variations, WMDD maintains the efficiency of distribution matching approaches while achieving state-of-the-art results across various high-resolution datasets. Our extensive experiments demonstrate WMDD's effectiveness and adaptability, highlighting its potential for advancing machine learning applications at scale.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 40 of its 65 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

40 of 65 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2021Convex optimizationcited by 2,515
  5. 2020Dataset Condensation with Gradient Matchingcited by 516
  6. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  7. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  8. 2021Dataset Condensation with Distribution Matchingcited by 365
  9. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  10. 2018Dataset Distillationcited by 230
  11. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  12. 2013Fast Computation of Wasserstein Barycenterscited by 216
28 more
  1. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 181
  2. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  3. 2022Minimizing the Accumulated Trajectory Error to Improve Dataset Distillationcited by 142
  4. 2023Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspectivecited by 138
  5. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 126
  6. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 120
  7. 2022Dataset Distillation via Factorizationcited by 115
  8. 2023A Comprehensive Survey of Dataset Distillationcited by 115
  9. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  10. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  11. 2022Remember the Past: Distilling Datasets into Addressable Memories for Neural Networkscited by 104
  12. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  13. 2023DataDAM: Efficient Dataset Distillation with Attention Matchingcited by 84
  14. 2022Synthesizing Informative Training Samples with GANcited by 77
  15. 2023Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matchingcited by 69
  16. 2020Federated Learning via Synthetic Datacited by 54
  17. 2020Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networkscited by 50
  18. 2021A Fast Knowledge Distillation Framework for Visual Recognitioncited by 50
  19. 2023Energy-Based Sliced Wasserstein Distancecited by 39
  20. 2022Dataset Condensation with Latent Space Knowledge Factorization and Sharingcited by 34
  21. 2021More than Encoder: Introducing Transformer Decoder to Upsamplecited by 26
  22. 2019Progressive Wasserstein Barycenters of Persistence Diagramscited by 15
  23. 2018A Fast Globally Linearly Convergent Algorithm for the Computation of Wasserstein Barycenterscited by 8
  24. 2019On the Computational Complexity of Finding a Sparse Wasserstein Barycentercited by 8
  25. 2020Randomised Wasserstein Barycenter Computation: Resampling with Statistical Guaranteescited by 8
  26. 2021Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneitycited by 7
  27. 2022Compound Batch Normalization for Long-tailed Image Classificationcited by 5
  28. 2023Towards Understanding Adversarial Transferability in Federated Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 8; 2026: 14.

31 papers.

Sort:

By area:

  1. 2024Elucidating the Design Space of Dataset Condensationcited by 40
  2. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
  3. 2024DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillationcited by 20
  4. 2024Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarioscited by 18
  5. 2024Towards Adversarially Robust Dataset Distillation by Curvature Regularizationcited by 17
  6. 2024Self-supervised Dataset Distillation: A Good Compression Is All You Needcited by 14
  7. 2024Distributional Dataset Distillation with Subtask Decompositioncited by 7
  8. 2025Efficient Multimodal Dataset Distillation via Generative Modelscited by 7
  9. 2024CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecastingcited by 6
  10. 2024Dataset Condensation with Latent Quantile Matchingcited by 6
  11. 2025Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillationcited by 5
  12. 2024Prioritize Alignment in Dataset Distillationcited by 4
19 more
  1. 2025GeoDM: Geometry-aware Distribution Matching for Dataset Distillationcited by 3
  2. 2026DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Modelscited by 3
  3. 2026Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Modelscited by 3
  4. 2026Multimodal Distribution Matching for Vision-Language Dataset Distillationcited by 3
  5. 2026UniPROT: Uniform Prototype Selection via Partial Optimal Transport with Submodular Guaranteescited by 2
  6. 2025Dataset Distillation as Pushforward Optimal Quantizationcited by 1
  7. 2025A Discrepancy-Based Perspective on Dataset Condensationcited by 1
  8. 2026Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptationcited by 1
  9. 2026Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Spacecited by 1
  10. 2026Condensing Large-Scale Datasets Directly with Minimal Information Losscited by 1
  11. 2025KD2M: A unifying framework for feature knowledge distillationcited by 0
  12. 2025Distill Video Datasets into Imagescited by 0
  13. 2026Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Taskscited by 0
  14. 2026Towards Principled Dataset Distillation: A Spectral Distribution Perspectivecited by 0
  15. 2026HFP-SAM: Hierarchical Frequency Prompted SAM for Efficient Marine Animal Segmentationcited by 0
  16. 2026Omnimodal Dataset Distillation via High-order Proxy Alignmentcited by 0
  17. 2026Soft Label Pruning and Quantization for Large-Scale Dataset Distillationcited by 0
  18. 2026Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillationcited by 0
  19. 2026EMS Coreset: An Efficient Expectation-Maximization Algorithm for Sinkhorn Coresetcited by 0