mlsanityData to 14 Oct 2026

Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching

arXiv 2311.17950 ↗First posted 29 Nov 2023cs.CV cs.AIcited by 69

Not on the map. Why?

Abstract (arXiv)

The lightweight "local-match-global" matching introduced by SRe2L successfully creates a distilled dataset with comprehensive information on the full 224x224 ImageNet-1k. However, this one-sided approach is limited to a particular backbone, layer, and statistics, which limits the improvement of the generalization of a distilled dataset. We suggest that sufficient and various "local-match-global" matching are more precise and effective than a single one and has the ability to create a distilled dataset with richer information and better generalization. We call this perspective "generalized matching" and propose Generalized Various Backbone and Statistical Matching (G-VBSM) in this work, which aims to create a synthetic dataset with densities, ensuring consistency with the complete dataset across various backbones, layers, and statistics. As experimentally demonstrated, G-VBSM is the first algorithm to obtain strong performance across both small-scale and large-scale datasets. Specifically, G-VBSM achieves a performance of 38.7% on CIFAR-100 with 128-width ConvNet, 47.6% on Tiny-ImageNet with ResNet18, and 31.4% on the full 224x224 ImageNet-1k with ResNet18, under images per class (IPC) 10, 50, and 10, respectively. These results surpass all SOTA methods by margins of 3.9%, 6.5%, and 10.1%, respectively.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

37 of 47 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2016Wide Residual Networkscited by 3,070
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  10. 2020Knowledge Distillation: A Surveycited by 1,603
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  12. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
25 more
  1. 2020Dataset Condensation with Gradient Matchingcited by 516
  2. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  3. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  4. 2022Decoupled Knowledge Distillationcited by 388
  5. 2021Dataset Condensation with Distribution Matchingcited by 365
  6. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  7. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  8. 2022Knowledge Distillation from A Stronger Teachercited by 208
  9. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 188
  10. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 181
  11. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  12. 2022Minimizing the Accumulated Trajectory Error to Improve Dataset Distillationcited by 142
  13. 2023Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspectivecited by 138
  14. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
  15. 2023Dataset Distillation: A Comprehensive Reviewcited by 127
  16. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  17. 2023DREAM: Efficient Dataset Distillation by Representative Matchingcited by 94
  18. 2023Data Distillation: A Surveycited by 87
  19. 2023DataDAM: Efficient Dataset Distillation with Attention Matchingcited by 84
  20. 2022Accelerating Dataset Distillation via Model Augmentationcited by 62
  21. 2021A Fast Knowledge Distillation Framework for Visual Recognitioncited by 50
  22. 2022FedSynth: Gradient Compression via Synthetic Data in Federated Learningcited by 31
  23. 2022Sample Condensation in Online Continual Learningcited by 31
  24. 2020Reducing catastrophic forgetting with learning on synthetic datacited by 26
  25. 2020Embedded Encoder-Decoder in Convolutional Networks Towards Explainable AIcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 15; 2025: 28; 2026: 23.

69 papers.

Sort:

By area:

  1. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 109
  2. 2025Dataset Distillation with Neural Characteristic Function: A Minmax Perspectivecited by 45
  3. 2024Elucidating the Design Space of Dataset Condensationcited by 40
  4. 2023Dataset Distillation via the Wasserstein Metriccited by 31
  5. 2024A Label is Worth a Thousand Images in Dataset Distillationcited by 31
  6. 2023Dataset Distillation via Curriculum Data Synthesis in Large Data Eracited by 26
  7. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
  8. 2024Unlocking Dataset Distillation with Diffusion Modelscited by 23
  9. 2024DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillationcited by 20
  10. 2024Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation?cited by 19
  11. 2025Taming Diffusion for Dataset Distillation with High Representativenesscited by 18
  12. 2025Hyperbolic Dataset Distillationcited by 15
38 more
  1. 2024Self-supervised Dataset Distillation: A Good Compression Is All You Needcited by 14
  2. 2025Dataset Distillation via Committee Votingcited by 14
  3. 2025CaO2: Rectifying Inconsistencies in Diffusion-Based Dataset Distillationcited by 14
  4. 2024Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensatorcited by 13
  5. 2025FADRM: Fast and Accurate Data Residual Matching for Dataset Distillationcited by 11
  6. 2025DD-Ranking: Rethinking the Evaluation of Dataset Distillationcited by 10
  7. 2024GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Costcited by 9
  8. 2024Distilling Long-tailed Datasetscited by 9
  9. 2026HIERAMP: Coarse-to-Fine Autoregressive Amplification for Generative Dataset Distillationcited by 9
  10. 2025Training-Free Dataset Pruning for Instance Segmentationcited by 7
  11. 2025Beyond Modality Collapse: Representations Blending for Multimodal Dataset Distillationcited by 7
  12. 2025MGD3: Mode-Guided Dataset Distillation using Diffusion Modelscited by 7
  13. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspectivecited by 7
  14. 2025Efficient Multimodal Dataset Distillation via Generative Modelscited by 7
  15. 2024One Category One Prompt: Dataset Distillation using Diffusion Modelscited by 6
  16. 2024Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networkscited by 6
  17. 2024A Large-Scale Study on Video Action Dataset Condensationcited by 6
  18. 2025Unifying Dataset Pruning and Distillation for Efficient Large-scale Compressioncited by 6
  19. 2025Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillationcited by 5
  20. 2024Heavy Labels Out! Dataset Distillation with Label Space Lighteningcited by 4
  21. 2026EVLF: Early Vision-Language Fusion for Generative Dataset Distillationcited by 4
  22. 2025GeoDM: Geometry-aware Distribution Matching for Dataset Distillationcited by 3
  23. 2026FD2: A Dedicated Framework for Fine-Grained Dataset Distillationcited by 3
  24. 2026Learnability-Guided Diffusion for Dataset Distillationcited by 3
  25. 2026DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Modelscited by 3
  26. 2024Efficiency for Free: Ideal Data Are Transportable Representationscited by 2
  27. 2025Understanding Dataset Distillation via Spectral Filteringcited by 2
  28. 2024Simple In-place Data Augmentation for Surveillance Object Detectioncited by 1
  29. 2025SCORE: Soft Label Compression-Centric Dataset Condensation via Coding Rate Optimizationcited by 1
  30. 2025Temporal Saliency-Guided Distillation: A Scalable Framework for Distilling Video Datasetscited by 1
  31. 2025Instance Data Condensation for Image Super-Resolutioncited by 1
  32. 2025Rectified Decoupled Dataset Distillation: A Closer Look for Fair and Comprehensive Evaluationcited by 1
  33. 2025Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillationcited by 1
  34. 2025DiRe: Diversity-promoting Regularization for Dataset Condensationcited by 1
  35. 2025Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Driftcited by 1
  36. 2026Grounding and Enhancing Informativeness and Utility in Dataset Distillationcited by 1
  37. 2026Multimodal Dataset Distillation via Phased Teacher Modelscited by 1
  38. 2026Rethinking Dataset Distillation: Hard Truths about Soft Labelscited by 1

50 most cited of 69 All 69 (JSON)