mlsanityData to 14 Oct 2026

Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective

arXiv 2306.13092 ↗First posted 22 Jun 2023cs.CV cs.AI cs.LGcited by 138

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a new dataset condensation framework termed Squeeze, Recover and Relabel (SRe2L) that decouples the bilevel optimization of model and synthetic data during training, to handle varying scales of datasets, model architectures and image resolutions for efficient dataset condensation. The proposed method demonstrates flexibility across diverse dataset scales and exhibits multiple advantages in terms of arbitrary resolutions of synthesized images, low training cost and memory consumption with high-resolution synthesis, and the ability to scale up to arbitrary evaluation network architectures. Extensive experiments are conducted on Tiny-ImageNet and full ImageNet-1K datasets. Under 50 IPC, our approach achieves the highest 42.5% and 60.8% validation accuracy on Tiny-ImageNet and ImageNet-1K, outperforming all previous state-of-the-art methods by margins of 14.5% and 32.9%, respectively. Our approach also surpasses MTT in terms of speed by approximately 52× (ConvNet-4) and 16× (ResNet-18) faster with less memory consumption of 11.6× and 6.4× during data synthesis. Our code and condensed datasets of 50, 200 IPC with 4K recovery budget are available at https://github.com/VILA-Lab/SRe2L.

Key facts. Cited by 138 papers in the map (data through 14 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 22 Jun 2023.

Builds on

34 of 44 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  7. 2019Contrastive Multiview Codingcited by 1,301
  8. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  9. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  10. 2019On Variational Bounds of Mutual Informationcited by 559
  11. 2020Dataset Condensation with Gradient Matchingcited by 516
  12. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
22 more
  1. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 431
  2. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  3. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  4. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  5. 2021Dataset Condensation with Distribution Matchingcited by 365
  6. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 291
  7. 2018Dataset Distillationcited by 230
  8. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  9. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 188
  10. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 181
  11. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  12. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  13. 2022Dataset Distillation via Factorizationcited by 115
  14. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  15. 2022Dataset Condensation with Contrastive Signalscited by 105
  16. 2022Remember the Past: Distilling Datasets into Addressable Memories for Neural Networkscited by 104
  17. 2023Data Distillation: A Surveycited by 87
  18. 2017Practical Coreset Constructions for Machine Learningcited by 78
  19. 2022Synthesizing Informative Training Samples with GANcited by 77
  20. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  21. 2021A Fast Knowledge Distillation Framework for Visual Recognitioncited by 50
  22. 2022Bounding generalization error with input compression: An empirical study with infinite-width networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 43; 2025: 53; 2026: 36.

138 papers.

Sort:

By area:

  1. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 109
  2. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 83
  3. 2024D4M: Dataset Distillation via Disentangled Diffusion Modelcited by 77
  4. 2023Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matchingcited by 69
  5. 2025Dataset Distillation with Neural Characteristic Function: A Minmax Perspectivecited by 45
  6. 2024Elucidating the Design Space of Dataset Condensationcited by 40
  7. 2024Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustmentcited by 33
  8. 2023Dataset Distillation via the Wasserstein Metriccited by 31
  9. 2024A Label is Worth a Thousand Images in Dataset Distillationcited by 31
  10. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  11. 2023Dataset Distillation via Curriculum Data Synthesis in Large Data Eracited by 26
  12. 2024Dataset Distillation by Automatic Training Trajectoriescited by 26
38 more
  1. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
  2. 2024Unlocking Dataset Distillation with Diffusion Modelscited by 23
  3. 2024SelMatch: Effectively Scaling Up Dataset Distillation via Selection-Based Initialization and Partial Updates by Trajectory Matchingcited by 22
  4. 2024DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillationcited by 20
  5. 2024Multisize Dataset Condensationcited by 19
  6. 2024Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation?cited by 19
  7. 2025Dataset Distillation via Vision-Language Category Prototypecited by 19
  8. 2024Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Imagescited by 18
  9. 2024Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarioscited by 18
  10. 2025Taming Diffusion for Dataset Distillation with High Representativenesscited by 18
  11. 2024Towards Adversarially Robust Dataset Distillation by Curvature Regularizationcited by 17
  12. 2024Low-Rank Similarity Mining for Multimodal Dataset Distillationcited by 17
  13. 2025Hyperbolic Dataset Distillationcited by 15
  14. 2025Dataset Distillation via Committee Votingcited by 14
  15. 2025CaO2: Rectifying Inconsistencies in Diffusion-Based Dataset Distillationcited by 14
  16. 2024What Variables Affect Out-of-Distribution Generalization in Pretrained Models?cited by 13
  17. 2024Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillationcited by 13
  18. 2024Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensatorcited by 13
  19. 2024Generative Dataset Distillation Based on Diffusion Modelcited by 13
  20. 2024Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matchingcited by 13
  21. 2024One-shot Federated Learning via Synthetic Distiller-Distillate Communicationcited by 12
  22. 2024Distilling the Knowledge in Data Pruningcited by 11
  23. 2024FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Modelscited by 11
  24. 2025FADRM: Fast and Accurate Data Residual Matching for Dataset Distillationcited by 11
  25. 2024CemiFace: Center-based Semi-hard Synthetic Face Generation for Face Recognitioncited by 10
  26. 2025DD-Ranking: Rethinking the Evaluation of Dataset Distillationcited by 10
  27. 2025Information-Guided Diffusion Sampling for Dataset Distillationcited by 10
  28. 2023Dataset Distillation via Adversarial Prediction Matchingcited by 9
  29. 2024GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Costcited by 9
  30. 2024Distilling Long-tailed Datasetscited by 9
  31. 2025Enhancing Dataset Distillation via Non-Critical Region Refinementcited by 9
  32. 2024DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillationcited by 8
  33. 2024Improve Cross-Architecture Generalization on Dataset Distillationcited by 7
  34. 2024Distributional Dataset Distillation with Subtask Decompositioncited by 7
  35. 2025Training-Free Dataset Pruning for Instance Segmentationcited by 7
  36. 2025Beyond Modality Collapse: Representations Blending for Multimodal Dataset Distillationcited by 7
  37. 2025MGD3: Mode-Guided Dataset Distillation using Diffusion Modelscited by 7
  38. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspectivecited by 7

50 most cited of 138 All 138 (JSON)