mlsanityData to 7 Oct 2026

Towards Robust Dataset Learning

arXiv 2211.10752 ↗First posted 19 Nov 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Adversarial training has been actively studied in recent computer vision research to improve the robustness of models. However, due to the huge computational cost of generating adversarial samples, adversarial training methods are often slow. In this paper, we study the problem of learning a robust dataset such that any classifier naturally trained on the dataset is adversarially robust. Such a dataset benefits the downstream tasks as natural training is much faster than adversarial training, and demonstrates that the desired property of robustness is transferable between models and data. In this work, we propose a principled, tri-level optimization to formulate the robust dataset learning problem. We show that, under an abstraction model that characterizes robust vs. non-robust features, the proposed method provably learns a robust dataset. Extensive experiments on MNIST, CIFAR10, and TinyImageNet demostrate the effectiveness of our algorithm with different network initializations and architectures.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 38 of its 48 references that are in the map. First posted on arXiv 19 Nov 2022.

Builds on

38 of 48 references.

References · 38

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  4. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  5. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  6. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  7. 2018Robustness May Be at Odds with Accuracycited by 873
  8. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  9. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  10. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  11. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  12. 2020Dataset Condensation with Gradient Matchingcited by 510
26 more
  1. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  2. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  3. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  4. 2017Mitigating Adversarial Effects Through Randomizationcited by 382
  5. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  6. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  7. 2018Certified Defenses against Adversarial Examplescited by 305
  8. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  9. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  10. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  11. 2018Adversarial Logit Pairingcited by 224
  12. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  13. 2018Certified Adversarial Robustness with Additive Noisecited by 178
  14. 2018Scaling provable adversarial defensescited by 141
  15. 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
  16. 2020Distilled One-Shot Federated Learningcited by 108
  17. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 106
  18. 2020Flexible Dataset Distillation: Learn Labels Instead of Imagescited by 91
  19. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  20. 2018Training for Faster Adversarial Robustness Verification via Inducing ReLU Stabilitycited by 67
  21. 2019Adversarial Defense by Restricting the Hidden Space of Deep Neural Networkscited by 56
  22. 2018Provable Robustness of ReLU networks via Maximization of Linear Regionscited by 51
  23. 2021Soft-Label Anonymous Gastric X-ray Image Distillationcited by 51
  24. 2020Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Frameworkcited by 45
  25. 2021Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neuronscited by 40
  26. 2020SecDD: Efficient and Secure Method for Remotely Training Neural Networkscited by 14

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 2.

9 papers.

  1. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  2. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  3. 2023A Survey on Dataset Distillation: Approaches, Applications and Future Directionscited by 32
  4. 2022Adversarial Weight Perturbation Improves Generalization in Graph Neural Networkscited by 21
  5. 2024Importance-Aware Adaptive Dataset Distillationcited by 18
  6. 2024Group Distributionally Robust Dataset Distillation with Risk Minimizationcited by 10
  7. 2024BEARD: Benchmarking the Adversarial Robustness for Dataset Distillationcited by 2
  8. 2025Robust Dataset Distillation by Matching Adversarial Trajectoriescited by 1
  9. 2025A Discrepancy-Based Perspective on Dataset Condensationcited by 1