mlsanityData to 13 Oct 2026

Certifying Some Distributional Robustness with Principled Adversarial Training

arXiv 1710.10571 ↗First posted 29 Oct 2017stat.ML cs.LGcited by 336

Not on the map. Why?

Abstract (arXiv)

Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms. We address this problem through the principled lens of distributionally robust optimization, which guarantees performance under adversarial input perturbations. By considering a Lagrangian penalty formulation of perturbing the underlying data distribution in a Wasserstein ball, we provide a training procedure that augments model parameter updates with worst-case perturbations of training data. For smooth losses, our procedure provably achieves moderate levels of robustness with little computational or statistical cost relative to empirical risk minimization. Furthermore, our statistical guarantees allow us to efficiently certify robustness for the population loss. For imperceptible perturbations, our method matches or outperforms heuristic approaches.

Key facts. Cited by 336 papers in the map (data through 13 Oct 2026). First posted on arXiv 29 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2020: 1; 2021: 75; 2022: 68; 2023: 62; 2024: 50; 2025: 41; 2026: 39.

336 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,889
  2. 2021Domain Generalization: A Surveycited by 760
  3. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 432
  4. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  5. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  6. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 92
  7. 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
  8. 2021Heterogeneous Risk Minimizationcited by 90
  9. 2021Adversarially Adaptive Normalization for Single Domain Generalizationcited by 85
  10. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  11. 2021Cross-Domain Few-Shot Classification via Adversarial Task Augmentationcited by 63
  12. 2024Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimizationcited by 53
38 more
  1. 2021The Complexity of Nonconvex-Strongly-Concave Minimax Optimizationcited by 45
  2. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  3. 2021Distributionally Robust Learningcited by 44
  4. 2021Constrained Learning with Non-Convex Lossescited by 42
  5. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 42
  6. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  7. 2023Out-of-distribution Detection with Implicit Outlier Transformationcited by 42
  8. 2021Uncertainty-guided Model Generalization to Unseen Domainscited by 41
  9. 2021On the Certified Robustness for Ensemble Models and Beyondcited by 40
  10. 2021Adversarial Robustness with Semi-Infinite Constrained Learningcited by 39
  11. 2023On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysiscited by 39
  12. 2021TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothnesscited by 38
  13. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 38
  14. 2022Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shiftcited by 38
  15. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  16. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  17. 2022Adversarial Feature Augmentation for Cross-domain Few-shot Classificationcited by 36
  18. 2021Quantifying and Improving Transferability in Domain Generalizationcited by 35
  19. 2023Anomaly Detection under Distribution Shiftcited by 35
  20. 2023Mind the Label Shift of Augmentation-based Graph OOD Generalizationcited by 34
  21. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 34
  22. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  23. 2022Learning to Solve Routing Problems via Distributionally Robust Optimizationcited by 31
  24. 2022Grounding Visual Representations with Texts for Domain Generalizationcited by 30
  25. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
  26. 2021Modeling the Second Player in Distributionally Robust Optimizationcited by 29
  27. 2023Certified Minimax Unlearning with Generalization Rates and Deletion Capacitycited by 29
  28. 2022The Many Faces of Adversarial Riskcited by 28
  29. 2021Stochastic Bias-Reduced Gradient Methodscited by 25
  30. 2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 23
  31. 2022Differentially Private SGDA for Minimax Problemscited by 23
  32. 2021Mixed Nash Equilibria in the Adversarial Examples Gamecited by 22
  33. 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoningcited by 22
  34. 2021P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Promptscited by 22
  35. 2021Robust Models Are More Interpretable Because Attributions Look Normalcited by 21
  36. 2021Kernelized Heterogeneous Risk Minimizationcited by 21
  37. 2023Understanding Contrastive Learning via Distributionally Robust Optimizationcited by 21
  38. 2023Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learningcited by 20

50 most cited of 336 All 336 (JSON)