mlsanityData to 7 Oct 2026

Distributionally and Adversarially Robust Logistic Regression via Intersecting Wasserstein Balls

arXiv 2407.13625 ↗First posted 18 Jul 2024math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarially robust optimization (ARO) has emerged as the *de facto* standard for training models that hedge against adversarial attacks in the test stage. While these models are robust against adversarial attacks, they tend to suffer severely from overfitting. To address this issue, some successful methods replace the empirical distribution in the training stage with alternatives including *(i)* a worst-case distribution residing in an ambiguity set, resulting in a distributionally robust (DR) counterpart of ARO; *(ii)* a mixture of the empirical distribution with a distribution induced by an auxiliary (*e.g.*, synthetic, external, out-of-domain) dataset. Inspired by the former, we study the Wasserstein DR counterpart of ARO for logistic regression and show it admits a tractable convex optimization reformulation. Adopting the latter setting, we revise the DR approach by intersecting its ambiguity set with another ambiguity set built using the auxiliary dataset, which offers a significant improvement whenever the Wasserstein distance between the data generating and auxiliary distributions can be estimated. We study the underlying optimization problem, develop efficient solution algorithms, and demonstrate that the proposed method outperforms benchmark approaches on standard datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 92 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

46 of 92 references.

References · 46

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  6. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  7. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  8. 2019Adversarial Training for Free!cited by 638
  9. 2020Overfitting in adversarially robust deep learningcited by 504
  10. 2019On Evaluating Adversarial Robustnesscited by 442
  11. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  12. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
34 more
  1. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  2. 2021Improving Robustness using Generated Datacited by 233
  3. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  4. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  5. 2019Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learningcited by 169
  6. 2019Adversarial Training Can Hurt Generalizationcited by 131
  7. 2015Distributionally Robust Logistic Regressioncited by 124
  8. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  9. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
  10. 2017Regularization via Mass Transportationcited by 79
  11. 2019Adversarially robust transfer learningcited by 66
  12. 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
  13. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 50
  14. 2015JuMP: A Modeling Language for Mathematical Optimizationcited by 49
  15. 2020Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionalitycited by 49
  16. 2018Improving the Generalization of Adversarial Training with Domain Adaptationcited by 48
  17. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  18. 2022The Many Faces of Adversarial Riskcited by 28
  19. 2020On Linear Optimization over Wasserstein Ballscited by 23
  20. 2020Improving Adversarial Robustness via Unlabeled Out-of-Domain Datacited by 23
  21. 2022Understanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularizationcited by 22
  22. 2022Holistic Robust Data-Driven Decisionscited by 17
  23. 2015A Practical Guide to Robust Optimizationcited by 16
  24. 2022Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classificationcited by 15
  25. 2021Sequential Domain Adaptation by Synthesizing Distributionally Robust Expertscited by 14
  26. 2022Global-Local Regularization Via Distributional Robustnesscited by 11
  27. 2023Certified Robust Neural Networks: Generalization and Corruption Resistancecited by 11
  28. 2024Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularizationcited by 11
  29. 2024Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependencycited by 9
  30. 2017K-Adaptability in Two-Stage Mixed-Integer Robust Optimizationcited by 6
  31. 2022Wasserstein Logistic Regression with Mixed Featurescited by 5
  32. 2022Unlabeled Data Help: Minimax Analysis and Adversarial Robustnesscited by 4
  33. 2022Distributionally Robust Data Joincited by 2
  34. 2024Contextual Optimization under Covariate Shift: A Robust Approach by Intersecting Wasserstein Ballscited by 2

Built on by

Not cited yet.