mlsanityData to 14 Oct 2026

Outlier Robust Adversarial Training

arXiv 2309.05145 ↗First posted 10 Sep 2023cs.LG cs.AI stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Supervised learning models are challenged by the intrinsic complexities of training data such as outliers and minority subpopulations and intentional attacks at inference time with adversarial samples. While traditional robust learning methods and the recent adversarial training approaches are designed to handle each of the two challenges, to date, no work has been done to develop models that are robust with regard to the low-quality training data and the potential adversarial attack at inference time simultaneously. It is for this reason that we introduce Outlier Robust Adversarial Training (ORAT) in this work. ORAT is based on a bi-level optimization formulation of adversarial training with a robust rank-based loss function. Theoretically, we show that the learning objective of ORAT satisfies the H-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss. Furthermore, we analyze its generalization ability and provide uniform convergence rates in high probability. ORAT can be optimized with a simple algorithm. Experimental evaluations on three benchmark datasets demonstrate the effectiveness and robustness of ORAT in handling outliers and adversarial attacks. Our code is available at https://github.com/discovershu/ORAT.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 39 of its 67 references that are in the map. First posted on arXiv 10 Sep 2023.

Builds on

39 of 67 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  7. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  8. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  9. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  10. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  11. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
  12. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 316
27 more
  1. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  2. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  3. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  4. 2019Deep Self-Learning From Noisy Labelscited by 122
  5. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  6. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  7. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  8. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  9. 2022Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settingscited by 66
  10. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  11. 2021DORO: Distributional and Outlier Robust Optimizationcited by 56
  12. 2021Exploring Memorization in Adversarial Trainingcited by 55
  13. 2020Adversarial Learning Guarantees for Linear Hypotheses and Neural Networkscited by 47
  14. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  15. 2020How benign is benign overfitting?cited by 40
  16. 2023Improving Fairness in Deepfake Detectioncited by 40
  17. 2020Training Binary Neural Networks through Learning with Noisy Supervisioncited by 39
  18. 2020Calibrated Surrogate Losses for Adversarially Robust Classificationcited by 36
  19. 2019Defending against Whitebox Adversarial Attacks via Randomized Discretizationcited by 29
  20. 2021Robust Attentive Deep Neural Network for Exposing GAN-generated Facescited by 28
  21. 2020Learning by Minimizing the Sum of Ranked Rangecited by 27
  22. 2021TkML-AP: Adversarial Attacks to Top-k Multi-Label Learningcited by 26
  23. 2022Rank-based Decomposable Losses in Machine Learning: A Surveycited by 26
  24. 2021Sum of Ranked Range Loss for Supervised Learningcited by 23
  25. 2021Understanding the Interaction of Adversarial Training with Noisy Labelscited by 19
  26. 2022Distributionally Robust Survival Analysis: A Novel Fairness Loss Without Demographicscited by 15
  27. 2020Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attackscited by 14

Built on by

Cited by year

Citing papers per year: 2024: 8; 2025: 2.

10 papers.

  1. 2024Preserving Fairness Generalization in Deepfake Detectioncited by 70
  2. 2024Robust CLIP-Based Detector for Exposing Diffusion Model-Generated Imagescited by 20
  3. 2024Robust Light-Weight Facial Affective Behavior Recognition with CLIPcited by 16
  4. 2024Robust COVID-19 Detection in CT Images with CLIPcited by 12
  5. 2024Robustly Optimized Deep Feature Decoupling Network for Fatty Liver Diseases Detectioncited by 11
  6. 2024Uncertainty-Aware Explainable Recommendation with Large Language Modelscited by 10
  7. 2025Improving Generalization of Medical Image Registration Foundation Modelcited by 6
  8. 2025Robust AI-Generated Face Detection with Imbalanced Datacited by 5
  9. 2024Combining Machine Learning Defenses without Conflictscited by 2
  10. 2024Iterative thresholding for non-linear learning in the strong varepsilon-contamination modelcited by 0