mlsanityData to 7 Oct 2026

Adversarial Risk and the Dangers of Evaluating Against Weak Attacks

arXiv 1802.05666 ↗First posted 15 Feb 2018cs.LG cs.CR stat.MLcited by 221

Not on the map. Why?

Abstract (arXiv)

This paper investigates recently proposed approaches for defending against adversarial examples and evaluating adversarial robustness. We motivate 'adversarial risk' as an objective for achieving models robust to worst-case inputs. We then frame commonly used attacks and evaluation metrics as defining a tractable surrogate objective to the true adversarial risk. This suggests that models may optimize this surrogate rather than the true adversarial risk. We formalize this notion as 'obscurity to an adversary,' and develop tools and heuristics for identifying obscured models and designing transparent models. We demonstrate that this is a significant problem in practice by repurposing gradient-free optimization techniques into adversarial attacks, which we use to decrease the accuracy of several recently proposed defenses to near zero. Our hope is that our formulations and results will help researchers to develop more powerful defenses.

Key facts. Cited by 221 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 57; 2022: 43; 2023: 49; 2024: 31; 2025: 25; 2026: 15.

221 papers.

Sort:

By area:

  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  2. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  3. 2021Improving Robustness using Generated Datacited by 233
  4. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  5. 2021Data Augmentation Can Improve Robustnesscited by 193
  6. 2021Adversarial purification with Score-based generative modelscited by 132
  7. 2022(Certified!!) Adversarial Robustness for Free!cited by 130
  8. 2021Feature Importance-aware Transferable Adversarial Attackscited by 125
  9. 2023Anti-DreamBooth: Protecting users from personalized text-to-image synthesiscited by 114
  10. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  11. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  12. 2022Guided Diffusion Model for Adversarial Purificationcited by 90
38 more
  1. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  2. 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
  3. 2023Boosting Adversarial Transferability by Block Shuffle and Rotationcited by 64
  4. 2023Structure Invariant Transformation for better Adversarial Transferabilitycited by 61
  5. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  6. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  7. 2021Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraintscited by 50
  8. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  9. 2021Improved Branch and Bound for Neural Network Verification via Lagrangian Decompositioncited by 44
  10. 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
  11. 2021A Comprehensive Evaluation Framework for Deep Model Robustnesscited by 41
  12. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  13. 2021Harnessing Perceptual Adversarial Patches for Crowd Countingcited by 39
  14. 2022Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Priorcited by 38
  15. 2023DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purificationcited by 36
  16. 2021Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attackscited by 33
  17. 2021A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attackscited by 33
  18. 2022The Many Faces of Adversarial Riskcited by 28
  19. 2021Combating Adversaries with Anti-Adversariescited by 27
  20. 2021Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attackscited by 24
  21. 2021Boosting the Certified Robustness of L-infinity Distance Netscited by 24
  22. 2024ADBM: Adversarial diffusion bridge model for reliable adversarial purificationcited by 24
  23. 2021Efficient Certified Defenses Against Patch Attacks on Image Classifierscited by 23
  24. 2023Expressive Losses for Verified Robustness via Convex Combinationscited by 23
  25. 2021Adaptive Image Transformations for Transfer-based Adversarial Attackcited by 22
  26. 2021Adversarial Training is Not Ready for Robot Learningcited by 20
  27. 2021Robustness and Generalization via Generative Adversarial Trainingcited by 20
  28. 2022Scaling Adversarial Training to Large Perturbation Boundscited by 20
  29. 2021Center Smoothing: Certified Robustness for Networks with Structured Outputscited by 18
  30. 2022Watermark Vaccine: Adversarial Attacks to Prevent Watermark Removalcited by 18
  31. 2022Recognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networkscited by 18
  32. 2022PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognitioncited by 16
  33. 2023Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challengescited by 16
  34. 2025LLM-Safety Evaluations Lack Robustnesscited by 16
  35. 2021Meta-Learning with Neural Tangent Kernelscited by 15
  36. 2021Certifiably Robust Variational Autoencoderscited by 14
  37. 2021Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfittingcited by 14
  38. 2023Understanding and Improving Ensemble Adversarial Defensecited by 14

50 most cited of 221 All 221 (JSON)