mlsanityData to 7 Oct 2026

On Evaluating Adversarial Robustness

arXiv 1902.06705 ↗First posted 18 Feb 2019cs.LG cs.CR stat.MLcited by 442

Not on the map. Why?

Cited by 31 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that withstand adaptive attacks, few have succeeded; most papers that propose defenses are quickly shown to be incorrect. We believe a large contributing factor is the difficulty of performing security evaluations. In this paper, we discuss the methodological foundations, review commonly accepted best practices, and suggest new methods for evaluating defenses to adversarial examples. We hope that both researchers developing defenses as well as readers and reviewers who wish to understand the completeness of an evaluation consider our advice in order to avoid common pitfalls.

Key facts. Cited by 442 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 103; 2022: 86; 2023: 82; 2024: 69; 2025: 58; 2026: 44.

442 papers.

Sort:

By area:

  1. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  2. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  3. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  4. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  5. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  6. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  7. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
  8. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  9. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  10. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  11. 2022On Adversarial Robustness of Trajectory Prediction for Autonomous Vehiclescited by 89
  12. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
38 more
  1. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  2. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  3. 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
  4. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  5. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  6. 2022Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptionscited by 78
  7. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacycited by 78
  8. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  9. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Imagescited by 69
  10. 2024Do Unlearning Methods Remove Information from Language Model Weights?cited by 66
  11. 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
  12. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  13. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  14. 2021Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraintscited by 50
  15. 2021Poison Ink: Robust and Invisible Backdoor Attackcited by 50
  16. 2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49
  17. 2023Adversarial Attacks and Defenses in Large Language Models: Old and New Threatscited by 49
  18. 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
  19. 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
  20. 2021Random Noise Defense Against Query-Based Black-Box Attackscited by 46
  21. 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 46
  22. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  23. 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
  24. 2021Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learningcited by 44
  25. 2021Constrained Learning with Non-Convex Lossescited by 42
  26. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  27. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  28. 2021A Comprehensive Evaluation Framework for Deep Model Robustnesscited by 41
  29. 2021WaveGuard: Understanding and Mitigating Audio Adversarial Examplescited by 39
  30. 2021Adversarial Robustness with Semi-Infinite Constrained Learningcited by 39
  31. 2022Certified Neural Network Watermarks with Randomized Smoothingcited by 39
  32. 2023Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compressioncited by 39
  33. 2021TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothnesscited by 37
  34. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 37
  35. 2023Improving the Transferability of Adversarial Samples by Path-Augmented Methodcited by 36
  36. 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 36
  37. 2021FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspectivecited by 35
  38. 2021Dual-Key Multimodal Backdoors for Visual Question Answeringcited by 35

50 most cited of 442 All 442 (JSON)