# Benchmarking Adversarial Robustness

arXiv: https://arxiv.org/abs/1912.11852 · HTML: https://mlsanity.com/paper/1912.11852/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Dec 2019.

## Built on by (most cited first, 15 of 15)

- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks](https://mlsanity.com/paper/2107.01809/) (2021)
- [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart](https://mlsanity.com/paper/2105.14785/) (2021)
- [Accumulative Poisoning Attacks on Real-time Data](https://mlsanity.com/paper/2106.09993/) (2021)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Noise Optimization for Artificial Neural Networks](https://mlsanity.com/paper/2102.04450/) (2021)
- [Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators](https://mlsanity.com/paper/2302.13464/) (2023)
- [Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model](https://mlsanity.com/paper/2105.02803/) (2021)
- [Benchmarking Adversarial Robustness of Compressed Deep Learning Models](https://mlsanity.com/paper/2308.08160/) (2023)
- [Machine Translation Testing via Syntactic Tree Pruning](https://mlsanity.com/paper/2401.00751/) (2024)
- [Medical Aegis: Robust adversarial protectors for medical images](https://mlsanity.com/paper/2111.10969/) (2021)
- [Do we need entire training data for adversarial training?](https://mlsanity.com/paper/2303.06241/) (2023)
- [The Impact of Scaling Training Data on Adversarial Robustness](https://mlsanity.com/paper/2509.25927/) (2025)
- [Evaluating Model Robustness Using Adaptive Sparse L0 Regularization](https://mlsanity.com/paper/2408.15702/) (2024)
- [Revisiting the Relationship between Adversarial and Clean Training: Why Clean Training Can Make Adversarial Training Better](https://mlsanity.com/paper/2504.00038/) (2025)

## Abstract (arXiv)

Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning. While a lot of efforts have been made in recent years, it is of great significance to perform correct and complete evaluations of the adversarial attack and defense algorithms. In this paper, we establish a comprehensive, rigorous, and coherent benchmark to evaluate adversarial robustness on image classification tasks. After briefly reviewing plenty of representative attack and defense methods, we perform large-scale experiments with two robustness curves as the fair-minded evaluation criteria to fully understand the performance of these methods. Based on the evaluation results, we draw several important findings and provide insights for future research.