# Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attacks

arXiv: https://arxiv.org/abs/2105.10304 · HTML: https://mlsanity.com/paper/2105.10304/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 57 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 30 of 37)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://mlsanity.com/paper/2010.03593/) (2020)
- [Attacks Which Do Not Kill Training Make Adversarial Learning Stronger](https://mlsanity.com/paper/2002.11242/) (2020)
- [Adversarial Risk and the Dangers of Evaluating Against Weak Attacks](https://mlsanity.com/paper/1802.05666/) (2018)
- [Geometry-aware Instance-reweighted Adversarial Training](https://mlsanity.com/paper/2010.01736/) (2020)
- [You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle](https://mlsanity.com/paper/1905.00877/) (2019)
- [MMA Training: Direct Input Space Margin Maximization through Adversarial Training](https://mlsanity.com/paper/1812.02637/) (2018)
- [Adding Gradient Noise Improves Learning for Very Deep Networks](https://mlsanity.com/paper/1511.06807/) (2015)
- [Self-Adaptive Training: beyond Empirical Risk Minimization](https://mlsanity.com/paper/2002.10319/) (2020)
- [Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition](https://mlsanity.com/paper/1903.10346/) (2019)
- [Boosting Adversarial Training with Hypersphere Embedding](https://mlsanity.com/paper/2002.08619/) (2020)
- [Learnable Boundary Guided Adversarial Training](https://mlsanity.com/paper/2011.11164/) (2020)
- [Adversarial Defense by Restricting the Hidden Space of Deep Neural Networks](https://mlsanity.com/paper/1904.00887/) (2019)
- [Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints](https://mlsanity.com/paper/2102.12827/) (2021)
- [Accurate, reliable and fast robustness evaluation](https://mlsanity.com/paper/1907.01003/) (2019)
- [Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data](https://mlsanity.com/paper/1805.12316/) (2018)
- [Efficient Robust Training via Backward Smoothing](https://mlsanity.com/paper/2010.01278/) (2020)

## Built on by (most cited first, 30 of 33)

- [Measure and Improve Robustness in NLP Models: A Survey](https://mlsanity.com/paper/2112.08313/) (2021)
- [Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space](https://mlsanity.com/paper/2402.09063/) (2024)
- [Adversarial Attacks and Defenses in Large Language Models: Old and New Threats](https://mlsanity.com/paper/2310.19737/) (2023)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Universal Adversarial Examples in Remote Sensing: Methodology and Benchmark](https://mlsanity.com/paper/2202.07054/) (2022)
- [Improving Robustness against Real-World and Worst-Case Distribution Shifts through Decision Region Quantification](https://mlsanity.com/paper/2205.09619/) (2022)
- [DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillation](https://mlsanity.com/paper/2403.13322/) (2024)
- [State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning](https://mlsanity.com/paper/2503.20613/) (2025)
- [Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attention](https://mlsanity.com/paper/2204.09093/) (2022)
- [Raising the Bar for Certified Adversarial Robustness with Diffusion Models](https://mlsanity.com/paper/2305.10388/) (2023)
- [Robust Deep Reinforcement Learning in Robotics via Adaptive Gradient-Masked Adversarial Attacks](https://mlsanity.com/paper/2503.20844/) (2025)
- [AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation](https://mlsanity.com/paper/2306.07197/) (2023)
- [Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks](https://mlsanity.com/paper/2505.08022/) (2025)
- [Universal Adversarial Defense in Remote Sensing Based on Pre-trained Denoising Diffusion Models](https://mlsanity.com/paper/2307.16865/) (2023)
- [Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning](https://mlsanity.com/paper/2505.13327/) (2025)
- [Byte Pair Encoding for Efficient Time Series Forecasting](https://mlsanity.com/paper/2505.14411/) (2025)
- [AFLOW: Developing Adversarial Examples under Extremely Noise-limited Settings](https://mlsanity.com/paper/2310.09795/) (2023)
- [Evaluating Adversarial Attacks on ImageNet: A Reality Check on Misclassification Classes](https://mlsanity.com/paper/2111.11056/) (2021)
- [Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors](https://mlsanity.com/paper/2410.02384/) (2024)
- [Adversarial Contrastive Learning by Permuting Cluster Assignments](https://mlsanity.com/paper/2204.10314/) (2022)
- [Towards Certified Probabilistic Robustness with High Accuracy](https://mlsanity.com/paper/2309.00879/) (2023)
- [FoolSDEdit: Deceptively Steering Your Edits Towards Targeted Attribute-aware Distribution](https://mlsanity.com/paper/2402.03705/) (2024)
- [Effective Data Pruning through Score Extrapolation](https://mlsanity.com/paper/2506.09010/) (2025)
- [Spatial-Frequency Discriminability for Revealing Adversarial Perturbations](https://mlsanity.com/paper/2305.10856/) (2023)
- [IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks](https://mlsanity.com/paper/2310.11890/) (2023)
- [How to Backdoor Image Knowledge Distillation](https://mlsanity.com/paper/2504.21323/) (2025)
- [Physically-Induced Atmospheric Adversarial Perturbations: Enhancing Transferability and Robustness in Remote Sensing Image Classification](https://mlsanity.com/paper/2604.14643/) (2026)
- [Robust Smart Home Face Recognition under Starving Federated Data](https://mlsanity.com/paper/2211.05410/) (2022)
- [AED-PADA:Improving Generalizability of Adversarial Example Detection via Principal Adversarial Domain Adaptation](https://mlsanity.com/paper/2404.12635/) (2024)
- [Active Fake: DeepFake Camouflage](https://mlsanity.com/paper/2409.03200/) (2024)

## Abstract (arXiv)

Progress in making neural networks more robust against adversarial attacks is mostly marginal, despite the great efforts of the research community. Moreover, the robustness evaluation is often imprecise, making it difficult to identify promising approaches. We analyze the classification decisions of 19 different state-of-the-art neural networks trained to be robust against adversarial attacks. Our findings suggest that current untargeted adversarial attacks induce misclassification towards only a limited amount of different classes. Additionally, we observe that both over- and under-confidence in model predictions result in an inaccurate assessment of model robustness. Based on these observations, we propose a novel loss function for adversarial attacks that consistently improves attack success rate compared to prior loss functions for 19 out of 19 analyzed models.