mlsanityData to 7 Oct 2026

Towards Evaluating the Robustness of Neural Networks

arXiv 1608.04644 ↗First posted 16 Aug 2016cs.CR cs.CVcited by 2,976

Not on the map. Why?

Cited by 122 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x' that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%. In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.

Key facts. Cited by 2,976 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 656; 2022: 591; 2023: 570; 2024: 509; 2025: 404; 2026: 246.

2,976 papers.

Sort:

By area:

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  3. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  4. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  5. 2021Unsolved Problems in ML Safetycited by 291
  6. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
  7. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  8. 2021Improving Robustness using Generated Datacited by 233
  9. 2021Trustworthy AI: From Principles to Practicescited by 206
  10. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  11. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  12. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
38 more
  1. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
  2. 2021Data Augmentation Can Improve Robustnesscited by 193
  3. 2021Unlearnable Examples: Making Personal Data Unexploitablecited by 188
  4. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  5. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  6. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
  7. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 163
  8. 2021Admix: Enhancing the Transferability of Adversarial Attackscited by 155
  9. 2022(Certified!!) Adversarial Robustness for Free!cited by 130
  10. 2021Graph Unlearningcited by 129
  11. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
  12. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  13. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  14. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  15. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  16. 2021Feature Importance-aware Transferable Adversarial Attackscited by 125
  17. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
  18. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  19. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  20. 2021Adversarial Examples Make Strong Poisonscited by 115
  21. 2023Anti-DreamBooth: Protecting users from personalized text-to-image synthesiscited by 114
  22. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
  23. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  24. 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Modelscited by 106
  25. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  26. 2021Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attackscited by 101
  27. 2022Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenoncited by 101
  28. 2021Robustness of Graph Neural Networks at Scalecited by 100
  29. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  30. 2022Adversarial Texture for Fooling Person Detectors in the Physical Worldcited by 96
  31. 2021Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognitioncited by 95
  32. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
  33. 2021Trustworthy AI: A Computational Perspectivecited by 93
  34. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 93
  35. 2021LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognitioncited by 92
  36. 2021Adversarial Sticker: A Stealthy Attack Method in the Physical Worldcited by 92
  37. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  38. 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91

50 most cited of 2,976 All 2,976 (JSON)