mlsanityData to 7 Oct 2026

The Limitations of Deep Learning in Adversarial Settings

arXiv 1511.07528 ↗First posted 24 Nov 2015cs.CR cs.LG cs.NEcited by 784

Not on the map. Why?

Cited by 24 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning takes advantage of large datasets and computationally efficient training algorithms to outperform other approaches at various machine learning tasks. However, imperfections in the training phase of deep neural networks make them vulnerable to adversarial samples: inputs crafted by adversaries with the intent of causing deep neural networks to misclassify. In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs. In an application to computer vision, we show that our algorithms can reliably produce samples correctly classified by human subjects but misclassified in specific targets by a DNN with a 97% adversarial success rate while only modifying on average 4.02% of the input features per sample. We then evaluate the vulnerability of different sample classes to adversarial perturbations by defining a hardness measure. Finally, we describe preliminary work outlining defenses against adversarial samples by defining a predictive measure of distance between a benign input and a target classification.

Key facts. Cited by 784 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 223; 2022: 173; 2023: 145; 2024: 122; 2025: 81; 2026: 40.

784 papers.

Sort:

By area:

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  3. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  4. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  5. 2022Zero-Shot Machine Unlearningcited by 125
  6. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  7. 2021Globally-Robust Neural Networkscited by 106
  8. 2021Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attackscited by 101
  9. 2022Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenoncited by 101
  10. 2022Adversarial Texture for Fooling Person Detectors in the Physical Worldcited by 96
  11. 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91
  12. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
38 more
  1. 2022The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Trainingcited by 85
  2. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
  3. 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
  4. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  5. 2022Frequency-driven Imperceptible Adversarial Attack on Semantic Similaritycited by 76
  6. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  7. 2021Practical Blind Membership Inference Attack via Differential Comparisonscited by 73
  8. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  9. 2023BlackVIP: Black-Box Visual Prompting for Robust Transfer Learningcited by 70
  10. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  11. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  12. 2021PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifiercited by 63
  13. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 58
  14. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  15. 2022Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patchescited by 57
  16. 2021Boosting Adversarial Transferability through Enhanced Momentumcited by 53
  17. 2022Robust Semantic Communications with Masked VQ-VAE Enabled Codebookcited by 53
  18. 2023StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learningcited by 53
  19. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusalcited by 53
  20. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
  21. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  22. 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
  23. 2023Prompt Stealing Attacks Against Text-to-Image Generation Modelscited by 48
  24. 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
  25. 2022Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learningcited by 47
  26. 2022Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?cited by 47
  27. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 47
  28. 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
  29. 2021Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Datacited by 45
  30. 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
  31. 20213D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detectioncited by 43
  32. 2021Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertaintiescited by 41
  33. 2023Physically Realizable Natural-Looking Clothing Textures Evade Person Detectors via 3D Modelingcited by 41
  34. 2023DAP: A Dynamic Adversarial Patch for Evading Person Detectorscited by 40
  35. 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 40
  36. 2021WaveGuard: Understanding and Mitigating Audio Adversarial Examplescited by 39
  37. 2021Are Vision Transformers Robust to Patch Perturbations?cited by 39
  38. 2022How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspectivecited by 39

50 most cited of 784 All 784 (JSON)