mlsanityData to 14 Oct 2026

Adversarial Examples Are Not Bugs, They Are Superposition

arXiv 2508.17456 ↗First posted 24 Aug 2025cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Adversarial examples -- inputs with imperceptible perturbations that fool neural networks -- remain one of deep learning's most perplexing phenomena despite nearly a decade of research. While numerous defenses and explanations have been proposed, there is no consensus on the fundamental mechanism. One underexplored hypothesis is that superposition, a concept from mechanistic interpretability, may be a major contributing factor, or even the primary cause. We present four lines of evidence in support of this hypothesis, greatly extending prior arguments by Elhage et al. (2022): (1) superposition can theoretically explain a range of adversarial phenomena, (2) in toy models, intervening on superposition controls robustness, (3) in toy models, intervening on robustness (via adversarial training) controls superposition, and (4) in ResNet18, intervening on robustness (via adversarial training) controls superposition.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 24 Aug 2025.

Builds on

41 of 51 references.

References · 41

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  6. 2016Adversarial examples in the physical worldcited by 1,769
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  10. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  12. 2022Toy Models of Superpositioncited by 971
29 more
  1. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  2. 2018Robustness May Be at Odds with Accuracycited by 880
  3. 2016Universal adversarial perturbationscited by 784
  4. 2019Adversarial Training for Free!cited by 643
  5. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  6. 2020Overfitting in adversarially robust deep learningcited by 506
  7. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  8. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  9. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
  10. 2018Adversarially Robust Generalization Requires More Datacited by 318
  11. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 316
  12. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  13. 2018Are adversarial examples inevitable?cited by 113
  14. 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
  15. 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 77
  16. 2019Model Compression with Adversarial Robustness: A Unified Optimization Frameworkcited by 71
  17. 2019On the Effectiveness of Low Frequency Perturbationscited by 56
  18. 2018The Curse of Concentration in Robust Learning: Evasion and Poisoning Attacks from Concentration of Measurecited by 46
  19. 2018On the Geometry of Adversarial Examplescited by 43
  20. 2018Understanding and Enhancing the Transferability of Adversarial Examplescited by 41
  21. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41
  22. 2024On the Complexity of Neural Computation in Superpositioncited by 21
  23. 2023Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustnesscited by 20
  24. 2019Empirically Measuring Concentration: Fundamental Limits on Intrinsic Robustnesscited by 12
  25. 2022Understanding Adversarial Robustness Against On-manifold Adversarial Examplescited by 12
  26. 2024A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offscited by 7
  27. 2024Exploring the Interplay of Interpretability and Robustness in Deep Neural Networks: A Saliency-guided Approachcited by 7
  28. 2019An Information-Theoretic Explanation for the Adversarial Fragility of AI Classifierscited by 5
  29. 2020Derivation of Information-Theoretically Optimal Adversarial Attacks with Applications to Robust Machine Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 10.

13 papers.

Sort:

By area:

  1. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  2. 2026From Data Statistics to Feature Geometry: How Correlations Shape Superpositioncited by 10
  3. 2025Adversarial Attacks Leverage Interference Between Features in Superpositioncited by 8
  4. 2026Subliminal Learning is a LoRA Artifactcited by 6
  5. 2026Superposition Without Interference? Towards Isolated Interventions via Almost Orthogonal Features in Language Modelscited by 2
  6. 2026Backdoor Directions in Vision Transformerscited by 1
  7. 2025Features Emerge as Discrete States: The First Application of SAEs to 3D Representationscited by 0
  8. 2026Expanding the Role of Diffusion Models for Robust Classifier Trainingcited by 0
  9. 2026Adversarial Concept Search: Predicting Compositional Errors From Feature Geometrycited by 0
  10. 2026Why Does Robustness Reduce Superposition?cited by 0
  11. 2026Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representationscited by 0
  12. 2026Interference Beyond Geometry in Concept Extractioncited by 0
1 more
  1. 2026Representational Simplicity and Circuit Size Dissociate in a Threshold-Dependent Way: A Controlled Test via Adversarial Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Understanding Robustness in Teacher-Student Setting: A New Perspective
  2. 2026Feature Superposition in Neural Networks: From Theory to Practice
  3. 2023Adversarial Examples Are Not Real Features
  4. 2025Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability
  5. 2024CausAdv: A Causal-based Framework for Detecting Adversarial Examples
  6. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning
  7. 2021Adversarial Examples Make Strong Poisons
  8. 2026Effects of sparsity and superposition on loss in simple autoencoders