mlsanityData to 7 Oct 2026

Adversarial Attacks Leverage Interference Between Features in Superposition

arXiv 2510.11709 ↗First posted 13 Oct 2025cs.LG cs.AI cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Why do adversarial examples exist, and why do they transfer between models? Existing explanations appeal to high-dimensional geometry, non-robust patterns in the input, and decision boundary structure, but none provides a representation-level mechanism that explains why specific perturbations succeed and why attacks transfer between models. In this paper, we show that adversarial vulnerability can stem from efficient information encoding in neural networks. Specifically, vulnerability can arise from superposition - the phenomenon where networks represent more concepts than they have dimensions, forcing non-orthogonal representation and thus interference. This interference causes perturbations targeting one representation to affect others, creating vulnerabilities determined by interference patterns. In synthetic settings with precisely controlled superposition, we establish that superposition suffices to create adversarial vulnerability. The resulting attacks are predictable: PGD-discovered perturbations align with theoretically optimal perturbations derived from the interference geometry. Models trained on similar data develop similar interference patterns, explaining attack transferability. We then show that successful attacks on image classifiers exhibit the structure predicted by our proposed mechanism. These findings reveal that adversarial vulnerability can be a byproduct of networks' representational compression, complementing existing explanations based on data properties or architectural factors.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 61 of its 88 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

61 of 88 references.

References · 61

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2018On the Spectral Bias of Neural Networkscited by 1,321
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  11. 2022Toy Models of Superpositioncited by 928
  12. 2018Robustness May Be at Odds with Accuracycited by 873
38 more
  1. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  2. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  3. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  4. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  5. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  6. 2024Scaling and evaluating sparse autoencoderscited by 530
  7. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  8. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  9. 2018Adversarially Robust Generalization Requires More Datacited by 318
  10. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  11. 2018Certified Defenses against Adversarial Examplescited by 305
  12. 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 241
  13. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  14. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
  15. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  16. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  17. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  18. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  19. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  20. 2016Robustness of classifiers: from adversarial to random noisecited by 87
  21. 2022Polysemanticity and Capacity in Neural Networkscited by 74
  22. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptationcited by 66
  23. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  24. 2022Neural Networks Efficiently Learn Low-Dimensional Representations with SGDcited by 61
  25. 2019FDA: Feature Disruptive Attackcited by 55
  26. 2022Certified Training: Small Boxes are All You Needcited by 51
  27. 2022Rank Diminishing in Deep Neural Networkscited by 47
  28. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  29. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41
  30. 2025Grokking at the Edge of Numerical Stabilitycited by 41
  31. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
  32. 2024Mathematical Models of Computation in Superpositioncited by 34
  33. 2021A Frequency Perspective of Adversarial Robustnesscited by 28
  34. 2024Mechanistic?cited by 25
  35. 2023Expressive Losses for Verified Robustness via Convex Combinationscited by 23
  36. 2023A Geometric Notion of Causal Probingcited by 23
  37. 2022IBP Regularization for Verified Adversarial Robustness via Branch-and-Boundcited by 21
  38. 2023Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustnesscited by 20

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026From Data Statistics to Feature Geometry: How Correlations Shape Superpositioncited by 9
  2. 2026Subliminal Learning is a LoRA Artifactcited by 4
  3. 2026Expanding the Role of Diffusion Models for Robust Classifier Trainingcited by 0
  4. 2026High-probability guarantees for linear accessibility in feature superpositioncited by 0
  5. 2026World Modeling in Transformerscited by 0
  6. 2026Abstention and Noise Filtering: Two Missing Primitives of Softmax Attentioncited by 0
  7. 2026A Survey on the Linear Representation Hypothesiscited by 0