# Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networks

arXiv: https://arxiv.org/abs/2508.04925 · HTML: https://mlsanity.com/paper/2508.04925/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 84 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth](https://mlsanity.com/paper/2103.03404/) (2021)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Stabilizing Transformer Training by Preventing Attention Entropy Collapse](https://mlsanity.com/paper/2303.06296/) (2023)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [DeepLocalize: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/2103.03376/) (2021)
- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training](https://mlsanity.com/paper/2310.03294/) (2023)
- [Toward Understanding Deep Learning Framework Bugs](https://mlsanity.com/paper/2203.04026/) (2022)
- [Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow](https://mlsanity.com/paper/2112.13314/) (2021)
- [An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications](https://mlsanity.com/paper/2101.04930/) (2021)
- [DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs](https://mlsanity.com/paper/2112.04036/) (2021)
- [Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach](https://mlsanity.com/paper/2101.00135/) (2021)
- [DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs](https://mlsanity.com/paper/2205.01938/) (2022)
- [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022)
- [An Empirical Study of Obsolete Answers on Stack Overflow](https://mlsanity.com/paper/1903.12282/) (2019)
- [Understanding Performance Problems in Deep Learning Systems](https://mlsanity.com/paper/2112.01771/) (2021)
- [Bug Characterization in Machine Learning-based Systems](https://mlsanity.com/paper/2307.14512/) (2023)
- [Automatic Fault Detection for Deep Learning Programs Using Graph Transformations](https://mlsanity.com/paper/2105.08095/) (2021)
- [Demystifying Dependency Bugs in Deep Learning Stack](https://mlsanity.com/paper/2207.10347/) (2022)
- [From Today's Code to Tomorrow's Symphony: The AI Transformation of Developer's Routine by 2030](https://mlsanity.com/paper/2405.12731/) (2024)
- [An Empirical Study of C++ Vulnerabilities in Crowd-Sourced Code Examples](https://mlsanity.com/paper/1910.01321/) (2019)
- [How Different Are Different diff Algorithms in Git?](https://mlsanity.com/paper/1902.02467/) (2019)

## Built on by (most cited first, 1 of 1)

- [A Taxonomy of Runtime Faults in Model Context Protocol Servers](https://mlsanity.com/paper/2606.05339/) (2026)

## Abstract (arXiv)

Attention mechanisms are at the core of modern neural architectures, powering systems ranging from ChatGPT to autonomous vehicles and driving a major economic impact. However, high-profile failures, such as ChatGPT's nonsensical outputs or Google's suspension of Gemini's image generation due to attention weight errors, highlight a critical gap: existing deep learning fault taxonomies might not adequately capture the unique failures introduced by attention mechanisms. This gap leaves practitioners without actionable diagnostic guidance. To address this gap, we present the first comprehensive empirical study of faults in attention-based neural networks (ABNNs). Our work is based on a systematic analysis of 555 real-world faults collected from 96 projects across ten frameworks, including GitHub, Hugging Face, and Stack Overflow. Through our analysis, we develop a novel taxonomy comprising seven attention-specific fault categories, not captured by existing work. Our results show that over half of the ABNN faults arise from mechanisms unique to attention architectures. We further analyze the root causes and manifestations of these faults through various symptoms. Finally, by analyzing symptom-root cause associations, we identify four evidence-based diagnostic heuristics that explain 33.0% of attention-specific faults, offering the first systematic diagnostic guidance for attention-based models.