mlsanityData to 7 Oct 2026

Transformers need glasses! Information over-squashing in language tasks

arXiv 2406.04267 ↗First posted 6 Jun 2024cs.CL cs.LGcited by 87

Not on the map. Why?

Abstract (arXiv)

We study how information propagates in decoder-only Transformers, which are the architectural backbone of most existing frontier large language models (LLMs). We rely on a theoretical signal propagation analysis -- specifically, we analyse the representations of the last token in the final layer of the Transformer, as this is the representation used for next-token prediction. Our analysis reveals a representational collapse phenomenon: we prove that certain distinct sequences of inputs to the Transformer can yield arbitrarily close representations in the final token. This effect is exacerbated by the low-precision floating-point formats frequently used in modern LLMs. As a result, the model is provably unable to respond to these sequences in different ways -- leading to errors in, e.g., tasks involving counting or copying. Further, we show that decoder-only Transformer language models can lose sensitivity to specific tokens in the input, which relates to the well-known phenomenon of over-squashing in graph neural networks. We provide empirical evidence supporting our claims on contemporary LLMs. Our theory also points to simple solutions towards ameliorating these issues.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

20 of 37 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  9. 2020On Layer Normalization in the Transformer Architecturecited by 906
  10. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  11. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  12. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
8 more
  1. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 319
  2. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 286
  3. 2021Thinking Like Transformerscited by 182
  4. 2023On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topologycited by 163
  5. 2023Stabilizing Transformer Training by Preventing Attention Entropy Collapsecited by 154
  6. 2023Locality-Aware Graph-Rewiring in GNNscited by 55
  7. 2023How does over-squashing affect the power of GNNs?cited by 53
  8. 2023Asynchronous Algorithmic Alignment with Cocyclescited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 39; 2026: 35.

87 papers.

Sort:

By area:

  1. 2025Why do LLMs attend to the first token?cited by 122
  2. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  3. 2025On the Emergence of Position Bias in Transformerscited by 57
  4. 2025Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coincited by 47
  5. 2024Softmax is not Enough (for Sharp Size Generalisation)cited by 42
  6. 2025On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learningcited by 42
  7. 2025Hallucination Detection in LLMs Using Spectral Features of Attention Mapscited by 42
  8. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  9. 2025Don't Blind Your VLA: Aligning Visual Representations for OOD Generalizationcited by 34
  10. 2025Inference-Time Hyper-Scaling with KV Cache Compressioncited by 28
  11. 2024When Can Transformers Count to n?cited by 22
  12. 2025Position: The Future of Bayesian Prediction Is Prior-Fittedcited by 19
38 more
  1. 2025LAMD: Context-driven Android Malware Detection and Classification with LLMscited by 18
  2. 2025Long-Context Generalization with Sparse Attentioncited by 18
  3. 2025Attention Sinks in Diffusion Language Modelscited by 18
  4. 2025Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddingscited by 16
  5. 2025Interpreting the Repeated Token Phenomenon in Large Language Modelscited by 15
  6. 2025Towards Scalable Language-Image Pre-training for 3D Medical Imagingcited by 13
  7. 2025Long Context In-Context Compression by Getting to the Gist of Gistingcited by 12
  8. 2025Beyond Transcription: Mechanistic Interpretability in ASRcited by 12
  9. 2025DGP: A Dual-Granularity Prompting Framework for Fraud Detection with Graph-Enhanced LLMscited by 11
  10. 2025Attention Consistency for LLMs Explanationcited by 10
  11. 2024Words in Motion: Extracting Interpretable Control Vectors for Motion Transformerscited by 9
  12. 2024Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoningcited by 9
  13. 2024More Expressive Attention with Negative Weightscited by 9
  14. 2024Fine-Tuning Large Language Models for Stock Return Prediction Using Newsflowcited by 8
  15. 2024Mitigating Propensity Bias of Large Language Models for Recommender Systemscited by 8
  16. 2026Perplexity Cannot Always Tell Right from Wrongcited by 8
  17. 2025Masks Can Be Distracting: On Context Comprehension in Diffusion Language Modelscited by 7
  18. 2026The Illusion of Stochasticity in LLMscited by 7
  19. 2024Bayes' Power for Explaining In-Context Learning Generalizationscited by 6
  20. 2024Best of Both Worlds: Advantages of Hybrid Graph Sequence Modelscited by 6
  21. 2025Over-squashing in Spatiotemporal Graph Neural Networkscited by 6
  22. 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAGcited by 6
  23. 2026Spectral Attention Steering for Prompt Highlightingcited by 6
  24. 2024Smart Vision-Language Reasonerscited by 5
  25. 2025Scale-invariant Attentioncited by 5
  26. 2025What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformerscited by 5
  27. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 5
  28. 2024Quantum Hamiltonian Embedding of Images for Data Reuploading Classifierscited by 4
  29. 2025You Do Not Fully Utilize Transformer's Representation Capacitycited by 4
  30. 2025Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compressioncited by 4
  31. 2025Training Transformers for Mesh-Based Simulationscited by 4
  32. 2025Rethinking Layer-wise Model Merging through Chain of Mergescited by 4
  33. 2025Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddingscited by 3
  34. 2026A Structural Theory of Position Bias in Transformerscited by 3
  35. 2026AdaSplash-2: Faster Differentiable Sparse Attentioncited by 3
  36. 2026NITP: Next Implicit Token Prediction for LLM Pre-trainingcited by 3
  37. 2025Continuity and Isolation Lead to Doubts or Dilemmas in Large Language Modelscited by 2
  38. 2025Stream: Scaling up Mechanistic Interpretability to Long Context in LLMs via Sparse Attentioncited by 2

50 most cited of 87 All 87 (JSON)