mlsanityData to 14 Oct 2026

Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features

arXiv 2402.02969 ↗First posted 5 Feb 2024stat.ML cs.CL cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Understanding the reasons behind the exceptional success of transformers requires a better analysis of why attention layers are suitable for NLP tasks. In particular, such tasks require predictive models to capture contextual meaning which often depends on one or few words, even if the sentence is long. Our work studies this key property, dubbed word sensitivity (WS), in the prototypical setting of random features. We show that attention layers enjoy high WS, namely, there exists a vector in the space of embeddings that largely perturbs the random attention features map. The argument critically exploits the role of the softmax in the attention layer, highlighting its benefit compared to other activations (e.g., ReLU). In contrast, the WS of standard random features is of order 1/√n, n being the number of words in the textual sample, and thus it decays with the length of the context. We then translate these results on the word sensitivity into generalization bounds: due to their low WS, random features provably cannot learn to distinguish between two sentences that differ only in a single word; in contrast, due to their high WS, random attention features have higher generalization capabilities. We validate our theoretical results with experimental evidence over the BERT-Base word embeddings of the imdb review dataset.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 60 of its 71 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

60 of 71 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  7. 2020Longformer: The Long-Document Transformercited by 3,413
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  9. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  12. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
38 more
  1. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  2. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  3. 2019Benign Overfitting in Linear Regressioncited by 549
  4. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  5. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
  6. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
  7. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 406
  8. 2021Deep learning: a statistical viewpointcited by 240
  9. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  10. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 182
  11. 2020The Lipschitz Constant of Self-Attentioncited by 174
  12. 2022Neural Networks can Learn Representations with Gradient Descentcited by 161
  13. 2023Birth of a Transformer: A Memory Viewpointcited by 161
  14. 2019Linearized two-layers neural networks in high dimensioncited by 155
  15. 2021A Universal Law of Robustness via Isoperimetrycited by 151
  16. 2020When Do Neural Networks Outperform Kernel Methods?cited by 150
  17. 2022High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representationcited by 150
  18. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  19. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
  20. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 128
  21. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  22. 2023Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformercited by 102
  23. 2017Structured Attention Networkscited by 95
  24. 2018Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?cited by 94
  25. 2022Vision Transformers provably learn spatial structurecited by 93
  26. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  27. 2017A Random Matrix Approach to Neural Networkscited by 80
  28. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
  29. 2023A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexitycited by 77
  30. 2023Mimetic Initialization of Self-Attention Layerscited by 54
  31. 2023On the Role of Attention in Prompt-tuningcited by 50
  32. 2023A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networkscited by 49
  33. 2020Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixturescited by 47
  34. 2023What can a Single Attention Layer Learn? A Study Through the Random Features Lenscited by 40
  35. 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
  36. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  37. 2023Deterministic equivalent and error universality of deep random features learningcited by 35
  38. 2023Scaling Laws for Associative Memoriescited by 35

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representationscited by 38
  2. 2024Mechanics of Next Token Prediction with Self-Attentioncited by 34
  3. 2024Transformers Learn Low Sensitivity Functions: Investigations and Implicationscited by 14
  4. 2025Dynamics of Spontaneous Topic Changes in Next Token Prediction with Self-Attentioncited by 2
  5. 2025Attention with Trained Embeddings Provably Selects Important Tokenscited by 1