mlsanityData to 13 Oct 2026

Noise Stability of Transformer Models

arXiv 2602.08287 ↗First posted 9 Feb 2026cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Understanding simplicity biases in deep learning offers a promising path toward developing reliable AI. A common metric for this, inspired by Boolean function analysis, is average sensitivity, which captures a model's robustness to single-token perturbations. We argue that average sensitivity has two key limitations: it lacks a natural generalization to real-valued domains and fails to explain the "junta-like" input dependence we empirically observe in modern LLMs. To address these limitations, we propose noise stability as a more comprehensive simplicity metric. Noise stability expresses a model's robustness to correlated noise applied to all input coordinates simultaneously. We provide a theoretical analysis of noise stability for single-layer attention and ReLU MLP layers and tackle the multi-layer propagation problem with a covariance interval propagation approach. Building on this theory, we develop a practical noise stability regularization method. Experiments on algorithmic and next-token-prediction tasks show that our regularizer consistently catalyzes grokking and accelerates training by approximately 35% and 75% respectively. Our results sculpt a new connection between signal propagation in neural networks and interpretability, with noise stability emerging as a powerful tool for understanding and improving modern Transformers.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

36 of 54 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,106
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,137
  3. 2018On the Spectral Bias of Neural Networkscited by 1,329
  4. 2023Progress measures for grokking via mechanistic interpretabilitycited by 831
  5. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 363
  6. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
  7. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 287
  8. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  9. 2016Exponential expressivity in deep neural networks through transient chaoscited by 259
  10. 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 180
  11. 2022Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapsecited by 138
  12. 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 131
24 more
  1. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  2. 2020A Unifying View on Implicit Bias in Training Linear Neural Networkscited by 74
  3. 2024Why are Sensitive Functions Hard for Transformers?cited by 74
  4. 2022Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functionscited by 69
  5. 2019A Fine-Grained Spectral Perspective on Neural Networkscited by 68
  6. 2021Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shapingcited by 52
  7. 2023Simplicity Bias in 1-Hidden Layer Neural Networkscited by 26
  8. 2024Nonlinear spiked covariance matrices and signal propagation in deep neural networkscited by 25
  9. 2022Signal Propagation: A Framework for Learning and Inference In a Forward Passcited by 24
  10. 2024A distributional simplicity bias in the learning dynamics of transformerscited by 24
  11. 2019Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networkscited by 20
  12. 2021Feature Learning and Signal Propagation in Deep Neural Networkscited by 18
  13. 2020The shape and simplicity biases of adversarially robust ImageNet-trained CNNscited by 14
  14. 2025Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wildcited by 13
  15. 2022Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularizationcited by 12
  16. 2024An Analytic Solution to Covariance Propagation in Neural Networkscited by 12
  17. 2021Implicit Bias of Linear RNNscited by 11
  18. 2024Simplicity Bias of Two-Layer Networks beyond Linearly Separable Datacited by 11
  19. 2024Simplicity bias and optimization threshold in two-layer ReLU networkscited by 11
  20. 2023Understanding Self-attention Mechanism via Dynamical System Perspectivecited by 9
  21. 2024Can Implicit Bias Imply Adversarial Robustness?cited by 5
  22. 2024Simplicity Bias via Global Convergence of Sharpness Minimizationcited by 4
  23. 2018Critical initialisation for deep signal propagation in noisy rectifier neural networkscited by 1
  24. 2024Algorithms for Sparse LPN and LSPN Against Low-noisecited by 1

Built on by

Not cited yet.