mlsanityData to 7 Oct 2026

Transformer Block Coupling and its Correlation with Generalization in LLMs

arXiv 2407.07810 ↗First posted 10 Jul 2024cs.LG cs.AI cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have made significant strides in natural language processing, and a precise understanding of the internal mechanisms driving their success is essential. In this work, we analyze the trajectories of token embeddings as they pass through transformer blocks, linearizing the system along these trajectories through their Jacobian matrices. By examining the relationships between these block Jacobians, we uncover the phenomenon of transformer block coupling in a multitude of LLMs, characterized by the coupling of their top singular vectors across tokens and depth. Our findings reveal that coupling textitpositively correlates with model performance, and that this relationship is stronger than with other hyperparameters such as parameter count, model depth, and embedding dimension. We further investigate how these properties emerge during training, observing a progressive development of coupling, increased linearity, and layer-wise exponential growth in token trajectories. Additionally, experiments with Vision Transformers (ViTs) corroborate the emergence of coupling and its relationship with generalization, reinforcing our findings in LLMs. Collectively, these insights offer a novel perspective on token interactions in transformers, opening new directions for studying their mechanisms as well as improving training and generalization.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 43 of its 64 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

43 of 64 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2018Neural Ordinary Differential Equationscited by 3,733
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  10. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
31 more
  1. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  2. 2019Deep Equilibrium Modelscited by 603
  3. 2017Stable Architectures for Deep Neural Networkscited by 340
  4. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
  5. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 274
  6. 2023Birth of a Transformer: A Memory Viewpointcited by 154
  7. 2023The emergence of clusters in self-attention dynamicscited by 133
  8. 2020Traces of Class/Cross-Class Structure Pervade Deep Learning Spectracited by 94
  9. 2023A Theory of Emergent In-Context Learning as Implicit Structure Inductioncited by 86
  10. 2017Residual Connections Encourage Iterative Inferencecited by 74
  11. 2019How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representationscited by 63
  12. 2016Highway and Residual Networks learn Unrolled Iterative Estimationcited by 52
  13. 2016Convolutional Neural Networks Analyzed via Convolutional Sparse Codingcited by 51
  14. 2023Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Modelcited by 43
  15. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 36
  16. 2023VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformerscited by 32
  17. 2023Understanding Deep Representation Learning via Layerwise Feature Compression and Discriminationcited by 31
  18. 2020Separation and Concentration in Deep Networkscited by 28
  19. 2022Nearest Class-Center Simplification through Intermediate Layerscited by 20
  20. 2023Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networkscited by 20
  21. 2021Scaling Properties of Deep Residual Networkscited by 16
  22. 2022A Neural ODE Interpretation of Transformer Layerscited by 13
  23. 2023Spectral alignment of stochastic gradient descent for high-dimensional classification taskscited by 13
  24. 2024Residual Alignment: Uncovering the Mechanisms of Residual Networkscited by 13
  25. 2021ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translationcited by 12
  26. 2024Progressive Feedforward Collapse of ResNet Trainingcited by 12
  27. 2024Unifying Low Dimensional Spectra in Deep Learningcited by 11
  28. 2024Provable Emergence of Deep Neural Collapse and Low-Rank Bias in L2-Regularized Nonlinear Networkscited by 10
  29. 2021Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Spacecited by 8
  30. 2021Probing neural networks with t-SNE, class-specific projections and a guided tourcited by 6
  31. 2019Residual Networks as Nonlinear Systems: Stability Analysis using Linearizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 6.

9 papers.

  1. 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Controlcited by 8
  2. 2025Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layerscited by 7
  3. 2024Lines of Thought in Large Language Modelscited by 5
  4. 2026Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topologycited by 5
  5. 2026Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reductioncited by 1
  6. 2025Feature Dynamics as Implicit Data Augmentation: A Depth-Decomposed View on Deep Neural Network Generalizationcited by 0
  7. 2026Attribution-Guided Continual Learning for Large Language Modelscited by 0
  8. 2026Dynamic Mode Decomposition along Depth in Vision Transformerscited by 0
  9. 2026Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimizationcited by 0