mlsanityData to 7 Oct 2026

Abrupt Learning in Transformers: A Case Study on Matrix Completion

arXiv 2410.22244 ↗First posted 29 Oct 2024cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent analysis on the training dynamics of Transformers has unveiled an interesting characteristic: the training loss plateaus for a significant number of training steps, and then suddenly (and sharply) drops to near--optimal values. To understand this phenomenon in depth, we formulate the low-rank matrix completion problem as a masked language modeling (MLM) task, and show that it is possible to train a BERT model to solve this task to low error. Furthermore, the loss curve shows a plateau early in training followed by a sudden drop to near-optimal values, despite no changes in the training procedure or hyper-parameters. To gain interpretability insights into this sudden drop, we examine the model's predictions, attention heads, and hidden states before and after this transition. Concretely, we observe that (a) the model transitions from simply copying the masked input to accurately predicting the masked entries; (b) the attention heads transition to interpretable patterns relevant to the task; and (c) the embeddings and hidden states encode information relevant to the problem. We also analyze the training dynamics of individual model components to understand the sudden drop in loss.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 29 Oct 2024.

Builds on

34 of 43 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  5. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  6. 2022In-context Learning and Induction Headscited by 930
  7. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  9. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  10. 2022Transformers learn in-context by gradient descentcited by 714
  11. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  12. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
22 more
  1. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  2. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  3. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  4. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  5. 2019Revealing the Dark Secrets of BERTcited by 300
  6. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
  7. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 274
  8. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
  9. 2022Predictability and Surprise in Large Generative Modelscited by 209
  10. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  11. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  12. 2023One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attentioncited by 168
  13. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  14. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 142
  15. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  16. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  17. 2023Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Contextcited by 63
  18. 2021Linear algebra with transformerscited by 62
  19. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Modelscited by 50
  20. 2023Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Datacited by 32
  21. 2024A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Languagecited by 26
  22. 2023Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problemscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 5.

11 papers.

  1. 2024ICLR: In-Context Learning of Representationscited by 49
  2. 2025Training Dynamics of In-Context Learning in Linear Attentioncited by 44
  3. 2026There Will Be a Scientific Theory of Deep Learningcited by 21
  4. 2025Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Modelcited by 13
  5. 2026Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentioncited by 8
  6. 2025What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformerscited by 5
  7. 2026Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernelcited by 5
  8. 2025SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Trainingcited by 4
  9. 2025Understanding the Staged Dynamics of Transformers in Learning Latent Structurecited by 1
  10. 2026Emergent Capabilities Arise Randomly from Learning Sparse Attention Patternscited by 1
  11. 2026The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspacescited by 0