mlsanityData to 7 Oct 2026

How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding

arXiv 2303.04245 ↗First posted 7 Mar 2023cs.LG cs.CL stat.MLcited by 85

Not on the map. Why?

Abstract (arXiv)

While the successes of transformers across many domains are indisputable, accurate understanding of the learning mechanics is still largely lacking. Their capabilities have been probed on benchmarks which include a variety of structured and reasoning tasks -- but mathematical understanding is lagging substantially behind. Recent lines of work have begun studying representational aspects of this question: that is, the size/depth/complexity of attention-based networks to perform certain tasks. However, there is no guarantee the learning dynamics will converge to the constructions proposed. In our paper, we provide fine-grained mechanistic understanding of how transformers learn "semantic structure", understood as capturing co-occurrence structure of words. Precisely, we show, through a combination of mathematical analysis and experiments on Wikipedia data and synthetic data modeled by Latent Dirichlet Allocation (LDA), that the embedding layer and the self-attention layer encode the topical structure. In the former case, this manifests as higher average inner product of embeddings between same-topic words. In the latter, it manifests as higher average pairwise attention between same-topic words. The mathematical results involve several assumptions to make the analysis tractable, which we verify on data, and might be of independent interest as well.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

41 of 60 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  6. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  7. 2020On Layer Normalization in the Transformer Architecturecited by 906
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  9. 2019Designing and Interpreting Probes with Control Taskscited by 608
  10. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
  11. 2019Revealing the Dark Secrets of BERTcited by 300
  12. 2016Using the Output Embedding to Improve Language Modelscited by 266
29 more
  1. 2022Transformers Learn Shortcuts to Automatacited by 260
  2. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 241
  3. 2020Understanding the Difficulty of Training Transformerscited by 210
  4. 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
  5. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  6. 2018On the Practical Computational Power of Finite Precision RNNs for Language Recognitioncited by 117
  7. 2021Self-Attention Networks Can Process Bounded Hierarchical Languagescited by 105
  8. 2022Unveiling Transformers with LEGO: a synthetic reasoning taskcited by 105
  9. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 101
  10. 2022Vision Transformers provably learn spatial structurecited by 92
  11. 2019Do Attention Heads in BERT Track Syntactic Dependencies?cited by 87
  12. 2020On the Computational Power of Transformers and its Implications in Sequence Modelingcited by 85
  13. 2020Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!cited by 73
  14. 2020RNNs can generate bounded hierarchical languages with optimal memorycited by 59
  15. 2019Sequential Neural Networks as Automatacited by 54
  16. 2020Contrastive estimation reveals topic posterior information to linear modelscited by 54
  17. 2020How Can Self-Attention Networks Recognize Dyck-n Languages?cited by 51
  18. 2019LSTM Networks Can Perform Dynamic Countingcited by 45
  19. 2020Topic Modeling with Contextualized Word Representation Clusterscited by 41
  20. 2021Approximating How Single Head Attention Learnscited by 38
  21. 2022Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topicscited by 35
  22. 2022Topic Discovery via Latent Space Clustering of Pretrained Language Model Representationscited by 34
  23. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15
  24. 2022An Analysis of Attention via the Lens of Exchangeability and Latent Variable Modelscited by 15
  25. 2021Effective Attention Sheds Light On Interpretabilitycited by 11
  26. 2016Provable Algorithms for Inference in Topic Modelscited by 6
  27. 2021The Limitations of Limited Context for Constituency Parsingcited by 5
  28. 2015On some provably correct cases of variational inference for topic modelscited by 1
  29. 2023Topics in Contextualised Attention Embeddingscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 27; 2025: 20; 2026: 11.

85 papers.

Sort:

By area:

  1. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  2. 2023Trained Transformers Learn Linear Models In-Contextcited by 331
  3. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  4. 2023Birth of a Transformer: A Memory Viewpointcited by 154
  5. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 120
  6. 2023Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformercited by 101
  7. 2023Linear attention is (maybe) all you need (to understand transformer optimization)cited by 97
  8. 2023How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?cited by 93
  9. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 74
  10. 2023JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attentioncited by 52
  11. 2023Attention Scheme Inspired Softmax Regressioncited by 49
  12. 2023An Over-parameterized Exponential Regressioncited by 45
38 more
  1. 2024Why Larger Language Models Do In-context Learning Differently?cited by 44
  2. 2024An Information-Theoretic Analysis of In-Context Learningcited by 39
  3. 2024Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chainscited by 39
  4. 2025When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformerscited by 38
  5. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  6. 2023An Iterative Algorithm for Rescaled Hyperbolic Functions Regressioncited by 35
  7. 2024How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?cited by 35
  8. 2024Mechanics of Next Token Prediction with Self-Attentioncited by 34
  9. 2023In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trickcited by 32
  10. 2023A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Timecited by 30
  11. 2023Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammarscited by 30
  12. 2025Taming Knowledge Conflicts in Language Modelscited by 27
  13. 2023The Crucial Role of Normalization in Sharpness-Aware Minimizationcited by 26
  14. 2024In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitznesscited by 22
  15. 2024Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Timecited by 22
  16. 2024Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descentcited by 21
  17. 2023Zero-th Order Algorithm for Softmax Attention Optimizationcited by 16
  18. 2024Implicit Regularization of Gradient Flow on One-Layer Softmax Attentioncited by 16
  19. 2024Attention layers provably solve single-location regressioncited by 15
  20. 2024On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixturescited by 15
  21. 2024Tracking the Feature Dynamics in LLM Training: A Mechanistic Studycited by 15
  22. 2024What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encodingcited by 14
  23. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 14
  24. 2024Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysiscited by 14
  25. 2024One-Layer Transformer Provably Learns One-Nearest Neighbor In Contextcited by 14
  26. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 14
  27. 2024How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regressioncited by 13
  28. 2023Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weightscited by 11
  29. 2024Attention Meets Post-hoc Interpretability: A Mathematical Perspectivecited by 11
  30. 2024What Should Embeddings Embed? Autoregressive Models Represent Latent Generating Distributionscited by 10
  31. 2025On the Query Complexity of Verifier-Assisted Language Generationcited by 10
  32. 2023Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regressioncited by 9
  33. 2024Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimizationcited by 9
  34. 2025On the Emergence of Linear Analogies in Word Embeddingscited by 9
  35. 2025From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamicscited by 9
  36. 2023Solving Tensor Low Cycle Rank Approximationcited by 8
  37. 2024Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysiscited by 8
  38. 2024Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysiscited by 8

50 most cited of 85 All 85 (JSON)