# How do language models learn facts? Dynamics, curricula and hallucinations

arXiv: https://arxiv.org/abs/2503.21676 · HTML: https://mlsanity.com/paper/2503.21676/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Data Distributional Properties Drive Emergent In-Context Learning in Transformers](https://mlsanity.com/paper/2205.05055/) (2022)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models](https://mlsanity.com/paper/2402.19427/) (2024)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [Zoology: Measuring and Improving Recall in Efficient Language Models](https://mlsanity.com/paper/2312.04927/) (2023)
- [Open-Ended Learning Leads to Generally Capable Agents](https://mlsanity.com/paper/2107.12808/) (2021)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws](https://mlsanity.com/paper/2404.05405/) (2024)
- [The mechanistic basis of data dependence and abrupt learning in an in-context classification task](https://mlsanity.com/paper/2312.03002/) (2023)
- [How Do Large Language Models Acquire Factual Knowledge During Pretraining?](https://mlsanity.com/paper/2406.11813/) (2024)
- [The Transient Nature of Emergent In-Context Learning in Transformers](https://mlsanity.com/paper/2311.08360/) (2023)
- [Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots](https://mlsanity.com/paper/1301.4862/) (2013)

## Built on by (most cited first, 30 of 37)

- [On the generalization of language models from in-context learning and finetuning: a controlled study](https://mlsanity.com/paper/2505.00661/) (2025)
- [Tracing the Representation Geometry of Language Models from Pretraining to Post-training](https://mlsanity.com/paper/2509.23024/) (2025)
- [Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts](https://mlsanity.com/paper/2505.17127/) (2025)
- [The emergence of sparse attention: impact of data distribution and benefits of repetition](https://mlsanity.com/paper/2505.17863/) (2025)
- [Robust Hallucination Detection in LLMs via Adaptive Token Selection](https://mlsanity.com/paper/2504.07863/) (2025)
- [Data Mixing Can Induce Phase Transitions in Knowledge Acquisition](https://mlsanity.com/paper/2505.18091/) (2025)
- [Do Activation Verbalization Methods Convey Privileged Information?](https://mlsanity.com/paper/2509.13316/) (2025)
- [Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning](https://mlsanity.com/paper/2507.10624/) (2025)
- [Training AI Scientists to Replicate Research](https://mlsanity.com/paper/2608.13331/) (2026)
- [Why Fine-Tuning Encourages Hallucinations and How to Fix It](https://mlsanity.com/paper/2604.15574/) (2026)
- [RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling](https://mlsanity.com/paper/2506.08672/) (2025)
- [Rote Learning Considered Useful: Generalizing over Memorized Data in LLMs](https://mlsanity.com/paper/2507.21914/) (2025)
- [Universal Redundancies in Time Series Foundation Models](https://mlsanity.com/paper/2602.01605/) (2026)
- [How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models](https://mlsanity.com/paper/2510.02370/) (2025)
- [Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis](https://mlsanity.com/paper/2505.14406/) (2025)
- [Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization](https://mlsanity.com/paper/2510.16096/) (2025)
- [Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge](https://mlsanity.com/paper/2603.20969/) (2026)
- [Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts](https://mlsanity.com/paper/2604.08519/) (2026)
- [Understanding Data Temporality Impact on Large Language Models Pre-training](https://mlsanity.com/paper/2605.22769/) (2026)
- [Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions](https://mlsanity.com/paper/2603.06248/) (2026)
- [Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale](https://mlsanity.com/paper/2603.06592/) (2026)
- [Structure Before Collapse: Transient semantic geometry in next-token prediction](https://mlsanity.com/paper/2606.26749/) (2026)
- [Probability Consistency in Large Language Models: Theoretical Foundations Meet Empirical Discrepancies](https://mlsanity.com/paper/2505.08739/) (2025)
- [How Do Large Language Models Learn Concepts During Continual Pre-Training?](https://mlsanity.com/paper/2601.03570/) (2026)
- [Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates](https://mlsanity.com/paper/2605.20005/) (2026)
- [The Future of Facts: Tracing the Factual Generation-Verification Gap](https://mlsanity.com/paper/2605.27564/) (2026)
- [Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining](https://mlsanity.com/paper/2606.26050/) (2026)
- [MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers](https://mlsanity.com/paper/2607.10034/) (2026)
- [FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition](https://mlsanity.com/paper/2506.05639/) (2025)
- [Tracing Multilingual Knowledge Acquisition Dynamics in Domain Adaptation: A Case Study of English-Japanese Biomedical Adaptation](https://mlsanity.com/paper/2510.12115/) (2025)

## Abstract (arXiv)

Large language models accumulate vast knowledge during pre-training, yet the dynamics governing this acquisition remain poorly understood. This work investigates the learning dynamics of language models on a synthetic factual recall task, uncovering three key findings: First, language models learn in three phases, exhibiting a performance plateau before acquiring precise factual knowledge. Mechanistically, this plateau coincides with the formation of attention-based circuits that support recall. Second, the training data distribution significantly impacts learning dynamics, as imbalanced distributions lead to shorter plateaus. Finally, hallucinations emerge simultaneously with knowledge, and integrating new knowledge into the model through fine-tuning is challenging, as it quickly corrupts its existing parametric memories. Our results emphasize the importance of data distribution in knowledge acquisition and suggest novel data scheduling strategies to accelerate neural network training.