# How Do Large Language Models Learn Concepts During Continual Pre-Training?

arXiv: https://arxiv.org/abs/2601.03570 · HTML: https://mlsanity.com/paper/2601.03570/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks](https://mlsanity.com/paper/2311.12786/) (2023)
- [LLM Circuit Analyses Are Consistent Across Training and Scale](https://mlsanity.com/paper/2407.10827/) (2024)
- [Knowledge Circuits in Pretrained Transformers](https://mlsanity.com/paper/2405.17969/) (2024)
- [The Geometry of Concepts: Sparse Autoencoder Feature Structure](https://mlsanity.com/paper/2410.19750/) (2024)
- [Knowledge Mechanisms in Large Language Models: A Survey and Perspective](https://mlsanity.com/paper/2407.15017/) (2024)
- [AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge](https://mlsanity.com/paper/2412.13670/) (2024)
- [How do language models learn facts? Dynamics, curricula and hallucinations](https://mlsanity.com/paper/2503.21676/) (2025)
- [Language Model Analysis for Ontology Subsumption Inference](https://mlsanity.com/paper/2302.06761/) (2023)
- [How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training](https://mlsanity.com/paper/2502.11196/) (2025)
- [COPEN: Probing Conceptual Knowledge in Pre-trained Language Models](https://mlsanity.com/paper/2211.04079/) (2022)
- [The Geometry of Reasoning: Flowing Logics in Representation Space](https://mlsanity.com/paper/2510.09782/) (2025)
- [Unpacking Large Language Models with Conceptual Consistency](https://mlsanity.com/paper/2209.15093/) (2022)
- [Tracking the Feature Dynamics in LLM Training: A Mechanistic Study](https://mlsanity.com/paper/2412.17626/) (2024)
- [Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts](https://mlsanity.com/paper/2412.04614/) (2024)
- [Do language models have coherent mental models of everyday things?](https://mlsanity.com/paper/2212.10029/) (2022)
- [On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe](https://mlsanity.com/paper/2402.14404/) (2024)
- [Reasoning about concepts with LLMs: Inconsistencies abound](https://mlsanity.com/paper/2405.20163/) (2024)
- [Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models](https://mlsanity.com/paper/2408.11827/) (2024)

## Built on by (most cited first, 1 of 1)

- [Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation](https://mlsanity.com/paper/2605.31295/) (2026)

## Abstract (arXiv)

Human beings primarily understand the world through concepts (e.g., dog), abstract mental representations that structure perception, reasoning, and learning. However, how large language models (LLMs) acquire, retain, and forget such concepts during continual pretraining remains poorly understood. In this work, we study how individual concepts are acquired and forgotten, as well as how multiple concepts interact through interference and synergy. We link these behavioral dynamics to LLMs' internal concept circuits, computational subgraphs associated with specific concepts, and incorporate graph metrics to characterize circuit topology. Our analysis reveals: (1) LLMs concept circuits provide a non-trivial, consistent signal of concept learning and forgetting; (2) concept circuits exhibit a stage-wise temporal pattern during continual pretraining, with an early increase followed by gradual decrease and stabilization; (3) concepts with larger learning gains tend to exhibit greater forgetting under subsequent training; (4) semantically similar concepts induce stronger interference than weakly related ones; (5) conceptual knowledge differs in their transferability, with some significantly facilitating the learning of others. Together, our findings provide a circuit-level view of concept learning dynamics and motivate concept-aware training strategies, such as Circuit-aware Experience Replay, which uses circuit topology to prioritize concepts vulnerable to forgetting.