mlsanityData to 7 Oct 2026

How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training

arXiv 2502.11196 ↗First posted 16 Feb 2025cs.LG cs.AI cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Despite exceptional capabilities in knowledge-intensive tasks, Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge, particularly how to structurally embed acquired knowledge in their neural computations. We address this issue through the lens of knowledge circuit evolution, identifying computational subgraphs that facilitate knowledge storage and processing. Our systematic analysis of circuit evolution throughout continual pre-training reveals several key findings: (1) the acquisition of new knowledge is influenced by its relevance to pre-existing knowledge; (2) the evolution of knowledge circuits exhibits a distinct phase shift from formation to optimization; (3) the evolution of knowledge circuits follows a deep-to-shallow pattern. These insights not only advance our theoretical understanding of the mechanisms of new knowledge acquisition in LLMs, but also provide potential implications for improving continual pre-training strategies to enhance model performance. Code and data will be available at https://github.com/zjunlp/DynamicKnowledgeCircuits.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

38 of 58 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  11. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  12. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
26 more
  1. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  4. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  5. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  6. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  7. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  8. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  9. 2025Open Problems in Mechanistic Interpretabilitycited by 173
  10. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  11. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 144
  12. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  13. 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 137
  14. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  15. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 92
  16. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 89
  17. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  18. 2024Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Modelscited by 69
  19. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  20. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 60
  21. 2024Injecting New Knowledge into Large Language Models via Supervised Fine-Tuningcited by 56
  22. 2024Instruction-tuned Language Models are Better Knowledge Learnerscited by 54
  23. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  24. 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 39
  25. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  26. 2024Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisitioncited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 11.

22 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025How do language models learn facts? Dynamics, curricula and hallucinationscited by 33
  3. 2025Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Trainingcited by 8
  4. 2025Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Modelscited by 6
  5. 2026Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Modelscited by 4
  6. 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysiscited by 3
  7. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  8. 2025Discursive Circuits: How Do Language Models Understand Discourse Relations?cited by 3
  9. 2026What Does Loss Optimization Actually Teach, If Anything? Knowledge Dynamics in Continual Pre-training of LLMscited by 3
  10. 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Spacecited by 3
  11. 2025Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMscited by 2
  12. 2025Progressive Homeostatic and Plastic Prompt Tuning for Audio-Visual Multi-Task Incremental Learningcited by 1
10 more
  1. 2025A Review of Developmental Interpretability in Large Language Modelscited by 1
  2. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  3. 2026How Do Large Language Models Learn Concepts During Continual Pre-Training?cited by 1
  4. 2026Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Modelscited by 0
  5. 2026Learning What to Predict: Downstream-Guided Task Design for Continued Pretrainingcited by 0
  6. 2026V-DyKnow: A Dynamic Benchmark for Time-Sensitive Knowledge in Vision Language Modelscited by 0
  7. 2026DECSELFMASK: Leveraging Unlabeled Text via Self-Relevance-Guided Masking for Decoder-Only Classificationcited by 0
  8. 2026Benchmarking Knowledge Editing using Logical Rulescited by 0
  9. 2026A theoretical model for task routing in mixture-of-expert transformerscited by 0
  10. 2026Identifying Temporal Features within Transcoders for Time Sensitive Factual Recallcited by 0