mlsanityData to 7 Oct 2026

Dissecting Recall of Factual Associations in Auto-Regressive Language Models

arXiv 2304.14767 ↗First posted 28 Apr 2023cs.CLcited by 512

Not on the map. Why?

Cited by 40 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Transformer-based language models (LMs) are known to capture factual knowledge in their parameters. While previous work looked into where factual associations are stored, only little is known about how they are retrieved internally during inference. We investigate this question through the lens of information flow. Given a subject-relation query, we study how the model aggregates information about the subject and relation to predict the correct attribute. With interventions on attention edges, we first identify two critical points where information propagates to the prediction: one from the relation positions followed by another from the subject positions. Next, by analyzing the information at these points, we unveil a three-step internal mechanism for attribute extraction. First, the representation at the last-subject position goes through an enrichment process, driven by the early MLP sublayers, to encode many subject-related attributes. Second, information from the relation propagates to the prediction. Third, the prediction representation "queries" the enriched subject to extract the attribute. Perhaps surprisingly, this extraction is typically done via attention heads, which often encode subject-attribute mappings in their parameters. Overall, our findings introduce a comprehensive view of how factual associations are stored and extracted internally in LMs, facilitating future research on knowledge localization and editing.

Key facts. Cited by 512 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 28 Apr 2023.

Builds on

32 of 49 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Layer Normalizationcited by 5,579
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  7. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  8. 2022Mass-Editing Memory in a Transformercited by 976
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  11. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 590
20 more
  1. 2021Editing Factual Knowledge in Language Modelscited by 548
  2. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  3. 2021Fast Model Editing at Scalecited by 491
  4. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  5. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  6. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  7. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 148
  8. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  9. 2022Analyzing Transformers in Embedding Spacecited by 131
  10. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  11. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  12. 2023Crawling the Internal Knowledge-Base of Language Modelscited by 85
  13. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  14. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  15. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  16. 2023Quantifying Context Mixing in Transformerscited by 43
  17. 2021BERTnesia: Investigating the capture and forgetting of knowledge in BERTcited by 40
  18. 2022LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Modelscited by 28
  19. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28
  20. 2022How to Dissect a Muppet: The Structure of Transformer Embedding Spacescited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 43; 2024: 153; 2025: 160; 2026: 156.

512 papers.

Sort:

By area:

  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  4. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  5. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  6. 2023Language Models Represent Space and Timecited by 306
  7. 2023Function Vectors in Large Language Modelscited by 292
  8. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  9. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  10. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  11. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  12. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
38 more
  1. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  2. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  3. 2024How to use and interpret activation patchingcited by 191
  4. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  5. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  6. 2023PMET: Precise Model Editing in a Transformercited by 170
  7. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  8. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  9. 2023Birth of a Transformer: A Memory Viewpointcited by 154
  10. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  11. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  12. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  13. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
  14. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  15. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
  16. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  17. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 115
  18. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  19. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  20. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 110
  21. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  22. 2023How do Language Models Bind Entities in Context?cited by 102
  23. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  24. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  25. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  26. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 88
  27. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  28. 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
  29. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  30. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  31. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  32. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Modelscited by 77
  33. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 75
  34. 2025Do Multilingual LLMs Think In English?cited by 75
  35. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  36. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  37. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Modelscited by 66
  38. 2024Attention Heads of Large Language Models: A Surveycited by 65

50 most cited of 512 All 512 (JSON)