# Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models

arXiv: https://arxiv.org/abs/2403.19521 · HTML: https://mlsanity.com/paper/2403.19521/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla](https://mlsanity.com/paper/2307.09458/) (2023)
- [CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation](https://mlsanity.com/paper/2401.01275/) (2024)
- [The mechanistic basis of data dependence and abrupt learning in an in-context classification task](https://mlsanity.com/paper/2312.03002/) (2023)
- [Circuit Component Reuse Across Tasks in Transformer Language Models](https://mlsanity.com/paper/2310.08744/) (2023)
- [Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching](https://mlsanity.com/paper/2311.17030/) (2023)
- [Characterizing Mechanisms for Factual Recall in Language Models](https://mlsanity.com/paper/2310.15910/) (2023)
- [Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals](https://mlsanity.com/paper/2402.11655/) (2024)
- [Neuron-Level Knowledge Attribution in Large Language Models](https://mlsanity.com/paper/2312.12141/) (2023)
- [Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use](https://mlsanity.com/paper/2312.04455/) (2023)
- [An Analysis and Mitigation of the Reversal Curse](https://mlsanity.com/paper/2311.07468/) (2023)
- [Monotonic Representation of Numeric Properties in Language Models](https://mlsanity.com/paper/2403.10381/) (2024)

## Built on by (most cited first, 30 of 31)

- [Hymba: A Hybrid-head Architecture for Small Language Models](https://mlsanity.com/paper/2411.13676/) (2024)
- [A Primer on the Inner Workings of Transformer-based Language Models](https://mlsanity.com/paper/2405.00208/) (2024)
- [Attention Heads of Large Language Models: A Survey](https://mlsanity.com/paper/2409.03752/) (2024)
- [Knowledge Circuits in Pretrained Transformers](https://mlsanity.com/paper/2405.17969/) (2024)
- [Understanding Factual Recall in Transformers via Associative Memories](https://mlsanity.com/paper/2412.06538/) (2024)
- [Memory3: Language Modeling with Explicit Memory](https://mlsanity.com/paper/2407.01178/) (2024)
- [Taming Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2503.10996/) (2025)
- [Mixture of In-Context Experts Enhance LLMs' Long Context Awareness](https://mlsanity.com/paper/2406.19598/) (2024)
- [Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning](https://mlsanity.com/paper/2503.05212/) (2025)
- [Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules](https://mlsanity.com/paper/2407.06677/) (2024)
- [Revealing and Mitigating Over-Attention in Knowledge Editing](https://mlsanity.com/paper/2502.14838/) (2025)
- [Autonomy-of-Experts Models](https://mlsanity.com/paper/2501.13074/) (2025)
- [ResiDual Transformer Alignment with Spectral Decomposition](https://mlsanity.com/paper/2411.00246/) (2024)
- [Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitration](https://mlsanity.com/paper/2602.03677/) (2026)
- [AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection](https://mlsanity.com/paper/2505.07293/) (2025)
- [HARP: Hallucination Detection via Reasoning Subspace Projection](https://mlsanity.com/paper/2509.11536/) (2025)
- [Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge](https://mlsanity.com/paper/2509.24653/) (2025)
- [Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization](https://mlsanity.com/paper/2510.16096/) (2025)
- [AIR: Post-training Data Selection for Reasoning via Attention Head Influence](https://mlsanity.com/paper/2512.13279/) (2025)
- [DeepInsert: Early Layer Bypass for Efficient and Performant Multimodal Understanding](https://mlsanity.com/paper/2504.19327/) (2025)
- [Order-Level Attention Similarity Across Language Models: A Latent Commonality](https://mlsanity.com/paper/2511.05064/) (2025)
- [PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference Overhead](https://mlsanity.com/paper/2409.19745/) (2024)
- [LargePiG: Your Large Language Model is Secretly a Pointer Generator](https://mlsanity.com/paper/2410.11366/) (2024)
- [Information Anxiety in Large Language Models](https://mlsanity.com/paper/2411.10813/) (2024)
- [Benchmarking and Rethinking Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2505.18690/) (2025)
- [Listen to the Layers: Mitigating Hallucinations with Inter-Layer Disagreement](https://mlsanity.com/paper/2602.09486/) (2026)
- [Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task](https://mlsanity.com/paper/2604.12426/) (2026)
- [Context-Gated Associative Retrieval: From Theory to Transformers](https://mlsanity.com/paper/2605.10970/) (2026)
- [Relational Linear Properties in Language Models: An Empirical Investigation](https://mlsanity.com/paper/2605.22532/) (2026)
- [Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence](https://mlsanity.com/paper/2606.00570/) (2026)

## Abstract (arXiv)

In this paper, we delve into several mechanisms employed by Transformer-based language models (LLMs) for factual recall tasks. We outline a pipeline consisting of three major steps: (1) Given a prompt ``The capital of France is,'' task-specific attention heads extract the topic token, such as ``France,'' from the context and pass it to subsequent MLPs. (2) As attention heads' outputs are aggregated with equal weight and added to the residual stream, the subsequent MLP acts as an ``activation,'' which either erases or amplifies the information originating from individual heads. As a result, the topic token ``France'' stands out in the residual stream. (3) A deep MLP takes ``France'' and generates a component that redirects the residual stream towards the direction of the correct answer, i.e., ``Paris.'' This procedure is akin to applying an implicit function such as ``getcapital(X),'' and the argument X is the topic token information passed by attention heads. To achieve the above quantitative and qualitative analysis for MLPs, we proposed a novel analytic method aimed at decomposing the outputs of the MLP into components understandable by humans. Additionally, we observed a universal anti-overconfidence mechanism in the final layer of models, which suppresses correct predictions. We mitigate this suppression by leveraging our interpretation to improve factual recall confidence. The above interpretations are evaluated across diverse tasks spanning various domains of factual knowledge, using various language models from the GPT-2 families, 1.3B OPT, up to 7B Llama-2, and in both zero- and few-shot setups.