# Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models

arXiv: https://arxiv.org/abs/2601.09445 · HTML: https://mlsanity.com/paper/2601.09445/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 14 Jan 2026.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Continual Pre-training of Language Models](https://mlsanity.com/paper/2302.03241/) (2023)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering](https://mlsanity.com/paper/2211.05655/) (2022)
- [Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2402.18154/) (2024)
- [Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering](https://mlsanity.com/paper/2410.15999/) (2024)
- [Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method](https://mlsanity.com/paper/2310.17918/) (2023)
- [Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference](https://mlsanity.com/paper/2211.11875/) (2022)
- [Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals](https://mlsanity.com/paper/2402.11655/) (2024)
- [When Large Language Models Meet Vector Databases: A Survey](https://mlsanity.com/paper/2402.01763/) (2024)
- [DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2407.17023/) (2024)
- [Taming Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2503.10996/) (2025)
- [Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs](https://mlsanity.com/paper/2503.01090/) (2025)
- [Commonsense Knowledge Editing Based on Free-Text in LLMs](https://mlsanity.com/paper/2410.23844/) (2024)
- [Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge](https://mlsanity.com/paper/2410.04784/) (2024)
- [Conversational AI as a Catalyst for Informal Learning: An Empirical Large-Scale Study on LLM Use in Everyday Learning](https://mlsanity.com/paper/2506.11789/) (2025)

## Built on by (most cited first, 5 of 5)

- [Right Knowledge, Wrong Answer: Characterizing Parametric Temporal Conflict in Open-Weight Language Models](https://mlsanity.com/paper/2606.20959/) (2026)
- [Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution](https://mlsanity.com/paper/2602.04918/) (2026)
- [Where Concept Erasure Should Occur: Concept-Layer Alignment in Text-to-Video Diffusion Models](https://mlsanity.com/paper/2605.25941/) (2026)
- [CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models](https://mlsanity.com/paper/2609.38810/) (2026)
- [LMs as Task-Specific Knowledge Bases: An Interpretability Analysis](https://mlsanity.com/paper/2606.27237/) (2026)

## Abstract (arXiv)

In language models (LMs), intra-memory knowledge conflict arises when inconsistent information about the same subject is encoded within the model's parametric knowledge. Prior work has primarily focused on resolving conflicts between a model's internal knowledge and external sources, which is known as context-memory knowledge conflict, through approaches such as fine-tuning or knowledge editing, while the understanding of conflicts that arise internally remains largely unexplored. In this work, we design a framework to identify where internal conflicting knowledge is encoded within LMs. We test our framework on four LMs using both synthetic and real-world knowledge conflicts. We find that internal conflicts often arise and are resolved in the final layers across all models, but that interventions are markedly less effective on real-world knowledge conflicts. Targeted attention-head interventions outperform layer-wise ones, and a filtering analysis shows that heads specialized for a single competing fact are far more common in synthetic conflicts, helping explain this gap. Finally, we find no evidence of a single universal circuit for handling knowledge conflict. Instead, our results suggest that distinct circuits may separately encode competing pieces of knowledge, giving rise to conflict. Our results offer a first mechanistic account of intra-memory conflict resolution and highlight a substantial gap between synthetic and real-world settings.