mlsanityData to 7 Oct 2026

Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models

arXiv 2402.18154 ↗First posted 28 Feb 2024cs.CL cs.AI cs.IRcited by 87

Not on the map. Why?

Abstract (arXiv)

Recently, retrieval augmentation and tool augmentation have demonstrated a remarkable capability to expand the internal memory boundaries of language models (LMs) by providing external context. However, internal memory and external context inevitably clash, leading to knowledge conflicts within LMs. In this paper, we aim to interpret the mechanism of knowledge conflicts through the lens of information flow, and then mitigate conflicts by precise interventions at the pivotal point. We find there are some attention heads with opposite effects in the later layers, where memory heads can recall knowledge from internal memory, and context heads can retrieve knowledge from external context. Moreover, we reveal that the pivotal point at which knowledge conflicts emerge in LMs is the integration of inconsistent information flows by memory heads and context heads. Inspired by the insights, we propose a novel method called Pruning Head via PatH PatcHing (PH3), which can efficiently mitigate knowledge conflicts by pruning conflicting attention heads without updating model parameters. PH3 can flexibly control eight LMs to use internal memory (uparrow 44.0%) or external context (uparrow 38.5%). Moreover, PH3 can also improve the performance of LMs on open-domain QA tasks. We also conduct extensive experiments to demonstrate the cross-model, cross-relation, and cross-format generalization of our method.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

34 of 49 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2019Are Sixteen Heads Really Better than One?cited by 803
  11. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  12. 2023In-Context Retrieval-Augmented Language Modelscited by 694
22 more
  1. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  2. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  3. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  4. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  5. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  6. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  7. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  8. 2023In-Context Learning Creates Task Vectorscited by 302
  9. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 235
  10. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  11. 2023Localizing Model Behavior with Path Patchingcited by 182
  12. 2022Large Language Models with Controllable Working Memorycited by 169
  13. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  14. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  15. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 124
  16. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 93
  17. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Modelscited by 93
  18. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  19. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  20. 2022Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scalecited by 64
  21. 2023Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanismcited by 21
  22. 2023Bias A-head? Analyzing Bias in Transformer-Based Language Model Attention Headscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 38; 2026: 28.

87 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  2. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoningcited by 96
  3. 2024Attention Heads of Large Language Models: A Surveycited by 65
  4. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  5. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  6. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  7. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  8. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 33
  9. 2025Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfactscited by 30
  10. 2025Taming Knowledge Conflicts in Language Modelscited by 27
  11. 2024Finding Visual Task Vectorscited by 24
  12. 2024Controllable Context Sensitivity and the Knob Behind Itcited by 22
38 more
  1. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  3. 2024Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Modelscited by 21
  4. 2025Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflictcited by 15
  5. 2025RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimalitycited by 14
  6. 2024Correcting Negative Bias in Large Language Models through Negative Attention Score Alignmentcited by 12
  7. 2024OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing Systemcited by 12
  8. 2025ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generationcited by 11
  9. 2025The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentationcited by 11
  10. 2025When Seeing Overrides Knowing: Disentangling Knowledge Conflicts in Vision-Language Modelscited by 11
  11. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11
  12. 2025DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modificationcited by 10
  13. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  14. 2024Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Modelscited by 9
  15. 2024Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliancecited by 9
  16. 2024A Reality Check on Context Utilisation for Retrieval-Augmented Generationcited by 9
  17. 2025Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Modelscited by 9
  18. 2025Focus Directions Make Your Language Models Pay More Attention to Relevant Contextscited by 9
  19. 2025How Do Large Vision-Language Models See Text in Image? Unveiling the Distinctive Role of OCR Headscited by 9
  20. 2024O-Edit: Orthogonal Subspace Editing for Language Model Sequential Editingcited by 8
  21. 2024A Theory for Token-Level Harmonization in Retrieval-Augmented Generationcited by 7
  22. 2025HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Modelscited by 7
  23. 2025Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Modelscited by 7
  24. 2024CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performancecited by 6
  25. 2025Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMscited by 6
  26. 2026Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generationcited by 6
  27. 2024Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patternscited by 5
  28. 2025Stochastic Chameleons: Irrelevant Context Hallucinations Reveal Class-Based (Mis)Generalization in LLMscited by 5
  29. 2026Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Modelscited by 5
  30. 2026ReAttn: Improving Attention-based Re-ranking via Attention Re-weightingcited by 5
  31. 2025Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wildcited by 4
  32. 2025Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generationcited by 4
  33. 2025Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compressioncited by 4
  34. 2025CUB: Benchmarking Context Utilisation Techniques for Language Modelscited by 3
  35. 2025KScope: A Framework for Characterizing the Knowledge Status of Language Modelscited by 3
  36. 2025Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Datacited by 3
  37. 2025AHAMask: Reliable Task Specification for Large Audio Language Models without Instructionscited by 3
  38. 2025Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insightcited by 3

50 most cited of 87 All 87 (JSON)