Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs
On the map: Mechanistic interpretability → Activation steering methods
Abstract (arXiv)
In language models, the choice between believing the prompt and believing the weights is made by a handful of identifiable attention heads. Instruction tuning changes how models behave under conflict, but whether it rewires the underlying circuit or merely gates/reweights already present components, remains unknown. We provide the first mechanistic base-vs-instruct comparison of conflict-resolution circuits, across three families (Llama-3.2-3B, Qwen-2.5-3B, Gemma-3-4B). Five independent methods, node and edge attribution, superposition role analysis, causal ablation, and path patching, converge on gating, with the same heads, in the same late-layers, are found to be reweighted rather than replaced with a high node overlap (0.60-0.82). Behaviorally, tuning shifts models toward parametric memory, making instruct models reject a terse counterfactual context far more than base ones, the opposite of a naive user-following expectation. Yet this added skepticism is a factor of framing since it disappears when the same false claim is delivered as a coherent, evidential passage. The robustness that instruction tuning buys against terse injection is therefore real but narrow. More broadly, we believe that because the conflict circuit is preserved rather than rebuilt, interpretability and control tools calibrated on base models should transfer directly to their deployed instruct siblings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 18 of its 31 references that are in the map. First posted on arXiv 22 Sep 2026.
Builds on
18 of 31 references.
References · 18
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
- 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
- 2024Knowledge Conflicts for LLMs: A Surveycited by 283
- 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
- 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 149
- 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 91
- 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 86
- 2023From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuningcited by 56
6 more
- 2025Taming Knowledge Conflicts in Language Modelscited by 30
- 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documentscited by 27
- 2025EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identificationcited by 21
- 2025Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysiscited by 20
- 2025Mechanistic Interpretability as Statistical Estimation: A Variance Analysiscited by 15
- 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Stylecited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Does Instruction Tuning Make LLMs More Consistent?
- 2025Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact
- 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks
- 2026Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs
- 2025Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities
- 2024Learning or Self-aligning? Rethinking Instruction Fine-tuning
- 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias
- 2023Dynamics of Instruction Fine-Tuning for Chinese Large Language Models