# How Do Language Models Choose Between Context and Memory?

arXiv: https://arxiv.org/abs/2609.00753 · HTML: https://mlsanity.com/paper/2609.00753/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 28 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering](https://mlsanity.com/paper/2410.15999/) (2024)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models](https://mlsanity.com/paper/2508.02087/) (2025)
- [Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals](https://mlsanity.com/paper/2402.11655/) (2024)
- [Language Models use Lookbacks to Track Beliefs](https://mlsanity.com/paper/2505.14685/) (2025)
- [Taming Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2503.10996/) (2025)
- [Representational Analysis of Binding in Language Models](https://mlsanity.com/paper/2409.05448/) (2024)
- [Controllable Context Sensitivity and the Knob Behind It](https://mlsanity.com/paper/2411.07404/) (2024)
- [Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations](https://mlsanity.com/paper/2603.18353/) (2026)
- [A Mechanistic View of Authority Hierarchy in LLM Sycophancy](https://mlsanity.com/paper/2607.00415/) (2026)
- [Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models](https://mlsanity.com/paper/2601.13433/) (2026)
- [Causality is Key for Interpretability Claims to Generalise](https://mlsanity.com/paper/2602.16698/) (2026)
- [Causality neq Invariance: Function and Concept Vectors in LLMs](https://mlsanity.com/paper/2602.22424/) (2026)
- [Cell-Based Representation of Relational Binding in Language Models](https://mlsanity.com/paper/2604.19052/) (2026)
- [ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models](https://mlsanity.com/paper/2601.04131/) (2026)
- [Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models](https://mlsanity.com/paper/2606.12689/) (2026)
- [Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability](https://mlsanity.com/paper/2607.08349/) (2026)
- [SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.27786/) (2026)
- [When Does Activation Steering Change What a Model Computes From?](https://mlsanity.com/paper/2606.29522/) (2026)
- [Patch-Effect Graph Kernels for LLM Interpretability](https://mlsanity.com/paper/2605.06480/) (2026)
- [Decodable but Not Faithful: Coupling Natural-Language Rationales to Programmatic Verifiers](https://mlsanity.com/paper/2606.21678/) (2026)
- [A Mechanistic Lens on Semantic Conflicts: Using Activation Patching to Understand LLM Behavior](https://mlsanity.com/paper/2607.05587/) (2026)
- [Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering](https://mlsanity.com/paper/2607.28906/) (2026)
- [Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages](https://mlsanity.com/paper/2608.04183/) (2026)
- [Steering the Language Axis: From Linear Decodability to Causal Control](https://mlsanity.com/paper/2608.12334/) (2026)

## Abstract (arXiv)

When contextual information conflicts with knowledge stored in model parameters, activation directions can be used to decode and steer which source the model follows. However, successful steering does not establish that the unedited model uses those directions to choose between sources, or that they remain effective across tasks. To test these possibilities, we vary the stated authority of contextual claims while holding their content fixed. We first estimate authority directions from prompts in which context and parametric knowledge agree, then test their causal contribution when the two sources conflict. Interchanging naturally occurring activation values along these directions between matched high- and low-authority prompts reproduces 30--68% of the authority-induced shift in source choice across Qwen, Llama, and OLMo models, whereas matched controls reproduce almost none. We next ask what transfers across tasks: the learned direction versus the activation values exchanged along it. Using a direction learned on another task closed 9% of the source-choice gap, compared with 57% when learned on the task being evaluated. Both interventions exchanged activation values from the evaluated task. In a separate experiment, we kept its learned direction but exchanged values taken from another task, which closed 68% of the gap. These results show that authority-related activation values can causally influence source choice across tasks when inserted along directions learned for the task being evaluated.