To Copy or Not to Copy: Copying Is Easier to Induce Than Recall
On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval
Abstract (arXiv)
Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an arbitration vector from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: CopyrightarrowRecall (suppressing context use) and RecallrightarrowCopy (inducing the model to copy any token from the context). Experiments on three architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy ``reactivation'' process that can be triggered at different locations in the input, while restoring recall is a ``suppression'' process that is more fragile and strongly tied to object-token interventions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 32 references that are in the map. First posted on arXiv 17 Jan 2026.
Builds on
28 of 32 references.
Closest ancestors · 3
Same direction, newest first
- 2024Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models7 Oct
- 2024Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis1 Oct
- 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries18 Jun
Other references · 25
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
- 2023In-Context Retrieval-Augmented Language Modelscited by 694
13 more
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
- 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
- 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
- 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
- 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
- 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
- 2024Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactualscited by 40
- 2024Serial Position Effects of Large Language Modelscited by 30
- 2024Is Your LLM Outdated? A Deep Look at Temporal Generalizationcited by 25
- 2025Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptationcited by 25
- 2024Controllable Context Sensitivity and the Knob Behind Itcited by 22
- 2025Order Matters: Investigate the Position Bias in Multi-constraint Instruction Followingcited by 12
- 2025Positional Biases Shift as Inputs Approach Context Window Limitscited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
- 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
- 2026Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
- 2026Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation
- 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
- 2026The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context
- 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement
- 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models