# Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

arXiv: https://arxiv.org/abs/2410.08414 · HTML: https://mlsanity.com/paper/2410.08414/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 51 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 30 of 44)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [Tool Learning with Foundation Models](https://mlsanity.com/paper/2304.08354/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://mlsanity.com/paper/2405.14831/) (2024)

## Built on by (most cited first, 15 of 15)

- [Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding](https://mlsanity.com/paper/2502.01563/) (2025)
- [Disentangling Memory and Reasoning Ability in Large Language Models](https://mlsanity.com/paper/2411.13504/) (2024)
- [AA-Omniscience: Evaluating Cross-Domain Knowledge Reliability in Large Language Models](https://mlsanity.com/paper/2511.13029/) (2025)
- [Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement](https://mlsanity.com/paper/2506.05154/) (2025)
- [A Survey of Inductive Reasoning for Large Language Models](https://mlsanity.com/paper/2510.10182/) (2025)
- [CryptoX : Compositional Reasoning Evaluation of Large Language Models](https://mlsanity.com/paper/2502.07813/) (2025)
- [BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses](https://mlsanity.com/paper/2510.00232/) (2025)
- [InterpDetect: Interpretable Signals for Detecting Hallucinations in Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.21538/) (2025)
- [Dual-Evolving Reinforcement Learning for Automated Reward Design](https://mlsanity.com/paper/2512.13399/) (2025)
- [Fine-Tuning Dynamics of In-Context Factual Recall in Transformers](https://mlsanity.com/paper/2605.27774/) (2026)
- [CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps](https://mlsanity.com/paper/2607.10562/) (2026)
- [Large Language Models and Book Summarization: Reading or Remembering, Which Is Better?](https://mlsanity.com/paper/2603.09981/) (2026)
- [CALRK-Bench: Evaluating Context-Aware Legal Reasoning in Korean Law](https://mlsanity.com/paper/2603.26332/) (2026)
- [X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization](https://mlsanity.com/paper/2609.32993/) (2026)
- [Knowledge Interaction in Hypernetwork-based Continual Learning: Conflict and Composition](https://mlsanity.com/paper/2610.15677/) (2026)

## Abstract (arXiv)

Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (CK). Can LLMs effectively integrate their internal PK with external CK to solve complex problems? In this paper, we investigate the dynamic interaction between PK and CK, categorizing their relationships into four types: Supportive, Complementary, Conflicting, and Irrelevant. To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge. Our results show that LLMs tend to suppress their PK when contextual information is available, even when it is complementary or irrelevant. While tailored instructions can encourage LLMs to rely more on their PK, they still struggle to fully leverage it. These findings reveal a key vulnerability in LLMs, raising concerns about their reliability in knowledge-intensive tasks. Resources are available at https://github.com/sitaocheng/Knowledge_Interplay