mlsanityData to 7 Oct 2026

Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

arXiv 2410.08414 ↗First posted 10 Oct 2024cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (CK). Can LLMs effectively integrate their internal PK with external CK to solve complex problems? In this paper, we investigate the dynamic interaction between PK and CK, categorizing their relationships into four types: Supportive, Complementary, Conflicting, and Irrelevant. To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge. Our results show that LLMs tend to suppress their PK when contextual information is available, even when it is complementary or irrelevant. While tailored instructions can encourage LLMs to rely more on their PK, they still struggle to fully leverage it. These findings reveal a key vulnerability in LLMs, raising concerns about their reliability in knowledge-intensive tasks. Resources are available at https://github.com/sitaocheng/Knowledge_Interplay

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

44 of 51 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  11. 2024Qwen2 Technical Reportcited by 2,076
  12. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
32 more
  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  4. 2022Mass-Editing Memory in a Transformercited by 976
  5. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  6. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  8. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  9. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  10. 2023Active Retrieval Augmented Generationcited by 506
  11. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  12. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  13. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  14. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  15. 2023Tool Learning with Foundation Modelscited by 302
  16. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  17. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  18. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  19. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  20. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  21. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  22. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  23. 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 78
  24. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  25. 2024Understanding and Patching Compositional Reasoning in LLMscited by 45
  26. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  27. 2023ALCUNA: Large Language Models Meet New Knowledgecited by 35
  28. 2024Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environmentscited by 27
  29. 2024EfficientRAG: Efficient Retriever for Multi-Hop Question Answeringcited by 24
  30. 2024Evaluating the External and Parametric Knowledge Fusion of Large Language Modelscited by 9
  31. 2023MarkQA: A large scale KBQA dataset with numerical reasoningcited by 6
  32. 2024Thread: A Logic-Based Data Organization Paradigm for How-To Question Answering with Retrieval Augmented Generationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understandingcited by 44
  2. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 43
  3. 2025AA-Omniscience: Evaluating Cross-Domain Knowledge Reliability in Large Language Modelscited by 12
  4. 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcementcited by 11
  5. 2025A Survey of Inductive Reasoning for Large Language Modelscited by 8
  6. 2025CryptoX : Compositional Reasoning Evaluation of Large Language Modelscited by 5
  7. 2025BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responsescited by 2
  8. 2025InterpDetect: Interpretable Signals for Detecting Hallucinations in Retrieval-Augmented Generationcited by 1
  9. 2025Differentiable Evolutionary Reinforcement Learningcited by 1
  10. 2026Fine-Tuning Dynamics of In-Context Factual Recall in Transformerscited by 1
  11. 2026CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Trapscited by 1
  12. 2026Large Language Models and Book Summarization: Reading or Remembering, Which Is Better?cited by 0
2 more
  1. 2026CALRK-Bench: Evaluating Context-Aware Legal Reasoning in Korean Lawcited by 0
  2. 2026X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalizationcited by 0