mlsanityData to 14 Oct 2026

Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models

arXiv 2503.15888 ↗First posted 20 Mar 2025cs.CL cs.AIcited by 39

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) mitigates hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, conflicts between parametric knowledge and retrieved context pose challenges, particularly when retrieved information is unreliable or the model's internal knowledge is outdated. In such cases, LLMs struggle to determine whether to rely more on their own parameters or the conflicted context. To address this, we propose **CK-PLUG**, a plug-and-play method for controlling LLMs' reliance on parametric and contextual knowledge. We introduce a novel knowledge consistency metric, Confidence Gain, which detects knowledge conflicts by measuring entropy shifts in token probability distributions after context insertion. CK-PLUG then enables fine-grained control over knowledge preference by adjusting the probability distribution of tokens with negative confidence gain through a single tuning parameter. Experiments demonstrate CK-PLUG's ability to significantly regulate knowledge reliance in counterfactual RAG scenarios while maintaining generation fluency and knowledge accuracy. For instance, on Llama3-8B, memory recall (MR) of RAG response can be adjusted within a broad range (9.9%-71.9%), compared to the baseline of 42.1%. Moreover, CK-PLUG supports adaptive control based on the model's confidence in both internal and external knowledge, achieving consistent performance improvements across various general RAG tasks. Our code is available at: hrefhttps://github.com/byronBBL/CK-PLUGthis https URL.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 67 of its 73 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

67 of 73 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models9 Oct
  2. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales19 Jun

Other references · 65

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  12. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
38 more
  1. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  3. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  4. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  5. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  6. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  7. 2019ELI5: Long Form Question Answeringcited by 575
  8. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  9. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  10. 2023Active Retrieval Augmented Generationcited by 516
  11. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  12. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  13. 2023Detecting Language Model Attacks with Perplexitycited by 431
  14. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  15. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  16. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  17. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
  18. 2023Tool Learning with Foundation Modelscited by 302
  19. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  20. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  21. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  22. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
  23. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 256
  24. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  25. 2023Fine-tuning Language Models for Factualitycited by 221
  26. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  27. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  28. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 120
  29. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approachcited by 119
  30. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 107
  31. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  32. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 91
  33. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 83
  34. 2023Context-faithful Prompting for Large Language Modelscited by 80
  35. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 79
  36. 2021Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translationcited by 70
  37. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  38. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 47

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 19.

39 papers.

Sort:

By area:

  1. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
  2. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinkingcited by 20
  3. 2025RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programscited by 17
  4. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 14
  5. 2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13
  6. 2025a1: Steep Test-time Scaling Law via Environment Augmented Generationcited by 13
  7. 2025Pre-training Limited Memory Language Models with Internal and External Knowledgecited by 12
  8. 2025TruthfulRAG: Resolving Factual-level Conflicts in Retrieval-Augmented Generation with Knowledge Graphscited by 12
  9. 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcementcited by 11
  10. 2026What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generationcited by 8
  11. 2025Large Language Models as Computable Approximations to Solomonoff Inductioncited by 6
  12. 2025Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarizationcited by 6
27 more
  1. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  2. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  3. 2025"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Modelscited by 5
  4. 2025Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wildcited by 4
  5. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledgecited by 3
  6. 2025BudgetLeak: Membership Inference Attacks on RAG Systems via the Generation Budget Side Channelcited by 3
  7. 2026PromptCD: Test-Time Behavior Enhancement via Polarity-Prompt Contrastive Decodingcited by 3
  8. 2026SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generationcited by 3
  9. 2025Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generationcited by 2
  10. 2026IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Toolscited by 2
  11. 2025Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglementcited by 1
  12. 2026When Abundance Conceals Weakness: Knowledge Conflict in Multilingual Modelscited by 1
  13. 2026Probing for Knowledge Attribution in Large Language Modelscited by 1
  14. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1
  15. 2026Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflictscited by 1
  16. 2026UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editingcited by 1
  17. 2026CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Trapscited by 1
  18. 2026EvoTrustRAG: Evolution-Aware Conflict Attribution and Evidence Handling for Reliable Retrieval-Augmented Generationcited by 1
  19. 2025ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systemscited by 0
  20. 2025Exposing the Cracks: Vulnerabilities of Retrieval-Augmented LLM-based Machine Translationcited by 0
  21. 2026Can LLMs Take Retrieved Information with a Grain of Salt?cited by 0
  22. 2026On Improving Faithfulness of Podcasts from Documentscited by 0
  23. 2026PURPOSE: Poisoning Conflict Resolution in RAG via Proxy-Fact-Grounded Updatescited by 0
  24. 2026Co-Evolving Structured Knowledge and Reasoning in Language Modelscited by 0
  25. 2026The Answer Path and the Grounding Instruction in LLM Question Answering over Knowledge Graphscited by 0
  26. 2026Parameters vs. Context: TRACE Fine-Tuning for Robust Retrieval-Augmented Generationcited by 0
  27. 2026Which Knowledge Should LLMs Trust? Uncovering a Generalizable Steering Direction Across Knowledge Conflictscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents
  2. 2025Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG
  3. 2024KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models
  4. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
  5. 2024Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models
  6. 2024KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models
  7. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation
  8. 2026Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation