mlsanityData to 14 Oct 2026

Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement

arXiv 2506.05154 ↗First posted 5 Jun 2025cs.CL cs.AI cs.IRcited by 11

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation (RAG) improves performance on knowledge-intensive tasks but can be derailed by wrong, irrelevant, or conflicting retrieved text, causing models to rely on inaccurate evidence and cascade errors. We propose Knowledgeable-R1, a reinforcement-learning framework that explicitly trains large language models to use parametric knowledge (PK) to resist contextual interference while still exploiting external context when it is reliably helpful. Knowledgeable-R1 introduces a joint sampling scheme that generates paired responses with and without retrieval, and learns both local advantages (within each decoding regime) and global advantages under the same input to quantify when to ignore misleading context versus adopt it. We employ an asymmetric advantage transformation that amplifies exploratory behaviors toward parametric knowledge. Experiments show that Knowledgeable-R1 significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by +22.89% in counterfactual scenarios, and without degradation when the retrieved context is fully accurate.Our code are available at https://github.com/lcy80366872/knowledgeable-R1.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 38 of its 47 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

38 of 47 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Retrieval-Augmented Generation with Conflicting Evidence17 Apr
  2. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models20 Mar
  3. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models9 Oct
  4. 2024Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation8 Oct
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2020Learning to summarize from human feedbackcited by 2,588
  9. 2015Trust Region Policy Optimizationcited by 2,580
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  11. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
21 more
  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  2. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  4. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  7. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  8. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  9. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  10. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  11. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  12. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  13. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  14. 2024ContextCite: Attributing Model Generation to Contextcited by 98
  15. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  16. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  17. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  18. 2025R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learningcited by 30
  19. 2024Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generationcited by 19
  20. 2024Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systemscited by 15
  21. 2024Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Modelscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 10.

11 papers.

  1. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  2. 2026ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Tracescited by 5
  3. 2026Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agentscited by 3
  4. 2026SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generationcited by 3
  5. 2026Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generationcited by 2
  6. 2026Discourse-Role Labels as Presentation-Time Variables for Context Use in Language Modelscited by 1
  7. 2026Document-Authored Control-Signal Impersonation: A Low-Cost Indirect Prompt Attack on RAG Safety Boundariescited by 1
  8. 2026Parametric Knowledge in RAG-SFT for Domain-Specific Document Generationcited by 0
  9. 2026ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chainscited by 0
  10. 2026Parameters vs. Context: TRACE Fine-Tuning for Robust Retrieval-Augmented Generationcited by 0
  11. 2026Regime Boundary Alignment for Evidence-Gated Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents
  2. 2024KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models
  3. 2025UR2: Unify RAG and Reasoning through Reinforcement Learning
  4. 2025R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning
  5. 2025Conflict-Aware Soft Prompting for Retrieval-Augmented Generation
  6. 2024Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models
  7. 2025Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
  8. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries