mlsanityData to 14 Oct 2026

Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs

arXiv 2605.27157 ↗First posted 26 May 2026cs.AIcited by 2

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented LLMs are deployed for tasks where evidence quality determines action safety, yet evaluation protocols assume that single-turn robustness predicts robustness when evidence accumulates across turns. We show this assumption is fundamentally incorrect. Models exhibit a monitoring-control gap: they readily acknowledge contradictory evidence, yet this awareness fails to constrain their final recommendations - detecting epistemic conflict does not imply resolving it safely. Through a multi-turn document accumulation protocol across four model families (1.5B-32B parameters) and over 50,000 turn-level evaluations, we demonstrate that single-turn diagnostics systematically overestimate RAG safety, that contradiction acknowledgement is uncorrelated with safe resolution, a pattern corroborated by targeted human validation, and that no universal prompt fix exists. Converging mechanism evidence - hidden-state probing, attention analysis, and response-strategy taxonomy - points to action selection as the most plausible locus of the deficit: danger-relevant information is internally represented and receives enhanced attention during unsafe generation, yet fails to constrain output behavior. The gap between what models recognize and what they do must be measured and closed before retrieval-augmented systems can be trusted in high-stakes settings.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 28 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

27 of 28 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Retrieval-Augmented Generation with Conflicting Evidence17 Apr
  2. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals22 Feb
  3. 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia19 Jun
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 23

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  10. 2022Red Teaming Language Models with Language Modelscited by 1,026
  11. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
11 more
  1. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  2. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  3. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  4. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 267
  5. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  6. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 151
  7. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
  8. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 102
  9. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 67
  10. 2024Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoningcited by 64
  11. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 51

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Reasoning Reduces the Influence of Poisoned Context in RAGcited by 0
  2. 2026Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Don't Blindly Trust It: How Unreliable Feedback Breaks Tool-Using LLM Agents
  2. 2026Detection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelines
  3. 2026The Attribution Blind Spot: Layerwise Trajectory Diagnostics for Source Reliance in Retrieval-Augmented Language Models
  4. 2026Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity
  5. 2026Can LLMs Take Retrieved Information with a Grain of Salt?
  6. 2026Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG
  7. 2025The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness
  8. 2026Auditing Provenance Sensitivity in LLM Agent Action Selection