mlsanityData to 13 Oct 2026

Prompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG Models

arXiv 2608.22228 ↗First posted 23 Aug 2026cs.IRcited by 1

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Missing and misleading evidence are not the same problem in retrieval-augmented generation (RAG), but prompt-based abstention treats them alike. Models abstain when context is absent, not when it is misleading. We introduce GRAB-RAG (Graded Abstention Benchmark for Retrieval-Augmented Generation), a paired benchmark that tests the same questions across four context conditions (supportive, degraded, missing, and misleading) in Natural Questions and HotpotQA. In the misleading condition, we edit a gold passage to support a wrong answer and place it among other retrieved passages. We test five abstention policies on three small frozen models (3.8B--8B) across two QA benchmarks. Models abstain reliably when evidence is missing, but under explicit abstention prompting still answer 41.6% of misleading questions, with 63% of those answers echoing the planted wrong entity verbatim. Chain-of-thought provides little additional benefit. A generator-side conflict check cuts the rate to 13.3% but discards many correct answers, while an NLI verifier recovers that coverage but fails when parametric memory and the misleading passage agree on the same wrong answer. Prompt-based abstention asks whether context is sufficient, not whether it is correct. Neither verifier closes this gap without trading coverage for safety.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 26 of its 39 references that are in the map. First posted on arXiv 23 Aug 2026.

Builds on

26 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Retrieval-Augmented Generation with Conflicting Evidence17 Apr
  2. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systems9 Nov
  3. 2024Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework24 Sep
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,486
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
  3. 2017On Calibration of Modern Neural Networkscited by 4,902
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,239
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 829
  7. 2017Selective Classification for Deep Neural Networkscited by 825
  8. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 800
  9. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 562
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 508
  11. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 352
  12. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 337
10 more
  1. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 179
  2. 2024What Evidence Do Language Models Find Convincing?cited by 75
  3. 2023Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Methodcited by 51
  4. 2024DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Modelscited by 34
  5. 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
  6. 2024Open Domain Question Answering with Conflicting Contextscited by 18
  7. 2024Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attackscited by 14
  8. 2024A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimatorscited by 13
  9. 2025Why Uncertainty Estimation Methods Fall Short in RAG: An Axiomatic Analysiscited by 13
  10. 2025RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Modelscited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation
  2. 2025Retrieval Augmented Question Answering: When Should LLMs Admit Ignorance?
  3. 2025LLM-Specific Utility for Retrieval-Augmented Generation
  4. 2026Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict
  5. 2026CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation
  6. 2026Why RAGs Hallucinate: Penalty-Aware Evaluation of Retrieval-Augmented Generation Systems with Knowledge-Gap Canaries
  7. 2026Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation
  8. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence