mlsanityData to 7 Oct 2026

Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG

arXiv 2605.28044 ↗First posted 27 May 2026cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Cited RAG evaluation often treats visible sources as a grounding signal, but a real, topically relevant citation can still under-warrant the attached wording. We study this diagnostic failure as citation laundering: a related source is presented as warrant for an over-strong claim. We introduce FORCEBENCH, a contrastive stress test for evidence-force calibration. Each item holds a cited passage fixed and pairs an evidence-calibrated claim with a localized force-raised variant across five operational axes: relation, modality, scope, temporal validity, and numeric specificity. A calibrated evaluator should score the evidence-calibrated claim higher. Headline experiments use a fixed, locality-filtered 198-pair evaluation set. A citation-presence sanity check is uninformative by design; token and entity overlap still violate monotonicity on 32.8--36.4% of pairs. Across four reported model judges, standard generic support prompting is insufficient for this force-calibration stress test (aggregate MVR 47.2%), while explicit warrant-strength prompting lowers MVR to 24.5% but remains imperfect. We release the benchmark, prompts, outputs, and plug-in pipeline so citation evaluators can report monotonicity violation rate and force sensitivity alongside conventional support metrics.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

24 of 35 references.

References · 24

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  7. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  8. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  9. 2022Teaching language models to support answers with verified quotescited by 290
  10. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  11. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 166
  12. 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 129
12 more
  1. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  2. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  3. 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
  4. 2025Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMAcited by 23
  5. 2024Retrieval Augmented Generation for Domain-specific Question Answeringcited by 22
  6. 2026Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysiscited by 20
  7. 2025CiteEval: Principle-Driven Citation Evaluation for Source Attributioncited by 19
  8. 2025GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluationcited by 18
  9. 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
  10. 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflectioncited by 12
  11. 2025CATCH: A Modular Cross-domain Adaptive Template with Hookcited by 6
  12. 2025Alleviating Attention Hacking in Discriminative Reward Modeling through Interaction Distillationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidancecited by 12
  2. 2026Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluationcited by 8
  3. 2026DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderationcited by 4
  4. 2026When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agentscited by 3
  5. 2026Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Accesscited by 2
  6. 2026Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elitecited by 1
  7. 2026Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MMEcited by 1
  8. 2026Auditing the Audit: Five Failure Modes in Benchmark-Validity Auditscited by 1
  9. 2026CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answeringcited by 1
  10. 2026Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQLcited by 1
  11. 2026MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignmentcited by 0
  12. 2026Probe Choice Changes Canary-Memorization Verdicts: Three Post-Hoc Disagreement Case Studies in a Text-Dominant LoRA-Tuned Autoregressive Testbedcited by 0
2 more
  1. 2026Exploratory As-Analyzed No-Detection of Culturally-Marked Predicate-Triggered PII Amplification in a Synthetic-English RAG Probe: A Predicate-Resource-Confounded Auditcited by 0
  2. 2026Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspectivecited by 0