mlsanityData to 14 Oct 2026

ConfRAG: Confidence-Guided Retrieval-Augmenting Generation

arXiv 2506.07309 ↗First posted 8 Jun 2025cs.CLcited by 4

On the map: Retrieval-augmented generation → Hallucination detection in RAG

Abstract (arXiv)

Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with "I am unsure". Two design choices make this training effective: (1) a dampening prompt ("answer only if you are confident") that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 58 of its 77 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

58 of 77 references.

References · 58

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  11. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  12. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
38 more
  1. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  2. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  3. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  4. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  5. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  6. 2024Measuring short-form factuality in large language modelscited by 373
  7. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  8. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  9. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  10. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  11. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
  12. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  13. 2023Fine-tuning Language Models for Factualitycited by 221
  14. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  15. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  16. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  17. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  18. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 168
  19. 2024Long-form factuality in large language modelscited by 155
  20. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  21. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 149
  22. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 138
  23. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
  24. 2018CurriculumNet: Weakly Supervised Learning from Large-Scale Web Imagescited by 126
  25. 2020A Survey of Knowledge-Enhanced Text Generationcited by 120
  26. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 117
  27. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 111
  28. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  29. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 100
  30. 2021Efficient Nearest Neighbor Language Modelscited by 93
  31. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  32. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 90
  33. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  34. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  35. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  36. 2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
  37. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 59
  38. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 56

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025ICPO: Intrinsic Confidence-Driven Group Relative Preference Optimization for Efficient Reinforcement Learningcited by 4
  2. 2026A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Modelscited by 2
  3. 2025PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoningcited by 1
  4. 2026RPRA: Predicting an LLM-Judge for Efficient but Performant Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation
  2. 2025Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning
  3. 2025Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation
  4. 2025QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation
  5. 2025HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generation
  6. 2025Hybrid Retrieval for Hallucination Mitigation in Large Language Models: A Comparative Analysis
  7. 2023"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation
  8. 2026Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)