mlsanityData to 7 Oct 2026

"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generation

arXiv 2605.00957 ↗First posted 1 May 2026cs.IR cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Achieving the right amount of trust in AI systems is important, but challenging. The problem is exacerbated with the rise of Large Language Models (LLMs) as they provide human-level communication capabilities, but potentially hallucinate in the content that they generate. Moreover, they express over-confidence in their answers, making it difficult for users to judge their truthfulness. An important human value that users seek is benevolence, which can be met by LLM's self-reflection leading to reliable and honest answers. Accordingly, this paper proposes conveying appropriate levels of self-reflected certainty to build appropriate trust. Our contributions are twofold: 1) We develop CERTA (Certainty Enhanced RAG for Trustworthy Answers), a specialized Retrieval Augmented Generation (RAG) system that incorporates the relevance between question, context, and answer to reflect its uncertainty in answering questions; 2) We create the Certainty Benchmark with 90 question-context pairs of non-objective questions, divided over four categories (factuality, preference, sycophancy, morality) and three types of contexts (relevant, incomplete, irrelevant). We run experiments with a baseline RAG system and three CERTA settings using two LLMs. Our evaluations indicate that CERTA helps identify answers that are uncertain, decreases the cases of over-agreeing, and provides cautious behavior when prompted for moral judgments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 49 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

23 of 49 references.

References · 23

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  7. 2020Aligning AI With Shared Human Valuescited by 706
  8. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  9. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  10. 2023Large Language Model Alignment: A Surveycited by 241
  11. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  12. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
11 more
  1. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  2. 2024Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanationscited by 30
  3. 2025Revisiting Uncertainty Estimation and Calibration of Large Language Modelscited by 22
  4. 2023A Systematic Review on Fostering Appropriate Trust in Human-AI Interactioncited by 21
  5. 2024SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similaritycited by 18
  6. 2024Flattering to Deceive: The Impact of Sycophantic Behavior on User Trust in Large Language Modelcited by 15
  7. 2025Uncertainty-Aware Fusion: An Ensemble Framework for Mitigating Hallucinations in Large Language Modelscited by 12
  8. 2025RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performancecited by 10
  9. 2021More Similar Values, More Trust? -- the Effect of Value Similarity on Trust in Human-Agent Interactioncited by 7
  10. 2024Value Alignment from Unstructured Textcited by 4
  11. 2025ASTRID -- An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systemscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Trust Me on This: A User Study of Trustworthiness for RAG Responses ↗
  2. 2025After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generation
  3. 2026Trust or Abstain? A Self-Aware RAG Approach
  4. 2026Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs
  5. 2026PA-User: Simulating Trust and Verification under AI-Generated Content ↗
  6. 2024Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning
  7. 2025Query-Level Uncertainty in Large Language Models
  8. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Survey