mlsanityData to 7 Oct 2026

Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse

arXiv 2409.11242 ↗First posted 17 Sep 2024cs.CLcited by 33

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

LLMs are an integral component of retrieval-augmented generation (RAG) systems. While many studies focus on evaluating the overall quality of end-to-end RAG systems, there is a gap in understanding the appropriateness of LLMs for the RAG task. To address this, we introduce Trust-Score, a holistic metric that evaluates the trustworthiness of LLMs within the RAG framework. Our results show that various prompting methods, such as in-context learning, fail to effectively adapt LLMs to the RAG task as measured by Trust-Score. Consequently, we propose Trust-Align, a method to align LLMs for improved Trust-Score performance. 26 out of 27 models aligned using Trust-Align substantially outperform competitive baselines on ASQA, QAMPARI, and ELI5. Specifically, in LLaMA-3-8b, Trust-Align outperforms FRONT on ASQA (up 12.56), QAMPARI (up 36.04), and ELI5 (up 17.69). Trust-Align also significantly enhances models' ability to correctly refuse and provide quality citations. We also demonstrate the effectiveness of Trust-Align across different open-weight models, including the LLaMA series (1b to 8b), Qwen-2.5 series (0.5b to 7b), and Phi3.5 (3.8b). We release our code at https://github.com/declare-lab/trust-align.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 51 of its 69 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

51 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models10 May
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2024The Llama 3 Herd of Modelscited by 14,192
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
37 more
  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  4. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  5. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  6. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  7. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  8. 2019ELI5: Long Form Question Answeringcited by 567
  9. 2023Active Retrieval Augmented Generationcited by 506
  10. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  11. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  12. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  13. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  14. 2022Teaching language models to support answers with verified quotescited by 290
  15. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  16. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  17. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  18. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  19. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  20. 2024Reducing hallucination in structured outputs via Retrieval-Augmented Generationcited by 92
  21. 2023A Survey of Large Language Models Attributioncited by 71
  22. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  23. 2021Adaptive Nearest Neighbor Machine Translationcited by 58
  24. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  25. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 45
  26. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  27. 2024Training Language Models to Generate Text with Citations via Fine-grained Rewardscited by 40
  28. 2024Source-Aware Training Enables Knowledge Attribution in Language Modelscited by 29
  29. 2024Citation-Enhanced Generation for LLM-based Chatbotscited by 24
  30. 2024Improving Attributed Text Generation of Large Language Models via Preference Learningcited by 23
  31. 2024Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generationcited by 21
  32. 2024Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metricscited by 17
  33. 2024ALiiCE: Evaluating Positional Fine-grained Citation Generationcited by 15
  34. 2024Verifiable by Design: Aligning Language Models to Quote from Pre-Training Datacited by 13
  35. 2024Learning Fine-Grained Grounded Citations for Attributed Large Language Modelscited by 12
  36. 2024CaLM: Contrasting Large and Small Language Models to Verify Grounded Generationcited by 11
  37. 2024Attribute or Abstain: Large Language Models as Long Document Assistantscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 20; 2026: 10.

33 papers.

Sort:

By area:

  1. 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
  2. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  3. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  4. 2025Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Rewardcited by 25
  5. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  6. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 13
  7. 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
  8. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  9. 2026OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAGcited by 11
  10. 2025Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systemscited by 9
  11. 2025CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correctioncited by 8
  12. 2025Lessons from Training Grounded LLMs with Verifiable Rewardscited by 8
21 more
  1. 2025Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factualitycited by 8
  2. 2025Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Modelscited by 7
  3. 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidencecited by 6
  4. 2025After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generationcited by 3
  5. 2025Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selectioncited by 2
  6. 2026Efficient Multimodal Planning Agent for Visual Question-Answeringcited by 2
  7. 2024Instance-Level Difficulty: A Missing Perspective in Machine Unlearningcited by 1
  8. 2025InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Groundingcited by 1
  9. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 1
  10. 2026CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answeringcited by 1
  11. 2025Jointly Generating and Attributing Answers using Logits of Document-Identifier Tokenscited by 0
  12. 2025SSFO: Self-Supervised Faithfulness Optimization for Retrieval-Augmented Generationcited by 0
  13. 2025Prior Knowledge Makes It Possible: From Sublinear Graph Algorithms to LLM Test-Time Methodscited by 0
  14. 2025From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMscited by 0
  15. 2025RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generationcited by 0
  16. 2026Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QAcited by 0
  17. 2026Training-Free Refusal of MCP Exploits via Retrieval-Augmented Generationcited by 0
  18. 2026The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systemscited by 0
  19. 2026Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generationcited by 0
  20. 2026Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablationcited by 0
  21. 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Survey
  2. 2024A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems
  3. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey
  4. 2025RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning
  5. 2024Understanding the Impact of Confidence in Retrieval Augmented Generation: A Case Study in the Medical Domain
  6. 2024Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generation
  7. 2024When to Trust LLMs: Aligning Confidence with Response Quality
  8. 2026RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety