# Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse

arXiv: https://arxiv.org/abs/2409.11242 · HTML: https://mlsanity.com/paper/2409.11242/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 51 of its 69 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 30 of 51)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)

## Built on by (most cited first, 30 of 34)

- [Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning](https://mlsanity.com/paper/2505.14069/) (2025)
- [MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2501.03468/) (2025)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward](https://mlsanity.com/paper/2508.12800/) (2025)
- [RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment](https://mlsanity.com/paper/2412.13746/) (2024)
- [Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG](https://mlsanity.com/paper/2505.20871/) (2025)
- [Credible Plan-Driven RAG Method for Multi-Hop Question Answering](https://mlsanity.com/paper/2504.16787/) (2025)
- [Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.16415/) (2025)
- [OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG](https://mlsanity.com/paper/2601.09028/) (2026)
- [Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2510.13975/) (2025)
- [CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correction](https://mlsanity.com/paper/2504.15629/) (2025)
- [Lessons from Training Grounded LLMs with Verifiable Rewards](https://mlsanity.com/paper/2506.15522/) (2025)
- [Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality](https://mlsanity.com/paper/2506.20978/) (2025)
- [Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Models](https://mlsanity.com/paper/2508.15396/) (2025)
- [GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidence](https://mlsanity.com/paper/2601.04525/) (2026)
- [After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.17118/) (2025)
- [Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selection](https://mlsanity.com/paper/2502.06148/) (2025)
- [Copy-Paste to Mitigate Large Language Model Hallucinations](https://mlsanity.com/paper/2510.00508/) (2025)
- [Efficient Multimodal Planning Agent for Visual Question-Answering](https://mlsanity.com/paper/2601.20676/) (2026)
- [Instance-Level Difficulty: A Missing Perspective in Machine Unlearning](https://mlsanity.com/paper/2410.03043/) (2024)
- [InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding](https://mlsanity.com/paper/2509.16534/) (2025)
- [Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QA](https://mlsanity.com/paper/2604.06173/) (2026)
- [CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answering](https://mlsanity.com/paper/2607.24236/) (2026)
- [Jointly Generating and Attributing Answers using Logits of Document-Identifier Tokens](https://mlsanity.com/paper/2508.08942/) (2025)
- [SSFO: Self-Supervised Faithfulness Optimization for Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.17225/) (2025)
- [Prior Knowledge Makes It Possible: From Sublinear Graph Algorithms to LLM Test-Time Methods](https://mlsanity.com/paper/2510.16609/) (2025)
- [From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs](https://mlsanity.com/paper/2512.16795/) (2025)
- [RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.04196/) (2025)
- [Training-Free Refusal of MCP Exploits via Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.11217/) (2026)
- [The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems](https://mlsanity.com/paper/2607.19292/) (2026)

## Abstract (arXiv)

LLMs are an integral component of retrieval-augmented generation (RAG) systems. While many studies focus on evaluating the overall quality of end-to-end RAG systems, there is a gap in understanding the appropriateness of LLMs for the RAG task. To address this, we introduce Trust-Score, a holistic metric that evaluates the trustworthiness of LLMs within the RAG framework. Our results show that various prompting methods, such as in-context learning, fail to effectively adapt LLMs to the RAG task as measured by Trust-Score. Consequently, we propose Trust-Align, a method to align LLMs for improved Trust-Score performance. 26 out of 27 models aligned using Trust-Align substantially outperform competitive baselines on ASQA, QAMPARI, and ELI5. Specifically, in LLaMA-3-8b, Trust-Align outperforms FRONT on ASQA (up 12.56), QAMPARI (up 36.04), and ELI5 (up 17.69). Trust-Align also significantly enhances models' ability to correctly refuse and provide quality citations. We also demonstrate the effectiveness of Trust-Align across different open-weight models, including the LLaMA series (1b to 8b), Qwen-2.5 series (0.5b to 7b), and Phi3.5 (3.8b). We release our code at https://github.com/declare-lab/trust-align.