# Predicting Retrieval Utility and Answer Quality in Retrieval-Augmented Generation

arXiv: https://arxiv.org/abs/2601.14546 · HTML: https://mlsanity.com/paper/2601.14546/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 31 of its 61 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 30 of 31)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [Simple Entity-Centric Questions Challenge Dense Retrievers](https://mlsanity.com/paper/2109.08535/) (2021)
- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Evaluating Retrieval Quality in Retrieval-Augmented Generation](https://mlsanity.com/paper/2404.13781/) (2024)
- [An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels](https://mlsanity.com/paper/2203.11364/) (2022)
- [DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2407.17023/) (2024)
- [SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction](https://mlsanity.com/paper/2503.01478/) (2025)
- [Are Large Language Models Good at Utility Judgments?](https://mlsanity.com/paper/2403.19216/) (2024)
- [Adaptive Retrieval-Augmented Generation for Conversational Systems](https://mlsanity.com/paper/2407.21712/) (2024)
- [Deep-QPP: A Pairwise Interaction-based Deep Learning Model for Supervised Query Performance Prediction](https://mlsanity.com/paper/2202.07376/) (2022)
- [Explainable Prediction of Text Complexity: The Missing Preliminaries for Text Simplification](https://mlsanity.com/paper/2007.15823/) (2020)
- [Is Relevance Propagated from Retriever to Generator in RAG?](https://mlsanity.com/paper/2502.15025/) (2025)
- [Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA](https://mlsanity.com/paper/2409.15515/) (2024)
- [A Theory for Token-Level Harmonization in Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.00944/) (2024)
- [An Analysis of Variations in the Effectiveness of Query Performance Prediction](https://mlsanity.com/paper/2202.06306/) (2022)
- [A Deep Learning Approach for Selective Relevance Feedback](https://mlsanity.com/paper/2401.11198/) (2024)
- [Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.05220/) (2025)
- [Am I on the Right Track? What Can Predicted Query Performance Tell Us about the Search Behaviour of Agentic RAG](https://mlsanity.com/paper/2507.10411/) (2025)
- [One size doesn't fit all: Predicting the Number of Examples for In-Context Learning](https://mlsanity.com/paper/2403.06402/) (2024)

## Built on by (most cited first, 2 of 2)

- [Predicting the Benefit of Retrieval Augmentation in Open-Domain Question Answering](https://mlsanity.com/paper/2604.07985/) (2026)
- [Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG](https://mlsanity.com/paper/2609.37469/) (2026)

## Abstract (arXiv)

The quality of answers generated by large language models (LLMs) in retrieval-augmented generation (RAG) is largely influenced by the contextual information contained in the retrieved documents. A key challenge for improving RAG is to predict both the utility of retrieved documents -- quantified as the performance gain from using context over generation without context -- and the quality of the final answers in terms of correctness and relevance. In this paper, we define two prediction tasks within RAG. The first is retrieval performance prediction (RPP), which estimates the utility of retrieved documents. The second is generation performance prediction (GPP), which estimates the final answer quality. We hypothesise that in RAG, the topical relevance of retrieved documents correlates with their utility, suggesting that query performance prediction (QPP) approaches can be adapted for RPP and GPP. Beyond these retriever-centric signals, we argue that reader-centric features, such as the LLM's perplexity of the retrieved context conditioned on the input query, can further enhance prediction accuracy for both RPP and GPP. Finally, we propose that features reflecting query-agnostic document quality and readability can also provide useful signals to the predictions. We train linear regression models with the above categories of predictors for both RPP and GPP. Experiments on the Natural Questions (NQ) dataset show that combining predictors from multiple feature categories yields the most accurate estimates of RAG performance.