mlsanityData to 7 Oct 2026

Evaluating the Utility of Grounding Documents with Reference-Free LLM-based Metrics

arXiv 2601.23129 ↗First posted 30 Jan 2026cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval Augmented Generation (RAG)'s success depends on the utility the LLM derives from the content used for grounding. Quantifying content utility does not have a definitive specification and existing metrics ignore model-specific capabilities and/or rely on costly annotations. In this paper, we propose Grounding Generation Utility (GroGU), a model-specific and reference-free metric that defines utility as a function of the downstream LLM's generation confidence based on entropy. Despite having no annotation requirements, GroGU is largely faithful in distinguishing ground-truth documents while capturing nuances ignored by LLM-agnostic metrics. We apply GroGU to train a query-rewriter for RAG by identifying high-utility preference data for Direct Preference Optimization. Experiments show improvements by up to 18.2 points in Mean Reciprocal Rank and up to 9.4 points in answer accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 17 of its 25 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

17 of 25 references.

References · 17

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  5. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  6. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  7. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  8. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  9. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 147
  10. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
  11. 2024MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixturescited by 73
  12. 2023ConvGQR: Generative Query Reformulation for Conversational Searchcited by 59
5 more
  1. 2021Semantic Answer Similarity for Evaluating Question Answering Modelscited by 47
  2. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  3. 2023IterCQR: Iterative Conversational Query Reformulation with Retrieval Guidancecited by 20
  4. 2024Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answerscited by 15
  5. 2024AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignmentcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Redefining Retrieval Evaluation in the Era of LLMs
  2. 2025Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation
  3. 2026Beyond Relevance: Utility-Centric Retrieval in the LLM Era
  4. 2025LLM-Specific Utility for Retrieval-Augmented Generation
  5. 2025SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction
  6. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generation
  7. 2025PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation
  8. 2026LURE-RAG: Lightweight Utility-driven Reranking for Efficient RAG