# CoverageBench: Evaluating Information Coverage across Tasks and Domains

arXiv: https://arxiv.org/abs/2603.20034 · HTML: https://mlsanity.com/paper/2603.20034/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Builds on (most cited first, 14 of 14)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [Rank1: Test-Time Compute for Reranking in Information Retrieval](https://mlsanity.com/paper/2502.18418/) (2025)
- [Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track](https://mlsanity.com/paper/2406.16828/) (2024)
- [Rank-K: Test-Time Reasoning for Listwise Reranking](https://mlsanity.com/paper/2505.14432/) (2025)
- [Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework](https://mlsanity.com/paper/2411.09607/) (2024)
- [Support Evaluation for the TREC 2024 RAG Track: Comparing Human versus LLM Judges](https://mlsanity.com/paper/2504.15205/) (2025)
- [Auto-ARGUE: LLM-Based Report Generation Evaluation](https://mlsanity.com/paper/2509.26184/) (2025)
- [Controlled Retrieval-augmented Context Evaluation for Long-form RAG](https://mlsanity.com/paper/2506.20051/) (2025)
- [Overview of the TREC 2022 Fair Ranking Track](https://mlsanity.com/paper/2302.05558/) (2023)
- [Overview of the TREC 2025 RAGTIME Track](https://mlsanity.com/paper/2602.10024/) (2026)

## Built on by (most cited first, 2 of 2)

- [Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems](https://mlsanity.com/paper/2605.04018/) (2026)
- [Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines](https://mlsanity.com/paper/2609.24122/) (2026)

## Abstract (arXiv)

We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Information coverage is a central aspect for retrieval, especially when the retrieval system is integrated with generative models in a retrieval-augmented generation (RAG) system. The classic metrics for ad hoc retrieval, precision and recall, reward a system as more and more relevant documents are retrieved. However, since relevance in ad hoc test collections is defined for a document without any relation to other documents that might contain the same information, high recall is sufficient but not necessary to ensure coverage. The same is true for other metrics such as rank-biased precision (RBP), normalized discounted cumulative gain (nDCG), and mean average precision (MAP). Test collections developed around the notion of diversity ranking in web search incorporate multiple aspects that support a concept of coverage in the web domain. In this work, we construct a suite of collections for evaluating information coverage from existing collections. This suite offers researchers a unified testbed spanning multiple genres and tasks. All topics, nuggets, relevance labels, and baseline rankings are released on Hugging Face Datasets, along with instructions for accessing the publicly available document collections.