mlsanityData to 7 Oct 2026

Towards Query-Agnostic RAG Evaluation via Query Coverage and Claim Verifiability

arXiv 2608.11238 ↗First posted 31 Jul 2026cs.AIcited by 1

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation improves the factuality of large language models by grounding responses in retrieved evidence, yet existing evaluation frameworks struggle to provide consistent, fine-grained diagnostics across the diverse spectrum of user queries, ranging from close-ended fact-seeking to open-ended explanatory requests. We propose Q-CARE, a query-agnostic and fully reference-free framework that enables fine-grained assessment by decomposing queries into sub-queries and answers into atomic claims. Q-CARE establishes a unified evaluation principle based on query coverage and claim verifiability, yielding coverage-aware retriever metrics (C-Prec@k, C-nDCG@k) and claim-level generator metrics (Completeness, Conciseness, and Verifiableness). On a human-annotated benchmark spanning eight datasets, Q-CARE achieves higher correlation with human judgments than four existing RAG evaluation metrics, including RAGEval and RAGChecker, proving its effectiveness as a reliable, automated evaluation framework. Code and data are publicly available at https://github.com/DISL-Lab/Q-CaRE-COLM-26.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 47 of its 53 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

47 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage20 Oct

Other references · 46

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  11. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
34 more
  1. 2025OpenAI GPT-5 System Cardcited by 910
  2. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  3. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  4. 2019ELI5: Long Form Question Answeringcited by 567
  5. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  6. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  7. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  8. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  9. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  10. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  11. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  12. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  13. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  14. 2024Long-form factuality in large language modelscited by 146
  15. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  16. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  17. 2019Multi-hop Reading Comprehension through Question Decomposition and Rescoringcited by 119
  18. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  19. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  20. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generationcited by 90
  21. 2024FineSurE: Fine-grained Summarization Evaluation using LLMscited by 83
  22. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  23. 2024Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Modelscited by 60
  24. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  25. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53
  26. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  27. 2020PARADE: Passage Representation Aggregation for Document Rerankingcited by 44
  28. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Frameworkcited by 39
  29. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
  30. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  31. 2024Exploring the Best Practices of Query Expansion with Large Language Modelscited by 16
  32. 2024LLM-Assisted Relevance Assessments: When Should We Ask LLMs for Help?cited by 11
  33. 2022Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencodercited by 6
  34. 2024Evaluating D-MERIT of Partial-annotation on Information Retrievalcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Projecting BrowseComp-Plus onto ClimbMix: Toward More Realistic Corpora for Agentic Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
  2. 2026CQC-RAG: Robust Retrieval-Augmented Generation via Cross-Query Consistency
  3. 2025Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
  4. 2026Evaluating RAG Reliability under Clean, Misleading, and Mixed Retrieval
  5. 2025From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
  6. 2025Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality
  7. 2025CCRS: A Zero-Shot LLM-as-a-Judge Framework for Comprehensive RAG Evaluation
  8. 2025QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errors