mlsanityData to 14 Oct 2026

RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation

arXiv 2601.04196 ↗First posted 3 Dec 2025cs.CL cs.IRcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Evaluating Retrieval-Augmented Generation (RAG) systems remains a challenging task: existing metrics often collapse heterogeneous behaviors into single scores and provide little insight into whether errors arise from retrieval,reasoning, or grounding. In this paper, we introduce RAGVUE, a diagnostic and explainable framework for automated, reference-free evaluation of RAG pipelines. RAGVUE decomposes RAG behavior into retrieval quality, answer relevance and completeness, strict claim-level faithfulness, and judge calibration. Each metric includes a structured explanation, making the evaluation process transparent. Our framework supports both manual metric selection and fully automated agentic evaluation. It also provides a Python API, CLI, and a local Streamlit interface for interactive usage. In comparative experiments, RAGVUE surfaces fine-grained failures that existing tools such as RAGAS often overlook. We showcase the full RAGVUE workflow and illustrate how it can be integrated into research pipelines and practical RAG development. The source code and detailed instructions on usage are publicly available on GitHub

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 30 references that are in the map. First posted on arXiv 3 Dec 2025.

Builds on

23 of 30 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey21 Apr
  2. 2025Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models26 Feb
  3. 2025MEMERAG: A Multilingual End-to-End Meta-Evaluation Benchmark for Retrieval Augmented Generation24 Feb
  4. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse17 Sep
  5. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework2 Aug
  6. 2024Evaluation of Retrieval-Augmented Generation: A Survey13 May
  7. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov

Other references · 16

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  6. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  7. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  8. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  9. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  10. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  11. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  12. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
4 more
  1. 2024Can You Trust LLM Judgments? Reliability of LLM-as-a-Judgecited by 56
  2. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 51
  3. 2024CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systemscited by 25
  4. 2025GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluationcited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
  2. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation
  3. 2025RAGXplain: From Explainable Evaluation to Actionable Guidance of RAG Pipelines
  4. 2025RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines
  5. 2026The RAT: A Unified Bayesian Model for RAG Evaluation
  6. 2024InspectorRAGet: An Introspection Platform for RAG Evaluation
  7. 2024XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
  8. 2025RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation