mlsanityData to 14 Oct 2026

RAGXplain: From Explainable Evaluation to Actionable Guidance of RAG Pipelines

arXiv 2505.13538 ↗First posted 18 May 2025cs.IR cs.AIcited by 4

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems couple large language models with external knowledge, yet most evaluation methods report aggregate scores that reveal whether a pipeline underperforms but not where or why. We introduce RAGXplain, an evaluation framework that translates performance metrics into actionable guidance. RAGXplain structures evaluation around a 'Metric Diamond' connecting user input, retrieved context, generated answer, and (when available) ground truth via six diagnostic dimensions. It uses LLM reasoning to produce natural-language failure-mode explanations and prioritized interventions. Across five QA benchmarks, applying RAGXplain's recommendations in a single human-guided pass consistently improves RAG pipeline performance across multiple metrics. We release RAGXplain as open source to support reproducibility and community adoption.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 44 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

27 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models26 Feb
  2. 2024CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversity16 Oct
  3. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems25 Jun
  4. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research22 May
  5. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov

Other references · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  6. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  7. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  8. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  9. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  10. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  11. 2017Why We Need New Evaluation Metrics for NLGcited by 233
  12. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
10 more
  1. 2023INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedbackcited by 22
  2. 2024RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generationcited by 20
  3. 2018WikiPassageQA: A Benchmark Collection for Research on Non-factoid Answer Passage Retrievalcited by 19
  4. 2025Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systemscited by 9
  5. 2024Evidence Contextualization and Counterfactual Attribution for Conversational QA over Heterogeneous Data with RAG Systemscited by 8
  6. 2024Evaluating Quality of Answers for Retrieval-Augmented Generation: A Strong LLM Is All You Needcited by 7
  7. 2024GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answeringcited by 5
  8. 2024Experimenting with Legal AI Solutions: The Case of Question-Answering for Access to Justicecited by 5
  9. 2023Search-Engine-augmented Dialogue Response Generation with Cheaply Supervised Query Productioncited by 4
  10. 2025Retrieval Augmented Generation Based LLM Evaluation For Protocol State Machine Inference With Chain-of-Thought Reasoningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systemscited by 2
  2. 2026RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answeringcited by 2
  3. 2026Don't Offer What Can't Be Done: Deterministic Executability Gating for LLM Skill Selection at Scalecited by 1
  4. 2026RECTIFY: An Interactive Workbench for Post-Evaluation RAG Diagnosis, Repair, and Verificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation
  2. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  3. 2025RAG+: Enhancing Retrieval-Augmented Generation with Application-Aware Reasoning
  4. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation
  5. 2024RAGProbe: An Automated Approach for Evaluating RAG Applications
  6. 2026RAG-TESTER: Automated End-to-End Testing of Retrieval-Augmented Large Language Models
  7. 2025Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
  8. 2026The RAT: A Unified Bayesian Model for RAG Evaluation