mlsanityData to 14 Oct 2026

InspectorRAGet: An Introspection Platform for RAG Evaluation

arXiv 2404.17347 ↗First posted 26 Apr 2024cs.SE cs.HCcited by 3

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Large Language Models (LLM) have become a popular approach for implementing Retrieval Augmented Generation (RAG) systems, and a significant amount of effort has been spent on building good models and metrics. In spite of increased recognition of the need for rigorous evaluation of RAG systems, few tools exist that go beyond the creation of model output and automatic calculation. We present InspectorRAGet, an introspection platform for performing a comprehensive analysis of the quality of RAG system output. InspectorRAGet allows the user to analyze aggregate and instance-level performance of RAG systems, using both human and algorithmic metrics as well as annotator quality. InspectorRAGet is suitable for multiple use cases and is available publicly to the community. A live instance of the platform is available at https://ibm.biz/InspectorRAGet.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 15 of its 27 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

15 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec
  2. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov
  3. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 12

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  5. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  6. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  7. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  8. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 151
  9. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  10. 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36
  11. 2024CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systemscited by 25
  12. 2020A critical analysis of metrics used for measuring progress in artificial intelligencecited by 14

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024RAGProbe: An Automated Approach for Evaluating RAG Applicationscited by 7
  2. 2024Optimizing and Evaluating Enterprise Retrieval-Augmented Generation (RAG): A Content Design Perspectivecited by 7
  3. 2025RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Editscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
  2. 2025A System for Comprehensive Assessment of RAG Frameworks
  3. 2024XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
  4. 2025EncouRAGe: Evaluating RAG Local, Fast, and Reliable
  5. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  6. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
  7. 2025RAG-Stack: Co-Optimizing RAG Quality and Performance From the Vector Database Perspective
  8. 2025RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation