mlsanityData to 14 Oct 2026

Deepchecks: Evaluating Retrieval-Augmented Generation (RAG)

arXiv 2605.14488 ↗First posted 14 May 2026cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Large Language Models (LLMs) augmented with Retrieval-Augmented Generation (RAG) techniques are revolutionizing applications across multiple domains, such as healthcare, finance, and customer service. Despite their potential, evaluating RAG systems remains a complex challenge due to the stochastic nature of generated outputs and the intricate interplay between retrieval and generation components. This paper introduces Deepchecks, a comprehensive framework tailored for evaluating RAG applications. Deepchecks' evaluation framework addresses RAG applications evaluation through a multi-faceted approach, root cause analysis and production monitoring. By ensuring alignment with application-specific requirements, Deepchecks framework provides a robust foundation for assessing reliability, relevance, and user satisfaction in RAG systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 13 of its 19 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

13 of 19 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversity16 Oct
  2. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation15 Aug
  3. 2024Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback21 Jun
  4. 2024Generative Information Retrieval Evaluation11 Apr
  5. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 8

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  3. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  4. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  5. 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 118
  6. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
  7. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domaincited by 29
  8. 2024Retrieve, Annotate, Evaluate, Repeat: Leveraging Multimodal LLMs for Large-Scale Product Retrieval Evaluationcited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  2. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
  3. 2024InspectorRAGet: An Introspection Platform for RAG Evaluation
  4. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models
  5. 2024Evaluation of Retrieval-Augmented Generation: A Survey
  6. 2026RAGe: A Retrieval-Augmented Generation Evaluation Framework
  7. 2025RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation
  8. 2025Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation