mlsanityData to 7 Oct 2026

DRAGOn: Designing RAG On Periodically Updated Corpus

arXiv 2507.05713 ↗First posted 8 Jul 2025cs.CL cs.AIcited by 5

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

This paper introduces DRAGOn, method to design a RAG benchmark on a regularly updated corpus. It features recent reference datasets, a question generation framework, an automatic evaluation pipeline, and a public leaderboard. Specified reference datasets allow for uniform comparison of RAG systems, while newly generated dataset versions mitigate data leakage and ensure that all models are evaluated on unseen, comparable data. The pipeline for automatic question generation extracts the Knowledge Graph from the text corpus and produces multiple question-answer pairs utilizing modern LLM capabilities. A set of diverse LLM-as-Judge metrics is provided for a comprehensive model evaluation. We used Russian news outlets to form the datasets and demonstrate our methodology. We launch a public leaderboard to track the development of RAG systems and encourage community participation.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 22 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

12 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Evaluation of Retrieval-Augmented Generation: A Survey13 May

Other references · 11

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  5. 2022RealTime QA: What's the Answer Right Now?cited by 239
  6. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  7. 2022Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metricscited by 37
  8. 2024MERA: A Comprehensive LLM Evaluation in Russiancited by 22
  9. 2024The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model designcited by 20
  10. 2024RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialoguescited by 19
  11. 2024Facilitating large language model Russian adaptation with Learned Embedding Propagationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queriescited by 2
  2. 2025ChronoPlay: A Framework for Modeling Dual Dynamics and Authenticity in Game RAG Benchmarkscited by 0
  3. 2025Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novelscited by 0
  4. 2026OCC-RAG: Optimal Cognitive Core for Faithful Question Answeringcited by 0
  5. 2026Little Brains, Big Feats: Exploring Compact Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Automating Multi-Hop RAG Evaluation via TRIAD: From Context Extraction to Validated Dataset Generation
  2. 2024Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systems
  3. 2025Generating Diverse Q&A Benchmarks for RAG Evaluation with DataMorgana
  4. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models
  5. 2025RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition
  6. 2024CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systems
  7. 2024Evaluation of RAG Metrics for Question Answering in the Telecom Domain
  8. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall