mlsanityData to 7 Oct 2026

RAGe: A Retrieval-Augmented Generation Evaluation Framework

arXiv 2605.27445 ↗First posted 23 May 2026cs.IR cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Deploying Large Language Model (LLM) applications, particularly those relying on Retrieval-Augmented Generation (RAG), remains challenging due to high computational demands, outdated knowledge bases, and the need to manually select optimal pipeline components. In this work, we propose a modular framework for benchmarking and guiding the efficient development of RAG applications by focusing on resource telemetry and component recommendation, suggesting the best components for a domain-specific dataset. Our approach leverages core techniques in LLM applications, including document chunking, vector databases, embedding models, and retrievers, to evaluate trade-offs among accuracy, efficiency, and scalability. By directly correlating retrieval and generation quality with underlying hardware constraints, RAGe supports researchers to identify the most effective, domain-specific RAG setups for their specific operational needs, facilitating rapid prototyping even on consumer-grade hardware.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 31 of its 38 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

31 of 38 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems25 Feb
  2. 2024XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation20 Dec
  3. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks26 Jul
  4. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track24 Jun
  5. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research22 May
  6. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models10 May
  7. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  10. 2019Passage Re-ranking with BERTcited by 629
  11. 2024The Faiss librarycited by 370
  12. 2016NewsQA: A Machine Comprehension Datasetcited by 330
12 more
  1. 2024Simple and Scalable Strategies to Continually Pre-train Large Language Modelscited by 122
  2. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  3. 2022An Analysis of Fusion Functions for Hybrid Retrievalcited by 49
  4. 2024Is Semantic Chunking Worth the Computational Cost?cited by 43
  5. 2025A Survey on Large Language Models with some Insights on their Capabilities and Limitationscited by 41
  6. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  7. 2023Survey of Vector Database Management Systemscited by 38
  8. 2025From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMappscited by 19
  9. 2025LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Modelscited by 9
  10. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generationcited by 8
  11. 2023Grammatical information in BERT sentence embeddings as two-dimensional arrayscited by 4
  12. 2025Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
  2. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  3. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Survey
  5. 2025Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation
  6. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  7. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
  8. 2024Towards Understanding Systems Trade-offs in Retrieval-Augmented Generation Model Inference