# RAGGED: Towards Informed Design of Scalable and Stable RAG Systems

arXiv: https://arxiv.org/abs/2403.09040 · HTML: https://mlsanity.com/paper/2403.09040/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 10 of 10)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [UL2: Unifying Language Learning Paradigms](https://mlsanity.com/paper/2205.05131/) (2022)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [The Chronicles of RAG: The Retriever, the Chunk and the Generator](https://mlsanity.com/paper/2401.07883/) (2024)

## Built on by (most cited first, 24 of 24)

- [Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy](https://mlsanity.com/paper/2412.04697/) (2024)
- [BERGEN: A Benchmarking Library for Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01102/) (2024)
- [Context Embeddings for Efficient Answer Generation in RAG](https://mlsanity.com/paper/2407.09252/) (2024)
- [NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodes](https://mlsanity.com/paper/2504.11544/) (2025)
- [Provence: efficient and robust context pruning for retrieval-augmented generation](https://mlsanity.com/paper/2501.16214/) (2025)
- [Enhancing Retrieval-Augmented Generation: A Study of Best Practices](https://mlsanity.com/paper/2501.07391/) (2025)
- [PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation](https://mlsanity.com/paper/2502.20377/) (2025)
- [Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory](https://mlsanity.com/paper/2405.08707/) (2024)
- [RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation](https://mlsanity.com/paper/2408.02545/) (2024)
- [Hybrid Retrieval for Hallucination Mitigation in Large Language Models: A Comparative Analysis](https://mlsanity.com/paper/2504.05324/) (2025)
- [Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation](https://mlsanity.com/paper/2602.03689/) (2026)
- [Toward Optimal Search and Retrieval for RAG](https://mlsanity.com/paper/2411.07396/) (2024)
- [Synthetic Knowledge Ingestion: Towards Knowledge Refinement and Injection for Enhancing Large Language Models](https://mlsanity.com/paper/2410.09629/) (2024)
- [RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines](https://mlsanity.com/paper/2504.13587/) (2025)
- [Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards](https://mlsanity.com/paper/2510.04392/) (2025)
- [Magic Mushroom: A Customizable Benchmark for Fine-grained Analysis of Retrieval Noise Erosion in RAG Systems](https://mlsanity.com/paper/2506.03901/) (2025)
- [PRISM: Fast Online LLM Serving via Scheduling-Memory Co-design](https://mlsanity.com/paper/2605.08581/) (2026)
- [Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluation](https://mlsanity.com/paper/2607.16019/) (2026)
- [Think Inside the Chunk: RegulaRAG for Regulation-Compliant Scenario Generation using LLMs: A Case Study of UN Regulation No. 152](https://mlsanity.com/paper/2608.16394/) (2026)
- [MegaMem: A Retrieval Solution for Ultra-Large Context Windows](https://mlsanity.com/paper/2608.22137/) (2026)
- [Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG](https://mlsanity.com/paper/2608.26379/) (2026)
- [Cross Lingual Transfer in Tulu Legal Comprehension: Script-Dependent Improvement and RAG-Induced Knowledge Conflict](https://mlsanity.com/paper/2608.28645/) (2026)
- [SocialBuddy: Tailoring Search Agent for Social Scenarios](https://mlsanity.com/paper/2609.01641/) (2026)
- [When Retrieval Helps: Selective Retrieval for Single-Turn Mental-Health QA](https://mlsanity.com/paper/2609.03454/) (2026)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) enhances language models by integrating external knowledge, but its effectiveness is highly dependent on system configuration. Improper retrieval settings can degrade performance, making RAG less reliable than closed-book generation. In this work, we introduce RAGGED, a framework for systematically evaluating RAG systems across diverse retriever-reader configurations, retrieval depths, and datasets. Our analysis reveals that reader robustness to noise is the key determinant of RAG stability and scalability. Some readers benefit from increased retrieval depth, while others degrade due to their sensitivity to distracting content. Through large-scale experiments on open-domain, multi-hop, and specialized-domain datasets, we show that retrievers, rerankers, and prompts influence performance but do not fundamentally alter these reader-driven trends. By providing a principled framework and new metrics to assess RAG stability and scalability, RAGGED enables systematic evaluation of retrieval-augmented generation systems, guiding future research on optimizing retrieval depth and model robustness.