# RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines

arXiv: https://arxiv.org/abs/2504.13587 · HTML: https://mlsanity.com/paper/2504.13587/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 62 references that are in the map.
- First posted on arXiv 18 Apr 2025.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 22 of 22)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Seven Failure Points When Engineering a Retrieval Augmented Generation System](https://mlsanity.com/paper/2401.05856/) (2024)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG](https://mlsanity.com/paper/2410.05983/) (2024)
- [Interactive Debugging and Steering of Multi-Agent AI Systems](https://mlsanity.com/paper/2503.02068/) (2025)
- [ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models](https://mlsanity.com/paper/1704.01942/) (2017)
- [Visual Analytics for Explainable Deep Learning](https://mlsanity.com/paper/1804.02527/) (2018)
- [Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunications](https://mlsanity.com/paper/2404.15939/) (2024)
- [RAGGED: Towards Informed Design of Scalable and Stable RAG Systems](https://mlsanity.com/paper/2403.09040/) (2024)
- [Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows](https://mlsanity.com/paper/2101.04834/) (2021)
- [Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets](https://mlsanity.com/paper/2310.09985/) (2023)
- [PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines](https://mlsanity.com/paper/2005.00160/) (2020)
- [Symphony: Composing Interactive Interfaces for Machine Learning](https://mlsanity.com/paper/2202.08946/) (2022)
- [FACTS About Building Retrieval Augmented Generation-based Chatbots](https://mlsanity.com/paper/2407.07858/) (2024)
- [RAGProbe: An Automated Approach for Evaluating RAG Applications](https://mlsanity.com/paper/2409.19019/) (2024)
- [Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference](https://mlsanity.com/paper/2404.03085/) (2024)
- [RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practice](https://mlsanity.com/paper/2502.14930/) (2025)

## Built on by (most cited first, 3 of 3)

- [AgentStepper: Interactive Debugging of Software Development Agents](https://mlsanity.com/paper/2602.06593/) (2026)
- [Memory in Large Language Models: Mechanisms, Evaluation and Evolution](https://mlsanity.com/paper/2509.18868/) (2025)
- [RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systems](https://mlsanity.com/paper/2601.12991/) (2026)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) pipelines have become the de-facto approach for building AI assistants with access to external, domain-specific knowledge. Given a user query, RAG pipelines typically first retrieve (R) relevant information from external sources, before invoking a Large Language Model (LLM), augmented (A) with this information, to generate (G) responses. Modern RAG pipelines frequently chain multiple retrieval and generation components, in any order. However, developing effective RAG pipelines is challenging because retrieval and generation components are intertwined, making it hard to identify which component(s) cause errors in the eventual output. The parameters with the greatest impact on output quality often require hours of pre-processing after each change, creating prohibitively slow feedback cycles. To address these challenges, we present RAGGY, a developer tool that combines a Python library of composable RAG primitives with an interactive interface for real-time debugging. We contribute the design and implementation of RAGGY, insights into expert debugging patterns through a qualitative study with 12 engineers, and design implications for future RAG tools that better align with developers' natural workflows.