# RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelines

arXiv: https://arxiv.org/abs/2506.03401 · HTML: https://mlsanity.com/paper/2506.03401/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 12 of 12)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Searching for Best Practices in Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01219/) (2024)
- [From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?](https://mlsanity.com/paper/2308.01990/) (2023)
- [AgentOps: Enabling Observability of LLM Agents](https://mlsanity.com/paper/2411.05285/) (2024)
- [RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing](https://mlsanity.com/paper/2404.19543/) (2024)
- [Unified Language Representation for Question Answering over Text, Tables, and Images](https://mlsanity.com/paper/2306.16762/) (2023)
- [MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images](https://mlsanity.com/paper/2309.04790/) (2023)
- [The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice](https://mlsanity.com/paper/2301.09001/) (2023)
- [Privacy and Copyright Protection in Generative AI: A Lifecycle Perspective](https://mlsanity.com/paper/2311.18252/) (2023)
- [Knowing When to Ask -- Bridging Large Language Models and Data](https://mlsanity.com/paper/2409.13741/) (2024)
- [Blockchain-Enabled Accountability in Data Supply Chain: A Data Bill of Materials Approach](https://mlsanity.com/paper/2408.08536/) (2024)

## Built on by (most cited first, 5 of 5)

- [Everything is Context: Agentic File System Abstraction for Context Engineering](https://mlsanity.com/paper/2512.05470/) (2025)
- [AI Engineering Blueprint for On-Premises Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2604.01395/) (2026)
- [Gypscie: A Cross-Platform AI Artifact Management System](https://mlsanity.com/paper/2604.10311/) (2026)
- [A Dual-Track Framework for Template-Constrained LaTeX Conversion](https://mlsanity.com/paper/2606.23107/) (2026)
- [Beyond Benchmark Scores: How Synthetic and Authentic Query Distributions Diverge in RAG Evaluation](https://mlsanity.com/paper/2609.14579/) (2026)

## Abstract (arXiv)

Recent studies show that 60% of LLM-based compound systems in enterprise environments leverage some form of retrieval-augmented generation (RAG), which enhances the relevance and accuracy of LLM (or other genAI) outputs by retrieving relevant information from external data sources. LLMOps involves the practices and techniques for managing the lifecycle and operations of LLM compound systems in production environments. It supports enhancing LLM systems through continuous operations and feedback evaluation. RAGOps extends LLMOps by incorporating a strong focus on data management to address the continuous changes in external data sources. This necessitates automated methods for evaluating and testing data operations, enhancing retrieval relevance and generation quality. In this paper, we (1) characterize the generic architecture of RAG applications based on the 4+1 model view for describing software architectures, (2) outline the lifecycle of RAG systems, which integrates the management lifecycles of both the LLM and the data, (3) define the key design considerations of RAGOps across different stages of the RAG lifecycle and quality trade-off analyses, (4) highlight the overarching research challenges around RAGOps, and (5) present two use cases of RAG applications and the corresponding RAGOps considerations.