# RAG Does Not Work for Enterprises

arXiv: https://arxiv.org/abs/2406.04369 · HTML: https://mlsanity.com/paper/2406.04369/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 0 of its 1 references that are in the map.
- First posted on arXiv 31 May 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Built on by (most cited first, 15 of 15)

- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework](https://mlsanity.com/paper/2408.01262/) (2024)
- [Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments](https://mlsanity.com/paper/2510.27287/) (2025)
- [The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2502.20995/) (2025)
- [Towards Practical GraphRAG: Efficient Knowledge Graph Construction and Hybrid Retrieval at Scale](https://mlsanity.com/paper/2507.03226/) (2025)
- [Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding](https://mlsanity.com/paper/2510.15253/) (2025)
- [RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practice](https://mlsanity.com/paper/2502.14930/) (2025)
- [Position: Foundation Models for Tabular Data within Systemic Contexts Need Grounding](https://mlsanity.com/paper/2505.19825/) (2025)
- [FinBloom: Knowledge Grounding Large Language Model with Real-time Financial Data](https://mlsanity.com/paper/2502.18471/) (2025)
- [Pseudo-Knowledge Graph: Meta-Path Guided Retrieval and In-Graph Text for RAG-Equipped LLM](https://mlsanity.com/paper/2503.00309/) (2025)
- [Data Quality Challenges in Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.00552/) (2025)
- [AI-Driven Climate Policy Scenario Generation for Sub-Saharan Africa](https://mlsanity.com/paper/2505.18694/) (2025)
- [Deep GraphRAG: A Balanced Approach to Hierarchical Retrieval and Adaptive Integration](https://mlsanity.com/paper/2601.11144/) (2026)
- [AI Engineering Blueprint for On-Premises Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2604.01395/) (2026)
- [W-RAG: Source-Aware Retrieval for Enterprise Document Generation from Heterogeneous Knowledge Bases](https://mlsanity.com/paper/2608.22081/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) improves the accuracy and relevance of large language model outputs by incorporating knowledge retrieval. However, implementing RAG in enterprises poses challenges around data security, accuracy, scalability, and integration. This paper explores the unique requirements for enterprise RAG, surveys current approaches and limitations, and discusses potential advances in semantic search, hybrid queries, and optimized retrieval. It proposes an evaluation framework to validate enterprise RAG solutions, including quantitative testing, qualitative analysis, ablation studies, and industry case studies. This framework aims to help demonstrate the ability of purpose-built RAG architectures to deliver accuracy and relevance improvements with enterprise-grade security, compliance and integration. The paper concludes with implications for enterprise deployments, limitations, and future research directions. Close collaboration between researchers and industry partners may accelerate progress in developing and deploying retrieval-augmented generation technology.