mlsanityData to 14 Oct 2026

Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation

arXiv 2508.14066 ↗First posted 11 Aug 2025cs.IR cs.AIcited by 8

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) is a well-established and rapidly evolving field within AI that enhances the outputs of large language models by integrating relevant information retrieved from external knowledge sources. While industry adoption of RAG is now beginning, there is a significant lack of research on its practical application in industrial contexts. To address this gap, we conducted a semistructured interview study with 13 industry practitioners to explore the current state of RAG adoption in real-world settings. Our study investigates how companies apply RAG in practice, providing (1) an overview of industry use cases, (2) a consolidated list of system requirements, (3) key challenges and lessons learned from practical experiences, and (4) an analysis of current industry evaluation methods. Our main findings show that current RAG applications are mostly limited to domain-specific QA tasks, with systems still in prototype stages; industry requirements focus primarily on data protection, security, and quality, while issues such as ethics, bias, and scalability receive less attention; data preprocessing remains a key challenge, and system evaluation is predominantly conducted by humans rather than automated methods.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 29 of its 50 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

29 of 50 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025EnronQA: Towards Personalized RAG over Private Documents1 May
  2. 2025Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets28 Apr
  3. 2024Towards Optimizing and Evaluating a Retrieval Augmented QA Chatbot using LLMs with Human in the Loop8 Jul
  4. 2024DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation9 Jun
  5. 2024Retrieval-Augmented Generation for AI-Generated Content: A Survey29 Feb
  6. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries27 Jan
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec
  8. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  5. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
  6. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  7. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  8. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 249
  9. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  10. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 119
  11. 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 118
  12. 2023InferFix: End-to-End Program Repair with LLMscited by 114
9 more
  1. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  2. 2023Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Modelscited by 68
  3. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  4. 2023GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidencecited by 14
  5. 2025Privacy-Aware RAG: Secure and Isolated Knowledge Retrievalcited by 10
  6. 2022BashExplainer: Retrieval-Augmented Bash Code Comment Generation based on Fine-tuned CodeBERTcited by 9
  7. 2023Automated Code Editing with Search-Generate-Modifycited by 8
  8. 2024Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systemscited by 7
  9. 2024Evaluation of Semantic Search and its Role in Retrieved-Augmented-Generation (RAG) for Arabic Languagecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  2. 2026Five Queries Are Enough: Query-Efficient and Surrogate-Free Membership Inference Attacks on RAG via Entailmentcited by 4
  3. 2025Vendor-Aware Industrial Agents: RAG-Enhanced LLMs for Secure On-Premise PLC Code Generationcited by 1
  4. 2026Evaluating Multi-Hop Reasoning in RAG Systems: A Comparison of LLM-Based Retriever Evaluation Strategiescited by 1
  5. 2026AI Engineering Blueprint for On-Premises Retrieval-Augmented Generation Systemscited by 0
  6. 2026RAG-DIVE: A Dynamic Approach for Multi-Turn Dialogue Evaluation in Retrieval-Augmented Generationcited by 0
  7. 2026The Commercial Tax: Rent-vs-Own Blind Spots in Multi-Hop Retrieval Benchmarkscited by 0
  8. 2026Automating Multi-Hop RAG Evaluation via TRIAD: From Context Extraction to Validated Dataset Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Deploying Large Language Models With Retrieval Augmented Generation
  2. 2024RAG Does Not Work for Enterprises
  3. 2025An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEs
  4. 2025Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation
  5. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey
  6. 2024Optimizing and Evaluating Enterprise Retrieval-Augmented Generation (RAG): A Content Design Perspective
  7. 2026Pursuing Best Industrial Practices for Retrieval-Augmented Generation in the Medical Domain
  8. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation System