mlsanityData to 14 Oct 2026

FACTS About Building Retrieval Augmented Generation-based Chatbots

arXiv 2407.07858 ↗First posted 10 Jul 2024cs.LG cs.CLcited by 15

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Enterprise chatbots, powered by generative AI, are emerging as key applications to enhance employee productivity. Retrieval Augmented Generation (RAG), Large Language Models (LLMs), and orchestration frameworks like Langchain and Llamaindex are crucial for building these chatbots. However, creating effective enterprise chatbots is challenging and requires meticulous RAG pipeline engineering. This includes fine-tuning embeddings and LLMs, extracting documents from vector databases, rephrasing queries, reranking results, designing prompts, honoring document access controls, providing concise responses, including references, safeguarding personal information, and building orchestration agents. We present a framework for building RAG-based chatbots based on our experience with three NVIDIA chatbots: for IT/HR benefits, financial earnings, and general content. Our contributions are three-fold: introducing the FACTS framework (Freshness, Architectures, Cost, Testing, Security), presenting fifteen RAG pipeline control points, and providing empirical results on accuracy-latency tradeoffs between large and small LLMs. To the best of our knowledge, this is the first paper of its kind that provides a holistic view of the factors as well as solutions for building secure enterprise-grade chatbots."

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 16 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

11 of 16 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documents23 Mar
  2. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation System11 Jan
  3. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 8

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023Active Retrieval Augmented Generationcited by 516
  6. 2023Augmented Language Models: a Surveycited by 403
  7. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  8. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 174

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
  2. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 31
  3. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 30
  4. 2025TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrievalcited by 9
  5. 2025RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practicecited by 6
  6. 2025Generative AI Enhanced Financial Risk Management Information Retrievalcited by 3
  7. 2025RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelinescited by 3
  8. 2025Development of Application-Specific Large Language Models to Facilitate Research Ethics Reviewcited by 1
  9. 2025A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparentcited by 1
  10. 2026Evolutionary Context Search for Automated Skill Acquisitioncited by 1
  11. 2026Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvementcited by 1
  12. 2025Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvementcited by 0
3 more
  1. 2025Systematizing LLM Persona Design: A Four-Quadrant Technical Taxonomy for AI Companion Applicationscited by 0
  2. 2026Induced Numerical Instability: Hidden Costs in Multimodal Large Language Modelscited by 0
  3. 2026A Hierarchical Consistency Framework for Auditing Retrieval-Augmented Generation Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Deploying Large Language Models With Retrieval Augmented Generation
  2. 2024Reinforcement Learning for Optimizing RAG for Domain Chatbots
  3. 2024ERATTA: Extreme RAG for Table To Answers with Large Language Models
  4. 2025Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation
  5. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  6. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems
  7. 2024RAG-Fusion: a New Take on Retrieval-Augmented Generation
  8. 2024A Primer on Generative AI for Telecom: From Theory to Practice