mlsanityData to 7 Oct 2026

AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agents

arXiv 2510.08149 ↗First posted 9 Oct 2025cs.CL cs.AI cs.LGcited by 4

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

also in 1 more
Abstract (arXiv)

The utilization of conversational AI systems by leveraging Retrieval Augmented Generation (RAG) techniques to solve customer problems has been on the rise with the rapid progress of Large Language Models (LLMs). However, the absence of a company-specific dedicated knowledge base is a major barrier to the integration of conversational AI systems in contact centers. To this end, we introduce AI Knowledge Assist, a system that extracts knowledge in the form of question-answer (QA) pairs from historical customer-agent conversations to automatically build a knowledge base. Fine-tuning a lightweight LLM on internal data demonstrates state-of-the-art performance, outperforming larger closed-source LLMs. More specifically, empirical evaluation on 20 companies demonstrates that the proposed AI Knowledge Assist system that leverages the LLaMA-3.1-8B model eliminates the cold-start gap in contact centers by achieving above 90% accuracy in answering information-seeking questions. This enables immediate deployment of RAG-powered chatbots.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 47 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

23 of 47 references.

References · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  7. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  8. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  9. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  10. 2024Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answeringcited by 95
  11. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  12. 2023Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspectivecited by 34
11 more
  1. 2024Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization?cited by 32
  2. 2025Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judgecited by 13
  3. 2024Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarizationcited by 9
  4. 2024Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversationscited by 9
  5. 2024Natural Language Processing for Human Resources: A Surveycited by 9
  6. 2023AI Coach Assist: An Automated Approach for Call Recommendation in Contact Centers for Agent Coachingcited by 8
  7. 2022An Effective, Performant Named Entity Recognition System for Noisy Business Telephone Conversation Transcriptscited by 7
  8. 2022Improving Named Entity Recognition in Telephone Conversations via Effective Active Learning with Human in the Loopcited by 7
  9. 2022DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlogcited by 3
  10. 2025DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarizationcited by 2
  11. 2025Can Post-Training Quantization Benefit from an Additional QLoRA Integration?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agentscited by 4
  2. 2026Beyond Text-to-SQL: An Agentic LLM System for Governed Enterprise Analytics APIscited by 1
  3. 2026When Better Turns Do Not Make Better Agents: Diagnosing the Gap Between Next-Turn Metrics and Workflow Successcited by 1
  4. 2026Can One Adapted Model Do It All? Fine-Tuning Strategy Selection for Customer Support LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Chatty-KG: A Multi-Agent AI System for On-Demand Conversational Question Answering over Knowledge Graphs
  2. 2026From Transcripts to AI Agents: Knowledge Extraction, RAG Integration, and Robust Evaluation of Conversational AI Assistants
  3. 2025Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation
  4. 2025From Documents to Dialogue: Building KG-RAG Enhanced AI Assistants
  5. 2025A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparent
  6. 2025Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems
  7. 2025Graph-Enhanced Retrieval-Augmented Question Answering for E-Commerce Customer Support
  8. 2024A Study on the Implementation Method of an Agent-Based Advanced RAG System Using Graph