# Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering arXiv: https://arxiv.org/abs/2404.17723 ยท HTML: https://mlsanity.com/paper/2404.17723/ ## Key facts - Cited by 96 papers in the map (data through 14 Oct 2026). - Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation. - Builds on 6 of its 21 references that are in the map. - First posted on arXiv 26 Apr 2024. ## Placed in - Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/) ## Builds on (most cited first, 6 of 6) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022) - [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023) - [Large Language Models on Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2312.02783/) (2023) - [Open Question Answering with Weakly Supervised Embedding Models](https://mlsanity.com/paper/1404.4326/) (2014) - [FoodGPT: A Large Language Model in Food Testing Domain with Incremental Pre-training and Knowledge Graph Prompt](https://mlsanity.com/paper/2308.10173/) (2023) ## Built on by (most cited first, 30 of 96) - [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024) - [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025) - [Knowledge Graph-Guided Retrieval Augmented Generation](https://mlsanity.com/paper/2502.06864/) (2025) - [Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks](https://mlsanity.com/paper/2407.21059/) (2024) - [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025) - [Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities](https://mlsanity.com/paper/2505.20099/) (2025) - [KET-RAG: A Cost-Efficient Multi-Granular Indexing Framework for Graph-RAG](https://mlsanity.com/paper/2502.09304/) (2025) - [Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers](https://mlsanity.com/paper/2506.00054/) (2025) - [Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval](https://mlsanity.com/paper/2410.13765/) (2024) - [AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening](https://mlsanity.com/paper/2504.02870/) (2025) - [Multi-Head RAG: Solving Multi-Aspect Problems with LLMs](https://mlsanity.com/paper/2406.05085/) (2024) - [LeanRAG: Knowledge-Graph-Based Generation with Semantic Aggregation and Hierarchical Retrieval](https://mlsanity.com/paper/2508.10391/) (2025) - [Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization](https://mlsanity.com/paper/2410.02721/) (2024) - [Logical Consistency of Large Language Models in Fact-checking](https://mlsanity.com/paper/2412.16100/) (2024) - [Knowledge Graph-extended Retrieval Augmented Generation for Question Answering](https://mlsanity.com/paper/2504.08893/) (2025) - [The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal States](https://mlsanity.com/paper/2412.17056/) (2024) - [DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue](https://mlsanity.com/paper/2502.13847/) (2025) - [Follow-up Question Generation For Enhanced Patient-Provider Conversations](https://mlsanity.com/paper/2503.17509/) (2025) - [Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs](https://mlsanity.com/paper/2412.07618/) (2024) - [Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.16237/) (2025) - [GraphCompliance: Structure-Aware Regulatory Compliance through Policy-Context Graph Alignment](https://mlsanity.com/paper/2510.26309/) (2025) - [Enhancing RAG Efficiency with Adaptive Context Compression](https://mlsanity.com/paper/2507.22931/) (2025) - [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025) - [GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases](https://mlsanity.com/paper/2504.05478/) (2025) - [On the Fundamental Impossibility of Hallucination Control in Large Language Models](https://mlsanity.com/paper/2506.06382/) (2025) - [SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue](https://mlsanity.com/paper/2602.03548/) (2026) - [Scaling Knowledge Graph Construction through Synthetic Data Generation and Distillation](https://mlsanity.com/paper/2410.16597/) (2024) - [AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agents](https://mlsanity.com/paper/2510.08149/) (2025) - [Confundo: Learning to Generate Robust Poison for Practical RAG Systems](https://mlsanity.com/paper/2602.06616/) (2026) - [PlainQAFact: Retrieval-augmented Factual Consistency Evaluation Metric for Biomedical Plain Language Summarization](https://mlsanity.com/paper/2503.08890/) (2025) ## Abstract (arXiv) In customer service technical support, swiftly and accurately retrieving relevant past issues is critical for efficiently resolving customer inquiries. The conventional retrieval methods in retrieval-augmented generation (RAG) for large language models (LLMs) treat a large corpus of past issue tracking tickets as plain text, ignoring the crucial intra-issue structure and inter-issue relations, which limits performance. We introduce a novel customer service question-answering method that amalgamates RAG with a knowledge graph (KG). Our method constructs a KG from historical issues for use in retrieval, retaining the intra-issue structure and inter-issue relations. During the question-answering phase, our method parses consumer queries and retrieves related sub-graphs from the KG to generate answers. This integration of a KG not only improves retrieval accuracy by preserving customer service structure information but also enhances answering quality by mitigating the effects of text segmentation. Empirical assessments on our benchmark datasets, utilizing key retrieval (MRR, Recall@K, NDCG@K) and text generation (BLEU, ROUGE, METEOR) metrics, reveal that our method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU. Our method has been deployed within LinkedIn's customer service team for approximately six months and has reduced the median per-issue resolution time by 28.6%.