mlsanityData to 14 Oct 2026

RAG-KG-IL: A Multi-Agent Hybrid Framework for Reducing Hallucinations and Enhancing LLM Reasoning through RAG and Incremental Knowledge Graph Learning Integration

arXiv 2503.13514 ↗First posted 14 Mar 2025cs.CL cs.AI cs.IRcited by 9

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 2 more
Abstract (arXiv)

This paper presents RAG-KG-IL, a novel multi-agent hybrid framework designed to enhance the reasoning capabilities of Large Language Models (LLMs) by integrating Retrieval-Augmented Generation (RAG) and Knowledge Graphs (KGs) with an Incremental Learning (IL) approach. Despite recent advancements, LLMs still face significant challenges in reasoning with structured data, handling dynamic knowledge evolution, and mitigating hallucinations, particularly in mission-critical domains. Our proposed RAG-KG-IL framework addresses these limitations by employing a multi-agent architecture that enables continuous knowledge updates, integrates structured knowledge, and incorporates autonomous agents for enhanced explainability and reasoning. The framework utilizes RAG to ensure the generated responses are grounded in verifiable information, while KGs provide structured domain knowledge for improved consistency and depth of understanding. The Incremental Learning approach allows for dynamic updates to the knowledge base without full retraining, significantly reducing computational overhead and improving the model's adaptability. We evaluate the framework using real-world case studies involving health-related queries, comparing it to state-of-the-art models like GPT-4o and a RAG-only baseline. Experimental results demonstrate that our approach significantly reduces hallucination rates and improves answer completeness and reasoning accuracy. The results underscore the potential of combining RAG, KGs, and multi-agent systems to create intelligent, adaptable systems capable of real-time knowledge integration and reasoning in complex domains.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 37 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

23 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024How Much Can RAG Help the Reasoning of LLM?3 Oct

Other references · 22

  1. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  2. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  3. 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 315
  4. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
  5. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  6. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  7. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
  8. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 123
  9. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 122
  10. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  11. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  12. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
10 more
  1. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  2. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 56
  3. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verificationcited by 40
  4. 2024Estimating the Hallucination Rate of Generative AIcited by 20
  5. 2024LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-designcited by 18
  6. 2024Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrievalcited by 16
  7. 2023Journey of Hallucination-minimized Generative AI Solutions for Financial Decision Makerscited by 7
  8. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  9. 2024Genetic Approach to Mitigate Hallucination in Generative IRcited by 2
  10. 2024Hallucinations or Attention Misdirection? The Path to Strategic Value Extraction in Business Using Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 3.

9 papers.

  1. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
  2. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
  3. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  4. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  5. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 31
  6. 2025CliCARE: Grounding Large Language Models in Clinical Guidelines for Decision Support over Longitudinal Cancer Electronic Health Recordscited by 4
  7. 2026TRACE: Temporal Rule-Anchored Chain-of-Evidence on Knowledge Graphs for Interpretable Stock Movement Predictioncited by 1
  8. 2025ThinkTank: A Framework for Generalizing Domain-Specific AI Agent Systems into Universal Collaborative Intelligence Platformscited by 0
  9. 2026Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergy
  2. 2025Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning
  3. 2025HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation
  4. 2025Agentic RAG with Knowledge Graphs for Complex Multi-Hop Reasoning in Real-World Applications
  5. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  6. 2025Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent
  7. 2025Agent-UniRAG: A Trainable Open-Source LLM Agent Framework for Unified Retrieval-Augmented Generation Systems
  8. 2025SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework