KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Maintaining comprehensive and up-to-date knowledge graphs (KGs) is critical for modern AI systems, but manual curation struggles to scale with the rapid growth of scientific literature. This paper presents KARMA, a novel framework employing multi-agent large language models (LLMs) to automate KG enrichment through structured analysis of unstructured text. Our approach employs nine collaborative agents, spanning entity discovery, relation extraction, schema alignment, and conflict resolution that iteratively parse documents, verify extracted knowledge, and integrate it into existing graph structures while adhering to domain-specific schema. Experiments on 1,200 PubMed articles from three different domains demonstrate the effectiveness of KARMA in knowledge graph enrichment, with the identification of up to 38,230 new entities while achieving 83.1% LLM-verified correctness and reducing conflict edges by 18.6% through multi-layer assessments.
Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 33 references that are in the map. First posted on arXiv 10 Feb 2025.
Builds on
17 of 33 references.
References · 17
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
- 2020Self-supervised Learning: Generative or Contrastivecited by 693
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
- 2020Knowledge Graphscited by 619
- 2018AI safety via debatecited by 384
5 more
- 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
- 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
- 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75
- 2025Biomedical Knowledge Graph: A Survey of Domains, Tasks, and Real-World Applicationscited by 6
Built on by
Citing papers per year: 2025: 9; 2026: 17.
26 papers.
Sort:
By area:
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
- 2025LLM-empowered knowledge graph construction: A surveycited by 20
- 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systemscited by 14
- 2025D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Treecited by 7
- 2025RAKG:Document-level Retrieval Augmented Knowledge Graph Constructioncited by 4
- 2025Constructing coherent spatial memory in LLM agents through graph rectificationcited by 2
- 2026HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosiscited by 2
- 2025GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimizationcited by 1
- 2026Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correctioncited by 1
- 2026EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary Worldcited by 1
- 2025medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needscited by 0
- 2025EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledgecited by 0
14 more
- 2026Intent-Driven Smart Manufacturing Integrating Knowledge Graphs and Large Language Modelscited by 0
- 2026LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrievalcited by 0
- 2026Beyond Predefined Schemas: TRACE-KG for Context-Enriched Knowledge Graph Generationcited by 0
- 2026BioResearcher: Scenario-Guided Multi-Agent for Translational Medicinecited by 0
- 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domainscited by 0
- 2026Stage-Audit: Auditable Source-Frontier Discovery for Cross-Wiki Tablescited by 0
- 2026ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoningcited by 0
- 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaborationcited by 0
- 2026Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Managementcited by 0
- 2026From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analyticscited by 0
- 2026VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documentscited by 0
- 2026TRACE: Agentic Catalog Enrichment with Multi-source Evidence Groundingcited by 0
- 2026LLMs Interpret, Embeddings Organize, Graphs Emerge: Agent-Driven Compilation of Scientific Knowledgecited by 0
- 2026Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbookscited by 0
- 2026Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbooks1 Sep 2026
- 2026LLMs Interpret, Embeddings Organize, Graphs Emerge: Agent-Driven Compilation of Scientific Knowledge30 Aug 2026
- 2026TRACE: Agentic Catalog Enrichment with Multi-source Evidence Grounding21 Aug 2026
- 2026From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analytics23 Jul 2026
- 2026EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World19 Jul 2026
- 2026Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Management15 Jun 2026
- 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaboration15 Jun 2026
- 2026Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction27 May 2026
- 2026VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents21 May 2026
- 2026ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning21 May 2026
- 2026Stage-Audit: Auditable Source-Frontier Discovery for Cross-Wiki Tables19 May 2026
- 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains18 May 2026
14 more
- 2026BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine7 May 2026
- 2026Beyond Predefined Schemas: TRACE-KG for Context-Enriched Knowledge Graph Generation3 Apr 2026
- 2026LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrieval10 Mar 2026
- 2026HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosis1 Mar 2026
- 2026Intent-Driven Smart Manufacturing Integrating Knowledge Graphs and Large Language Models12 Feb 2026
- 2025GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimization10 Nov 2025
- 2025LLM-empowered knowledge graph construction: A survey23 Oct 2025
- 2025D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree15 Oct 2025
- 2025Constructing coherent spatial memory in LLM agents through graph rectification5 Oct 2025
- 2025EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge4 Jul 2025
- 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems2 Jul 2025
- 2025medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needs22 Jun 2025
- 2025RAKG:Document-level Retrieval Augmented Knowledge Graph Construction14 Apr 2025
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems31 Mar 2025
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
- 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systemscited by 14
- 2025Constructing coherent spatial memory in LLM agents through graph rectificationcited by 2
- 2026HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosiscited by 2
- 2026BioResearcher: Scenario-Guided Multi-Agent for Translational Medicinecited by 0
- 2026Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Managementcited by 0
- 2026TRACE: Agentic Catalog Enrichment with Multi-source Evidence Groundingcited by 0
- 2026LLMs Interpret, Embeddings Organize, Graphs Emerge: Agent-Driven Compilation of Scientific Knowledgecited by 0
- 2026LLMs Interpret, Embeddings Organize, Graphs Emerge: Agent-Driven Compilation of Scientific Knowledge30 Aug 2026
- 2026TRACE: Agentic Catalog Enrichment with Multi-source Evidence Grounding21 Aug 2026
- 2026Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Management15 Jun 2026
- 2026BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine7 May 2026
- 2026HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosis1 Mar 2026
- 2025Constructing coherent spatial memory in LLM agents through graph rectification5 Oct 2025
- 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems2 Jul 2025
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems31 Mar 2025
- 2025D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Treecited by 7
- 2025RAKG:Document-level Retrieval Augmented Knowledge Graph Constructioncited by 4
- 2025GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimizationcited by 1
- 2026Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correctioncited by 1
- 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domainscited by 0
- 2026ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoningcited by 0
- 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaborationcited by 0
- 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaboration15 Jun 2026
- 2026Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction27 May 2026
- 2026ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning21 May 2026
- 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains18 May 2026
- 2025GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimization10 Nov 2025
- 2025D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree15 Oct 2025
- 2025RAKG:Document-level Retrieval Augmented Knowledge Graph Construction14 Apr 2025
- 2025LLM-empowered knowledge graph construction: A surveycited by 20
- 2025medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needscited by 0
- 2025EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledgecited by 0
- 2026Intent-Driven Smart Manufacturing Integrating Knowledge Graphs and Large Language Modelscited by 0
- 2026LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrievalcited by 0
- 2026Beyond Predefined Schemas: TRACE-KG for Context-Enriched Knowledge Graph Generationcited by 0
- 2026Stage-Audit: Auditable Source-Frontier Discovery for Cross-Wiki Tablescited by 0
- 2026From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analyticscited by 0
- 2026From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analytics23 Jul 2026
- 2026Stage-Audit: Auditable Source-Frontier Discovery for Cross-Wiki Tables19 May 2026
- 2026Beyond Predefined Schemas: TRACE-KG for Context-Enriched Knowledge Graph Generation3 Apr 2026
- 2026LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrieval10 Mar 2026
- 2026Intent-Driven Smart Manufacturing Integrating Knowledge Graphs and Large Language Models12 Feb 2026
- 2025LLM-empowered knowledge graph construction: A survey23 Oct 2025
- 2025EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge4 Jul 2025
- 2025medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needs22 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2023Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction
- 2025AgREE: Agentic Reasoning for Knowledge Graph Completion on Emerging Entities
- 2026Agents-K1: Towards Agent-native Knowledge Orchestration
- 2024KG-RAG: Bridging the Gap Between Knowledge and Creativity
- 2026Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction
- 2026RAGA: Reading-And-Graph-building-Agent for Autonomous Knowledge Graph Construction and Retrieval-Augmented Generation
- 2023Enhancing Knowledge Graph Construction Using Large Language Models
- 2026GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning