# KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment

arXiv: https://arxiv.org/abs/2502.06472 · HTML: https://mlsanity.com/paper/2502.06472/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 10 Feb 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents](https://mlsanity.com/paper/2306.03314/) (2023)
- [Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks](https://mlsanity.com/paper/2411.04468/) (2024)
- [LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities](https://mlsanity.com/paper/2305.13168/) (2023)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [Biomedical Knowledge Graph: A Survey of Domains, Tasks, and Real-World Applications](https://mlsanity.com/paper/2501.11632/) (2025)

## Built on by (most cited first, 27 of 27)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [LLM-empowered knowledge graph construction: A survey](https://mlsanity.com/paper/2510.20345/) (2025)
- [The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems](https://mlsanity.com/paper/2507.02097/) (2025)
- [D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree](https://mlsanity.com/paper/2510.13363/) (2025)
- [RAKG:Document-level Retrieval Augmented Knowledge Graph Construction](https://mlsanity.com/paper/2504.09823/) (2025)
- [Constructing coherent spatial memory in LLM agents through graph rectification](https://mlsanity.com/paper/2510.04195/) (2025)
- [HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosis](https://mlsanity.com/paper/2603.01121/) (2026)
- [GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimization](https://mlsanity.com/paper/2511.06618/) (2025)
- [Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction](https://mlsanity.com/paper/2605.29168/) (2026)
- [EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World](https://mlsanity.com/paper/2607.17250/) (2026)
- [medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needs](https://mlsanity.com/paper/2506.17959/) (2025)
- [EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge](https://mlsanity.com/paper/2507.03617/) (2025)
- [Intent-Driven Smart Manufacturing Integrating Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2602.12419/) (2026)
- [LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrieval](https://mlsanity.com/paper/2603.26683/) (2026)
- [Beyond Predefined Schemas: TRACE-KG for Context-Enriched Knowledge Graph Generation](https://mlsanity.com/paper/2604.03496/) (2026)
- [BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine](https://mlsanity.com/paper/2605.05985/) (2026)
- [Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains](https://mlsanity.com/paper/2605.18261/) (2026)
- [Stage-Audit: Auditable Source-Frontier Discovery for Cross-Wiki Tables](https://mlsanity.com/paper/2605.20478/) (2026)
- [ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning](https://mlsanity.com/paper/2605.22734/) (2026)
- [AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaboration](https://mlsanity.com/paper/2606.16328/) (2026)
- [Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Management](https://mlsanity.com/paper/2606.17203/) (2026)
- [From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analytics](https://mlsanity.com/paper/2607.21327/) (2026)
- [VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents](https://mlsanity.com/paper/2607.24748/) (2026)
- [TRACE: Agentic Catalog Enrichment with Multi-source Evidence Grounding](https://mlsanity.com/paper/2608.20844/) (2026)
- [LLMs Interpret, Embeddings Organize, Graphs Emerge: Agent-Driven Compilation of Scientific Knowledge](https://mlsanity.com/paper/2608.29612/) (2026)
- [LLM-Based Knowledge Graph Completion Combining Discrete Structural Coding with Similar Entity Information](https://mlsanity.com/paper/2608.30235/) (2026)
- [Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbooks](https://mlsanity.com/paper/2609.00763/) (2026)

## Abstract (arXiv)

Maintaining comprehensive and up-to-date knowledge graphs (KGs) is critical for modern AI systems, but manual curation struggles to scale with the rapid growth of scientific literature. This paper presents KARMA, a novel framework employing multi-agent large language models (LLMs) to automate KG enrichment through structured analysis of unstructured text. Our approach employs nine collaborative agents, spanning entity discovery, relation extraction, schema alignment, and conflict resolution that iteratively parse documents, verify extracted knowledge, and integrate it into existing graph structures while adhering to domain-specific schema. Experiments on 1,200 PubMed articles from three different domains demonstrate the effectiveness of KARMA in knowledge graph enrichment, with the identification of up to 38,230 new entities while achieving 83.1% LLM-verified correctness and reducing conflict edges by 18.6% through multi-layer assessments.