# Are Large Language Models Effective Knowledge Graph Constructors?

arXiv: https://arxiv.org/abs/2510.11297 · HTML: https://mlsanity.com/paper/2510.11297/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 25 of 25)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey](https://mlsanity.com/paper/2311.07914/) (2023)
- [Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction](https://mlsanity.com/paper/2404.03868/) (2024)
- [Trustworthy AI](https://mlsanity.com/paper/2011.02272/) (2020)
- [Commonsense Knowledge Base Completion with Structural and Semantic Context](https://mlsanity.com/paper/1910.02915/) (2019)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [KGGen: Extracting Knowledge Graphs from Plain Text with Language Models](https://mlsanity.com/paper/2502.09956/) (2025)
- [SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graphs](https://mlsanity.com/paper/2410.02811/) (2024)
- [COKE: A Cognitive Knowledge Graph for Machine Theory of Mind](https://mlsanity.com/paper/2305.05390/) (2023)
- [Theory of Mind in Large Language Models: Assessment and Enhancement](https://mlsanity.com/paper/2505.00026/) (2025)
- [Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering](https://mlsanity.com/paper/2410.08085/) (2024)
- [LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environments](https://mlsanity.com/paper/2408.15903/) (2024)
- [DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population](https://mlsanity.com/paper/2201.03335/) (2022)
- [Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling](https://mlsanity.com/paper/2509.17289/) (2025)
- [KGrEaT: A Framework to Evaluate Knowledge Graphs via Downstream Tasks](https://mlsanity.com/paper/2308.10537/) (2023)

## Built on by (most cited first, 1 of 1)

- [PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning](https://mlsanity.com/paper/2609.18861/) (2026)

## Abstract (arXiv)

Knowledge graphs (KGs) are widely used in knowledge-intensive applications, yet it remains unclear how effectively current large language models (LLMs) can construct document-grounded KGs in a zero-shot, schema-free setting without relying on complex task-specific frameworks. We introduce Detail-to-Abstract Hierarchical Knowledge Graph (D2A-HKG) construction framework, which decomposes KG construction into three stages: initial extraction, splitting, and abstraction, and evaluates the resulting graphs from both semantic and structural perspectives. Using seven frontier LLMs, we benchmark zero-shot KG construction on CMW-Lit, a dataset derived from published paediatric research articles on children's mental well-being. CMW-Lit provides a challenging test bed due to its heterogeneous evidence, interconnected factors, and complex, statistically qualified relationships. Our results show that state-of-the-art LLMs can generally produce relevant and document-faithful triples with limited hallucination, while exhibiting substantially different extraction behaviors across the construction stages. These findings provide empirical insight into the strengths and limitations of frontier LLMs for direct knowledge graph construction. We further release CMW-Lit and the resulting knowledge graphs as resources for future research, with the generated graphs providing a strong foundation for expert refinement and downstream knowledge-intensive applications.