# Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction

arXiv: https://arxiv.org/abs/2404.03868 · HTML: https://mlsanity.com/paper/2404.03868/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 58 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Knowledge Graph Convolutional Networks for Recommender Systems](https://mlsanity.com/paper/1904.12575/) (2019)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [Ultra-Fine Entity Typing](https://mlsanity.com/paper/1807.04905/) (2018)
- [A Comprehensive Survey on Automatic Knowledge Graph Construction](https://mlsanity.com/paper/2302.05019/) (2023)
- [Revisiting DocRED -- Addressing the False Negative Problem in Relation Extraction](https://mlsanity.com/paper/2205.12696/) (2022)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [Fine-Grained Entity Typing for Domain Independent Entity Linking](https://mlsanity.com/paper/1909.05780/) (2019)
- [Anaphora and Coreference Resolution: A Review](https://mlsanity.com/paper/1805.11824/) (2018)
- [CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information](https://mlsanity.com/paper/1902.00172/) (2019)
- [CodeKGC: Code Language Model for Generative Knowledge Graph Construction](https://mlsanity.com/paper/2304.09048/) (2023)
- [A Survey on Neural Open Information Extraction: Current Status and Future Directions](https://mlsanity.com/paper/2205.11725/) (2022)
- [Joint Learning of Named Entity Recognition and Entity Linking](https://mlsanity.com/paper/1907.08243/) (2019)
- [LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution](https://mlsanity.com/paper/2205.12644/) (2022)
- [ReGen: Reinforcement Learning for Text and Knowledge Base Generation using Pretrained Language Models](https://mlsanity.com/paper/2108.12472/) (2021)
- [Knowledge Graph Generation From Text](https://mlsanity.com/paper/2211.10511/) (2022)

## Built on by (most cited first, 30 of 70)

- [KGGen: Extracting Knowledge Graphs from Plain Text with Language Models](https://mlsanity.com/paper/2502.09956/) (2025)
- [MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory](https://mlsanity.com/paper/2404.11672/) (2024)
- [AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora](https://mlsanity.com/paper/2505.23628/) (2025)
- [LLM-empowered knowledge graph construction: A survey](https://mlsanity.com/paper/2510.20345/) (2025)
- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications](https://mlsanity.com/paper/2404.14809/) (2024)
- [The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective](https://mlsanity.com/paper/2407.08583/) (2024)
- [Simulating Society Requires Simulating Thought](https://mlsanity.com/paper/2506.06958/) (2025)
- [Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology using Large Language Models -- A Case in Optimizing Intermodal Freight Transportation](https://mlsanity.com/paper/2405.19255/) (2024)
- [Ontology-grounded Automatic Knowledge Graph Construction by LLM under Wikidata schema](https://mlsanity.com/paper/2412.20942/) (2024)
- [Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning](https://mlsanity.com/paper/2501.07845/) (2025)
- [MORTAR: Multi-turn Metamorphic Testing for LLM-based Dialogue Systems](https://mlsanity.com/paper/2412.15557/) (2024)
- [MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph](https://mlsanity.com/paper/2508.12393/) (2025)
- [RAKG:Document-level Retrieval Augmented Knowledge Graph Construction](https://mlsanity.com/paper/2504.09823/) (2025)
- [LKD-KGC: Domain-Specific KG Construction via LLM-driven Knowledge Dependency Parsing](https://mlsanity.com/paper/2505.24163/) (2025)
- [Temporal Reasoning with Large Language Models Augmented by Evolving Knowledge Graphs](https://mlsanity.com/paper/2509.15464/) (2025)
- [Evaluating Multi-Hop Reasoning in Large Language Models: A Chemistry-Centric Case Study](https://mlsanity.com/paper/2504.16414/) (2025)
- [Lie to Me: Knowledge Graphs for Robust Hallucination Self-Detection in LLMs](https://mlsanity.com/paper/2512.23547/) (2025)
- [IE as Cache: Information Extraction Enhanced Agentic Reasoning](https://mlsanity.com/paper/2604.14930/) (2026)
- [Beyond path selection: Better LLMs for Scientific Information Extraction with MimicSFT and Relevance and Rule-induced(R2)GRPO](https://mlsanity.com/paper/2505.22068/) (2025)
- [ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts](https://mlsanity.com/paper/2506.08700/) (2025)
- [VAT-KG: Knowledge-Intensive Multimodal Knowledge Graph Dataset for Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.21556/) (2025)
- [Inside CORE-KG: Evaluating Structured Prompting and Coreference Resolution for Knowledge Graphs](https://mlsanity.com/paper/2510.26512/) (2025)
- [Guideline2Graph: Profile-Aware Multimodal Parsing for Executable Clinical Decision Graphs](https://mlsanity.com/paper/2604.02477/) (2026)
- [Research on the Proximity Relationships of Psychosomatic Disease Knowledge Graph Modules Extracted by Large Language Models](https://mlsanity.com/paper/2412.18419/) (2024)
- [Pseudo-Knowledge Graph: Meta-Path Guided Retrieval and In-Graph Text for RAG-Equipped LLM](https://mlsanity.com/paper/2503.00309/) (2025)
- [Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.24245/) (2025)
- [Relation Extraction Across Entire Books to Reconstruct Community Networks: The AffilKG Datasets](https://mlsanity.com/paper/2505.10798/) (2025)
- [Are Large Language Models Effective Knowledge Graph Constructors?](https://mlsanity.com/paper/2510.11297/) (2025)
- [Ontology-Based Knowledge Graph Framework for Industrial Standard Documents via Hierarchical and Propositional Structuring](https://mlsanity.com/paper/2512.08398/) (2025)

## Abstract (arXiv)

In this work, we are interested in automated methods for knowledge graph creation (KGC) from input text. Progress on large language models (LLMs) has prompted a series of recent works applying them to KGC, e.g., via zero/few-shot prompting. Despite successes on small domain-specific datasets, these models face difficulties scaling up to text common in many real-world applications. A principal issue is that, in prior methods, the KG schema has to be included in the LLM prompt to generate valid triplets; larger and more complex schemas easily exceed the LLMs' context window length. Furthermore, there are scenarios where a fixed pre-defined schema is not available and we would like the method to construct a high-quality KG with a succinct self-generated schema. To address these problems, we propose a three-phase framework named Extract-Define-Canonicalize (EDC): open information extraction followed by schema definition and post-hoc canonicalization. EDC is flexible in that it can be applied to settings where a pre-defined target schema is available and when it is not; in the latter case, it constructs a schema automatically and applies self-canonicalization. To further improve performance, we introduce a trained component that retrieves schema elements relevant to the input text; this improves the LLMs' extraction performance in a retrieval-augmented generation-like manner. We demonstrate on three KGC benchmarks that EDC is able to extract high-quality triplets without any parameter tuning and with significantly larger schemas compared to prior works. Code for EDC is available at https://github.com/clear-nus/edc.