# Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction

arXiv: https://arxiv.org/abs/2307.01128 · HTML: https://mlsanity.com/paper/2307.01128/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 18 of 18)

- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Language Models are Open Knowledge Graphs](https://mlsanity.com/paper/2010.11967/) (2020)
- [InPars: Data Augmentation for Information Retrieval using Large Language Models](https://mlsanity.com/paper/2202.05144/) (2022)
- [GPT-RE: In-context Learning for Relation Extraction using Large Language Models](https://mlsanity.com/paper/2305.02105/) (2023)
- [Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases](https://mlsanity.com/paper/2009.11564/) (2020)
- [A Survey on Open Information Extraction](https://mlsanity.com/paper/1806.05599/) (2018)
- [Enhancing Knowledge Graph Construction Using Large Language Models](https://mlsanity.com/paper/2305.04676/) (2023)
- [PromptNER: Prompting For Named Entity Recognition](https://mlsanity.com/paper/2305.15444/) (2023)
- [CodeKGC: Code Language Model for Generative Knowledge Graph Construction](https://mlsanity.com/paper/2304.09048/) (2023)
- [VicunaNER: Zero/Few-shot Named Entity Recognition using Vicuna](https://mlsanity.com/paper/2305.03253/) (2023)
- [Controllable Text Generation for Open-Domain Creativity and Fairness](https://mlsanity.com/paper/2209.12099/) (2022)

## Built on by (most cited first, 25 of 25)

- [UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All](https://mlsanity.com/paper/2403.12532/) (2024)
- [Making Large Language Models Perform Better in Knowledge Graph Completion](https://mlsanity.com/paper/2310.06671/) (2023)
- [Retrieval-Augmented Generation with Hierarchical Knowledge](https://mlsanity.com/paper/2503.10150/) (2025)
- [GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluation](https://mlsanity.com/paper/2503.12600/) (2025)
- [Graphusion: A RAG Framework for Knowledge Graph Construction with a Global Perspective](https://mlsanity.com/paper/2410.17600/) (2024)
- [Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?](https://mlsanity.com/paper/2309.17122/) (2023)
- [iText2KG: Incremental Knowledge Graphs Construction Using Large Language Models](https://mlsanity.com/paper/2409.03284/) (2024)
- [Glitter or Gold? Deriving Structured Insights from Sustainability Reports via Large Language Models](https://mlsanity.com/paper/2310.05628/) (2023)
- [Leveraging LLM for Automated Ontology Extraction and Knowledge Graph Generation](https://mlsanity.com/paper/2412.00608/) (2024)
- [Refining Wikidata Taxonomy using Large Language Models](https://mlsanity.com/paper/2409.04056/) (2024)
- [Zero-Shot End-to-End Relation Extraction in Chinese: A Comparative Study of Gemini, LLaMA and ChatGPT](https://mlsanity.com/paper/2502.05694/) (2025)
- [Automating Intervention Discovery from Scientific Literature: A Progressive Ontology Prompting and Dual-LLM Framework](https://mlsanity.com/paper/2409.00054/) (2024)
- [Knowledge Pyramid Construction for Multi-Level Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.21276/) (2024)
- [Enhancing Knowledge Graph Construction: Evaluating with Emphasis on Hallucination, Omission, and Graph Similarity Metrics](https://mlsanity.com/paper/2502.05239/) (2025)
- [Fine-tuning of Large Language Models for Domain-Specific Cybersecurity Knowledge](https://mlsanity.com/paper/2509.25241/) (2025)
- [Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval](https://mlsanity.com/paper/2604.15951/) (2026)
- [Interplay of Semantic Communication and Knowledge Learning](https://mlsanity.com/paper/2402.03339/) (2024)
- [Augmenting Knowledge Graph Hierarchies Using Neural Transformers](https://mlsanity.com/paper/2404.08020/) (2024)
- [Hidden Entity Detection from GitHub Leveraging Large Language Models](https://mlsanity.com/paper/2501.04455/) (2025)
- [Automated Construction of a Knowledge Graph of Nuclear Fusion Energy for Effective Elicitation and Retrieval of Information](https://mlsanity.com/paper/2504.07738/) (2025)
- [HyDRA: A Hybrid-Driven Reasoning Architecture for Verifiable Knowledge Graphs](https://mlsanity.com/paper/2507.15917/) (2025)
- [LEC-KG: An LLM-Embedding Collaborative Framework for Domain-Specific Knowledge Graph Construction -- A Case Study on SDGs](https://mlsanity.com/paper/2602.02090/) (2026)
- [The Semantic Ladder: A Framework for Progressive Formalization of Natural Language Content for Knowledge Graphs and AI Systems](https://mlsanity.com/paper/2603.22136/) (2026)
- [Construction of Knowledge Graph based on Language Model](https://mlsanity.com/paper/2604.19137/) (2026)
- [FineREX: Fine-Tuned NER-RE for Human Smuggling Knowledge Graphs](https://mlsanity.com/paper/2606.19710/) (2026)

## Abstract (arXiv)

In the current digitalization era, capturing and effectively representing knowledge is crucial in most real-world scenarios. In this context, knowledge graphs represent a potent tool for retrieving and organizing a vast amount of information in a properly interconnected and interpretable structure. However, their generation is still challenging and often requires considerable human effort and domain expertise, hampering the scalability and flexibility across different application fields. This paper proposes an innovative knowledge graph generation approach that leverages the potential of the latest generative large language models, such as GPT-3.5, that can address all the main critical issues in knowledge graph building. The approach is conveyed in a pipeline that comprises novel iterative zero-shot and external knowledge-agnostic strategies in the main stages of the generation process. Our unique manifold approach may encompass significant benefits to the scientific community. In particular, the main contribution can be summarized by: (i) an innovative strategy for iteratively prompting large language models to extract relevant components of the final graph; (ii) a zero-shot strategy for each prompt, meaning that there is no need for providing examples for "guiding" the prompt result; (iii) a scalable solution, as the adoption of LLMs avoids the need for any external resources or human expertise. To assess the effectiveness of our proposed model, we performed experiments on a dataset that covered a specific domain. We claim that our proposal is a suitable solution for scalable and versatile knowledge graph construction and may be applied to different and novel contexts.