# Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Text arXiv: https://arxiv.org/abs/2305.08804 ยท HTML: https://mlsanity.com/paper/2305.08804/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 21 of its 37 references that are in the map. - First posted on arXiv 15 May 2023. ## Builds on (most cited first, 21 of 21) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023) - [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021) - [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021) - [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017) - [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021) - [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019) - [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019) - [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020) - [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022) - [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019) - [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022) - [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021) - [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020) - [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019) - [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021) - [TALM: Tool Augmented Language Models](https://mlsanity.com/paper/2205.12255/) (2022) ## Built on by (most cited first, 19 of 19) - [Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text](https://mlsanity.com/paper/2308.02357/) (2023) - [ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction](https://mlsanity.com/paper/2310.12537/) (2023) - [Research Trends for the Interplay between Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2406.08223/) (2024) - [Scaling Knowledge Graph Construction through Synthetic Data Generation and Distillation](https://mlsanity.com/paper/2410.16597/) (2024) - [Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Study](https://mlsanity.com/paper/2506.19773/) (2025) - [BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filtering](https://mlsanity.com/paper/2404.03528/) (2024) - [KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction](https://mlsanity.com/paper/2404.15923/) (2024) - [Assessing LLMs Suitability for Knowledge Graph Completion](https://mlsanity.com/paper/2405.17249/) (2024) - [HRGraph: Leveraging LLMs for HR Data Knowledge Graphs with Information Propagation-based Job Recommendation](https://mlsanity.com/paper/2408.13521/) (2024) - [Scholarly Wikidata: Population and Exploration of Conference Data in Wikidata using LLMs](https://mlsanity.com/paper/2411.08696/) (2024) - [LINK-KG: LLM-Driven Coreference-Resolved Knowledge Graphs for Human Smuggling Networks](https://mlsanity.com/paper/2510.26486/) (2025) - [Learning to Select Visual In-Context Demonstrations](https://mlsanity.com/paper/2603.26775/) (2026) - [Enhancing Knowledge Graph Construction: Evaluating with Emphasis on Hallucination, Omission, and Graph Similarity Metrics](https://mlsanity.com/paper/2502.05239/) (2025) - [MaterioMiner -- An ontology-based text mining dataset for extraction of process-structure-property entities](https://mlsanity.com/paper/2408.04661/) (2024) - [Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent](https://mlsanity.com/paper/2508.15243/) (2025) - [Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding](https://mlsanity.com/paper/2511.06376/) (2025) - [Improving Graph Embeddings in Machine Learning Using Knowledge Completion with Validation in a Case Study on COVID-19 Spread](https://mlsanity.com/paper/2511.12071/) (2025) - [PediaMind-R1: A Temperament-Aware Language Model for Personalized Early Childhood Care Reasoning via Cognitive Modeling and Preference Alignment](https://mlsanity.com/paper/2601.08848/) (2025) - [Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models](https://mlsanity.com/paper/2609.17291/) (2026) ## Abstract (arXiv) Knowledge graphs can represent information about the real-world using entities and their relations in a structured and semantically rich manner and they enable a variety of downstream applications such as question-answering, recommendation systems, semantic search, and advanced analytics. However, at the moment, building a knowledge graph involves a lot of manual effort and thus hinders their application in some situations and the automation of this process might benefit especially for small organizations. Automatically generating structured knowledge graphs from a large volume of natural language is still a challenging task and the research on sub-tasks such as named entity extraction, relation extraction, entity and relation linking, and knowledge graph construction aims to improve the state of the art of automatic construction and completion of knowledge graphs from text. The recent advancement of foundation models with billions of parameters trained in a self-supervised manner with large volumes of training data that can be adapted to a variety of downstream tasks has helped to demonstrate high performance on a large range of Natural Language Processing (NLP) tasks. In this context, one emerging paradigm is in-context learning where a language model is used as it is with a prompt that provides instructions and some examples to perform a task without changing the parameters of the model using traditional approaches such as fine-tuning. This way, no computing resources are needed for re-training/fine-tuning the models and the engineering effort is minimal. Thus, it would be beneficial to utilize such capabilities for generating knowledge graphs from text.