# SKILL: Structured Knowledge Infusion for Large Language Models

arXiv: https://arxiv.org/abs/2205.08184 · HTML: https://mlsanity.com/paper/2205.08184/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 17 May 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries](https://mlsanity.com/paper/2008.09036/) (2020)
- [Entities as Experts: Sparse Memory Access with Entity Supervision](https://mlsanity.com/paper/2004.07202/) (2020)
- [KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering](https://mlsanity.com/paper/2110.04330/) (2021)
- [UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering](https://mlsanity.com/paper/2012.14610/) (2020)
- [Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training](https://mlsanity.com/paper/2010.12688/) (2020)

## Built on by (most cited first, 27 of 27)

- [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023)
- [Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph](https://mlsanity.com/paper/2307.07697/) (2023)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models](https://mlsanity.com/paper/2308.11764/) (2023)
- [Graph Reasoning for Question Answering with Triplet Retrieval](https://mlsanity.com/paper/2305.18742/) (2023)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [Biomedical knowledge graph-optimized prompt generation for large language models](https://mlsanity.com/paper/2311.17330/) (2023)
- [KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph](https://mlsanity.com/paper/2312.15880/) (2023)
- [Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation](https://mlsanity.com/paper/2310.08395/) (2023)
- [Fine-tuning Large Enterprise Language Models via Ontological Reasoning](https://mlsanity.com/paper/2306.10723/) (2023)
- [ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decomposition](https://mlsanity.com/paper/2403.18062/) (2024)
- [KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understanding](https://mlsanity.com/paper/2210.04105/) (2022)
- [Evaluating the External and Parametric Knowledge Fusion of Large Language Models](https://mlsanity.com/paper/2405.19010/) (2024)
- [cTBLS: Augmenting Large Language Models with Conversational Tables](https://mlsanity.com/paper/2303.12024/) (2023)
- [Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving](https://mlsanity.com/paper/2309.16436/) (2023)
- [Insights into Classifying and Mitigating LLMs' Hallucinations](https://mlsanity.com/paper/2311.08117/) (2023)
- [Joint Pre-training and Local Re-training: Transferable Representation Learning on Multi-source Knowledge Graphs](https://mlsanity.com/paper/2306.02679/) (2023)
- [Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers](https://mlsanity.com/paper/2210.11265/) (2022)
- [Addressing Hallucinations with RAG and NMISS in Italian Healthcare LLM Chatbots](https://mlsanity.com/paper/2412.04235/) (2024)
- [LabelPrompt: Effective Prompt-based Learning for Relation Classification](https://mlsanity.com/paper/2302.08068/) (2023)
- [KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering](https://mlsanity.com/paper/2308.03638/) (2023)
- [Human Centered AI for Indian Legal Text Analytics](https://mlsanity.com/paper/2403.10944/) (2024)
- [Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models](https://mlsanity.com/paper/2601.00003/) (2025)
- [Advantages of Domain Knowledge Injection for Legal Document Summarization: A Case Study on Summarizing Indian Court Judgments in English and Hindi](https://mlsanity.com/paper/2602.07382/) (2026)
- [Information for Conversation Generation: Proposals Utilising Knowledge Graphs](https://mlsanity.com/paper/2410.16196/) (2024)
- [PathE: Leveraging Entity-Agnostic Paths for Parameter-Efficient Knowledge Graph Embeddings](https://mlsanity.com/paper/2501.19095/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from text. In this work, we propose a method to infuse structured knowledge into LLMs, by directly training T5 models on factual triples of knowledge graphs (KGs). We show that models pre-trained on Wikidata KG with our method outperform the T5 baselines on FreebaseQA and WikiHop, as well as the Wikidata-answerable subset of TriviaQA and NaturalQuestions. The models pre-trained on factual triples compare competitively with the ones on natural language sentences that contain the same knowledge. Trained on a smaller size KG, WikiMovies, we saw 3x improvement of exact match score on MetaQA task compared to T5 baseline. The proposed method has an advantage that no alignment between the knowledge graph and text corpus is required in curating training data. This makes our method particularly useful when working with industry-scale knowledge graphs.