mlsanityData to 14 Oct 2026

SKILL: Structured Knowledge Infusion for Large Language Models

arXiv 2205.08184 ↗First posted 17 May 2022cs.CL cs.AI cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from text. In this work, we propose a method to infuse structured knowledge into LLMs, by directly training T5 models on factual triples of knowledge graphs (KGs). We show that models pre-trained on Wikidata KG with our method outperform the T5 baselines on FreebaseQA and WikiHop, as well as the Wikidata-answerable subset of TriviaQA and NaturalQuestions. The models pre-trained on factual triples compare competitively with the ones on natural language sentences that contain the same knowledge. Trained on a smaller size KG, WikiMovies, we saw 3x improvement of exact match score on MetaQA task compared to T5 baseline. The proposed method has an advantage that no alignment between the knowledge graph and text corpus is required in curating training data. This makes our method particularly useful when working with industry-scale knowledge graphs.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 17 May 2022.

Builds on

22 of 34 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  8. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  9. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  11. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  12. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
10 more
  1. 2019Knowledge Enhanced Contextual Word Representationscited by 296
  2. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  3. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  4. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  5. 2017Variational Reasoning for Question Answering with Knowledge Graphcited by 170
  6. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  7. 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
  8. 2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 67
  9. 2020UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answeringcited by 61
  10. 2020Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-trainingcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 16; 2024: 6; 2025: 2; 2026: 1.

27 papers.

Sort:

By area:

  1. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
  2. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 238
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  4. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  5. 2023Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Modelscited by 36
  6. 2023Graph Reasoning for Question Answering with Triplet Retrievalcited by 34
  7. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  8. 2023Biomedical knowledge graph-optimized prompt generation for large language modelscited by 31
  9. 2023KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graphcited by 22
  10. 2023Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generationcited by 21
  11. 2023Fine-tuning Large Enterprise Language Models via Ontological Reasoningcited by 14
  12. 2024ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decompositioncited by 14
15 more
  1. 2022KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understandingcited by 9
  2. 2024Evaluating the External and Parametric Knowledge Fusion of Large Language Modelscited by 9
  3. 2023cTBLS: Augmenting Large Language Models with Conversational Tablescited by 7
  4. 2023Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solvingcited by 6
  5. 2023Insights into Classifying and Mitigating LLMs' Hallucinationscited by 6
  6. 2023Joint Pre-training and Local Re-training: Transferable Representation Learning on Multi-source Knowledge Graphscited by 5
  7. 2022Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformerscited by 3
  8. 2024Addressing Hallucinations with RAG and NMISS in Italian Healthcare LLM Chatbotscited by 3
  9. 2023LabelPrompt: Effective Prompt-based Learning for Relation Classificationcited by 1
  10. 2023KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answeringcited by 1
  11. 2024Human Centered AI for Indian Legal Text Analyticscited by 1
  12. 2025Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Modelscited by 1
  13. 2026Advantages of Domain Knowledge Injection for Legal Document Summarization: A Case Study on Summarizing Indian Court Judgments in English and Hindicited by 1
  14. 2024Information for Conversation Generation: Proposals Utilising Knowledge Graphscited by 0
  15. 2025PathE: Leveraging Entity-Agnostic Paths for Parameter-Efficient Knowledge Graph Embeddingscited by 0