mlsanityData to 7 Oct 2026

Making Large Language Models Perform Better in Knowledge Graph Completion

arXiv 2310.06671 ↗First posted 10 Oct 2023cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) based knowledge graph completion (KGC) aims to predict the missing triples in the KGs with LLMs. However, research about LLM-based KGC fails to sufficiently harness LLMs' inference proficiencies, overlooking critical structural information integral to KGs. In this paper, we explore methods to incorporate structural information into the LLMs, with the overarching goal of facilitating structure-aware reasoning. We first discuss on the existing LLM paradigms like in-context learning and instruction tuning, proposing basic structural information injection approaches. Then we propose a Knowledge Prefix Adapter (KoPA) to fulfill this stated goal. The KoPA uses a structural pre-training phase to comprehend the intricate entities and relations within KGs, representing them as structural embeddings. Then KoPA communicates such cross-modal structural information understanding to the LLMs through a knowledge prefix adapter which projects the structural embeddings into the textual space and obtains virtual knowledge tokens positioned as a prefix of the input prompt. We conduct comprehensive experiments and provide incisive analysis concerning how the introduction of cross-modal structural information would be better for LLM's factual knowledge reasoning ability. Our code and data are available at https://github.com/zjukg/KoPA .

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 51 of its 78 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

51 of 78 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  12. 2023A Survey on Multimodal Large Language Modelscited by 898
38 more
  1. 2016Complex Embeddings for Simple Link Predictioncited by 897
  2. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  3. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  4. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  5. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  6. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  7. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  8. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  9. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 323
  10. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  11. 2023Language is All a Graph Needscited by 219
  12. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  13. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 168
  14. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  15. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 150
  16. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  17. 2022A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodalcited by 106
  18. 2020CoDEx: A Comprehensive Knowledge Graph Completion Benchmarkcited by 102
  19. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  20. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
  21. 2022Sequence-to-Sequence Knowledge Graph Completion and Question Answeringcited by 75
  22. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  23. 2020Relational Message Passing for Knowledge Graph Completioncited by 61
  24. 2022Generative Knowledge Graph Construction: A Reviewcited by 49
  25. 2022RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Groundingcited by 47
  26. 2023Exploring Large Language Models for Knowledge Graph Completioncited by 43
  27. 2023LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPTcited by 42
  28. 2023DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphscited by 40
  29. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysiscited by 35
  30. 2023Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphscited by 35
  31. 2022DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learningcited by 33
  32. 2022Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completioncited by 32
  33. 2021Meta-Knowledge Transfer for Inductive Knowledge Graph Embeddingcited by 31
  34. 2023Iterative Zero-Shot LLM Prompting for Knowledge Graph Constructioncited by 25
  35. 2023Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphscited by 24
  36. 2022LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injectioncited by 18
  37. 2022Tele-Knowledge Pre-training for Fault Analysiscited by 18
  38. 2023Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinesecited by 18

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 22; 2025: 18; 2026: 5.

48 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  2. 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generationcited by 67
  3. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  4. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  5. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  6. 2025TRIX: A More Expressive Model for Zero-shot Domain Transfer in Knowledge Graphscited by 23
  7. 2024MKGL: Mastery of a Three-Word Languagecited by 17
  8. 2023Large Knowledge Model: Perspectives and Challengescited by 12
  9. 2024Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphscited by 10
  10. 2024Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuningcited by 10
  11. 2024Struct-X: Enhancing Large Language Models Reasoning with Structured Datacited by 10
  12. 2024ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphscited by 8
36 more
  1. 2024ConstraintChecker: A Plugin for Large Language Models to Reason on Commonsense Knowledge Basescited by 6
  2. 2024Large Language Model Enhanced Knowledge Representation Learning: A Surveycited by 6
  3. 2025Rethinking Regularization Methods for Knowledge Graph Completioncited by 6
  4. 2023Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Constructioncited by 5
  5. 2024KGValidator: A Framework for Automatic Validation of Knowledge Graph Constructioncited by 5
  6. 2025OKG-LLM: Aligning Ocean Knowledge Graph with Observation Data via LLMs for Global Sea Surface Temperature Predictioncited by 4
  7. 2024Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsificationcited by 3
  8. 2024DRAK: Unlocking Molecular Insights with Domain-Specific Retrieval-Augmented Knowledge in LLMscited by 3
  9. 2024LLM+KG@VLDB'24 Workshop Summarycited by 3
  10. 2025Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluationcited by 3
  11. 2025From Symbolic to Neural and Back: Exploring Knowledge Graph-Large Language Model Synergiescited by 3
  12. 2023ConvD: Attention Enhanced Dynamic Convolutional Embeddings for Knowledge Graph Completioncited by 2
  13. 2024A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Modelscited by 2
  14. 2024A Self-matching Training Method with Annotation Embedding Models for Ontology Subsumption Predictioncited by 2
  15. 2025Large Language Model-Enhanced Symbolic Reasoning for Knowledge Base Completioncited by 2
  16. 2026Federated Personal Knowledge Graph Completion with Lightweight Large Language Models for Personalized Recommendationscited by 2
  17. 2024Temporal Relational Reasoning of Large Language Models for Detecting Stock Portfolio Crashescited by 1
  18. 2024Is Large Language Model Good at Triple Set Prediction? An Empirical Studycited by 1
  19. 2025DiffusionCom: Structure-Aware Multimodal Diffusion Model for Multimodal Knowledge Graph Completioncited by 1
  20. 2025KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Modelscited by 1
  21. 2025Flow-Modulated Scoring for Semantic-Aware Knowledge Graph Completioncited by 1
  22. 2025Ontology-Enhanced Knowledge Graph Completion using Large Language Modelscited by 1
  23. 2025ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Modelscited by 1
  24. 2025Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoningcited by 1
  25. 2024FLAME: Empowering Frozen LLMs for Knowledge Graph Completioncited by 0
  26. 2024Deep Sparse Latent Feature Models for Knowledge Graph Completioncited by 0
  27. 2025FAIRTOPIA: Envisioning Multi-Agent Guardianship for Disrupting Unfair AI Pipelinescited by 0
  28. 2025A Survey of Task-Oriented Knowledge Graph Reasoning: Status, Applications, and Prospectscited by 0
  29. 2025Beyond Prefixes: Graph-as-Memory Cross-Attention for Knowledge Graph Completion with Large Language Modelscited by 0
  30. 2025An Interdisciplinary and Cross-Task Review on Missing Data Imputationcited by 0
  31. 2025Building Specialized Software-Assistant ChatBot with Graph-Based Retrieval-Augmented Generationcited by 0
  32. 2025DialogGraph-LLM: Graph-Informed LLMs for End-to-End Audio Dialogue Intent Recognitioncited by 0
  33. 2026SASA: Semantic-Aware Contrastive Learning Framework with Separated Attention for Triple Classificationcited by 0
  34. 2026OMNIA: Closing the Loop by Leveraging LLMs for Knowledge Graph Completioncited by 0
  35. 2026Conditional Diffusion Guided Knowledge Transfer for Multi-Domain Knowledge Graph Completioncited by 0
  36. 2026Structure-Internalized Rule Language Model for Faithful Knowledge Graph Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning
  2. 2025Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs
  3. 2025Injecting Knowledge Graphs into Large Language Models
  4. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  5. 2025Enhancing Large Language Models with Reliable Knowledge Graphs
  6. 2024GS-KGC: A Generative Subgraph-based Framework for Knowledge Graph Completion with Large Language Models
  7. 2025Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs
  8. 2024Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion