# From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer

arXiv: https://arxiv.org/abs/2202.02113 · HTML: https://mlsanity.com/paper/2202.02113/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020)
- [Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs](https://mlsanity.com/paper/1906.01195/) (2019)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2110.13715/) (2021)
- [OntoProtein: Protein Pretraining With Gene Ontology Embedding](https://mlsanity.com/paper/2201.11147/) (2022)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [Relation Adversarial Network for Low Resource Knowledge Graph Completion](https://mlsanity.com/paper/1911.03091/) (2019)
- [Ontology-enhanced Prompt-tuning for Few-shot Learning](https://mlsanity.com/paper/2201.11332/) (2022)
- [Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings](https://mlsanity.com/paper/2201.05575/) (2022)

## Built on by (most cited first, 30 of 45)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2410.13080/) (2024)
- [Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting](https://mlsanity.com/paper/2307.01709/) (2023)
- [Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completion](https://mlsanity.com/paper/2209.07299/) (2022)
- [Editing Language Model-based Knowledge Graph Embeddings](https://mlsanity.com/paper/2301.10405/) (2023)
- [Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge](https://mlsanity.com/paper/2402.12352/) (2024)
- [Contextualization Distillation from Large Language Model for Knowledge Graph Completion](https://mlsanity.com/paper/2402.01729/) (2024)
- [Research Trends for the Interplay between Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2406.08223/) (2024)
- [Construction and Applications of Billion-Scale Pre-Trained Multimodal Business Knowledge Graph](https://mlsanity.com/paper/2209.15214/) (2022)
- [A Review on Knowledge Graphs for Healthcare: Resources, Applications, and Promises](https://mlsanity.com/paper/2306.04802/) (2023)
- [Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Models](https://mlsanity.com/paper/2402.01712/) (2024)
- [Hyper-CL: Conditioning Sentence Representations with Hypernetworks](https://mlsanity.com/paper/2403.09490/) (2024)
- [Contrastive Demonstration Tuning for Pre-trained Language Models](https://mlsanity.com/paper/2204.04392/) (2022)
- [Large Language Model Enhanced Knowledge Representation Learning: A Survey](https://mlsanity.com/paper/2407.00936/) (2024)
- [Revisiting Initializing Then Refining: An Incomplete and Missing Graph Imputation Network](https://mlsanity.com/paper/2302.07524/) (2023)
- [Large language models as oracles for instantiating ontologies with domain-specific knowledge](https://mlsanity.com/paper/2404.04108/) (2024)
- [SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.15272/) (2024)
- [LambdaKG: A Library for Pre-trained Language Model-Based Knowledge Graph Embeddings](https://mlsanity.com/paper/2210.00305/) (2022)
- [Finetuning Generative Large Language Models with Discrimination Instructions for Knowledge Graph Completion](https://mlsanity.com/paper/2407.16127/) (2024)
- [KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning](https://mlsanity.com/paper/2409.12865/) (2024)
- [Commonsense Knowledge Salience Evaluation with a Benchmark Dataset in E-commerce](https://mlsanity.com/paper/2205.10843/) (2022)
- [MoCoSA: Momentum Contrast for Knowledge Graph Completion with Structure-Augmented Pre-trained Language Models](https://mlsanity.com/paper/2308.08204/) (2023)
- [Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints](https://mlsanity.com/paper/2406.18085/) (2024)
- [Towards Semantically Enriched Embeddings for Knowledge Graph Completion](https://mlsanity.com/paper/2308.00081/) (2023)
- [Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model](https://mlsanity.com/paper/2312.01837/) (2023)
- [Personalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph](https://mlsanity.com/paper/2406.11943/) (2024)
- [NeocorRAG: Less Irrelevant Information, More Explicit Evidence, and More Effective Recall via Evidence Chains](https://mlsanity.com/paper/2604.27852/) (2026)
- [Multi-view Contrastive Learning for Entity Typing over Knowledge Graphs](https://mlsanity.com/paper/2310.12008/) (2023)

## Abstract (arXiv)

Knowledge graph completion aims to address the problem of extending a KG with missing triples. In this paper, we provide an approach GenKGC, which converts knowledge graph completion to sequence-to-sequence generation task with the pre-trained language model. We further introduce relation-guided demonstration and entity-aware hierarchical decoding for better representation learning and fast inference. Experimental results on three datasets show that our approach can obtain better or comparable performance than baselines and achieve faster inference speed compared with previous methods with pre-trained language models. We also release a new large-scale Chinese knowledge graph dataset AliopenKG500 for research purpose. Code and datasets are available in https://github.com/zjunlp/PromptKG/tree/main/GenKGC.