mlsanityData to 7 Oct 2026

Inductive Relation Prediction by BERT

arXiv 2103.07102 ↗First posted 12 Mar 2021cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

Relation prediction in knowledge graphs is dominated by embedding based methods which mainly focus on the transductive setting. Unfortunately, they are not able to handle inductive learning where unseen entities and relations are present and cannot take advantage of prior knowledge. Furthermore, their inference process is not easily explainable. In this work, we propose an all-in-one solution, called BERTRL (BERT-based Relational Learning), which leverages pre-trained language model and fine-tunes it by taking relation instances and their possible reasoning paths as training samples. BERTRL outperforms the SOTAs in 15 out of 18 cases in both inductive and transductive settings. Meanwhile, it demonstrates strong generalization capability in few-shot learning and is explainable.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 28 of its 39 references that are in the map. First posted on arXiv 12 Mar 2021.

Builds on

28 of 39 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  9. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  10. 2017Convolutional 2D Knowledge Graph Embeddingscited by 729
  11. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  12. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
16 more
  1. 2020Autoregressive Entity Retrievalcited by 333
  2. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 323
  3. 2019TuckER: Tensor Factorization for Knowledge Graph Completioncited by 290
  4. 2017DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoningcited by 265
  5. 2015A Review of Relational Machine Learning for Knowledge Graphscited by 264
  6. 2019Inductive Relation Prediction by Subgraph Reasoningcited by 237
  7. 2019DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphscited by 173
  8. 2016Multiple Instance Learning: A Survey of Problem Characteristics and Applicationscited by 163
  9. 2017Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learningcited by 161
  10. 2017Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approachcited by 102
  11. 2017Knowledge Graph Completion via Complex Tensor Factorizationcited by 80
  12. 2018Logic Attention Based Neighborhood Aggregation for Inductive Knowledge Graph Embeddingcited by 53
  13. 2020Communicative Message Passing for Inductive Relation Reasoningcited by 53
  14. 2021Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphscited by 47
  15. 2018Variational Knowledge Graph Reasoningcited by 27
  16. 2020Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptionscited by 7

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 12; 2024: 9; 2025: 1.

27 papers.

Sort:

By area:

  1. 2022A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodalcited by 106
  2. 2022SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Modelscited by 90
  3. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  4. 2023InGram: Inductive Knowledge Graph Embedding via Relation Graphscited by 50
  5. 2023Graph Reasoning for Question Answering with Triplet Retrievalcited by 34
  6. 2021Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Surveycited by 23
  7. 2023Double Equivariance for Inductive Link Prediction for Both New Nodes and New Relation Typescited by 21
  8. 2022KGLM: Integrating Knowledge Graph Structure in Language Models for Link Predictioncited by 15
  9. 2023Generalizing to Unseen Elements: A Survey on Knowledge Extrapolation for Knowledge Graphscited by 14
  10. 2023Knowledge Graph Embedding: An Overviewcited by 10
  11. 2024Large Language Models Can Better Understand Knowledge Graphs Than We Thoughtcited by 10
  12. 2023Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Predictioncited by 9
15 more
  1. 2023On the Evolution of Knowledge Graphs: A Survey and Perspectivecited by 9
  2. 2024Negative Sampling in Knowledge Graph Representation Learning: A Reviewcited by 9
  3. 2023A Survey On Few-shot Knowledge Graph Completion with Structural and Commonsense Knowledgecited by 6
  4. 2023Multi-Aspect Explainable Inductive Relation Prediction by Sentence Transformercited by 5
  5. 2023Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphscited by 3
  6. 2023Relation-Aware Network with Attention-Based Loss for Few-Shot Knowledge Graph Completioncited by 2
  7. 2024Textual Entailment for Effective Triple Validation in Object Predictioncited by 2
  8. 2024Certifying Knowledge Comprehension in LLMscited by 1
  9. 2024Knowledge Graph Completion using Structural and Textual Embeddingscited by 1
  10. 2024Predicting from a Different Perspective: A Re-ranking Model for Inductive Knowledge Graph Completioncited by 1
  11. 2025KG-CF: Knowledge Graph Completion with Context Filtering under the Guidance of Large Language Modelscited by 1
  12. 2021Temporal Knowledge Graph Reasoning Triggered by Memoriescited by 0
  13. 2024One Subgraph for All: Efficient Reasoning on Opening Subgraphs for Inductive Knowledge Graph Completioncited by 0
  14. 2024Ontological Relations from Word Embeddingscited by 0
  15. 2024Fast-and-Frugal Text-Graph Transformers are Effective Link Predictorscited by 0