# Friendly Neighbors: Contextualized Sequence-to-Sequence Link Prediction

arXiv: https://arxiv.org/abs/2305.13059 · HTML: https://mlsanity.com/paper/2305.13059/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 18 of 18)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs](https://mlsanity.com/paper/2103.09430/) (2021)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [PyTorch-BigGraph: A Large-scale Graph Embedding System](https://mlsanity.com/paper/1903.12287/) (2019)
- [SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models](https://mlsanity.com/paper/2203.02167/) (2022)
- [Sequence-to-Sequence Knowledge Graph Completion and Question Answering](https://mlsanity.com/paper/2203.10321/) (2022)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [DGL-KE: Training Knowledge Graph Embeddings at Scale](https://mlsanity.com/paper/2004.08532/) (2020)
- [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020)
- [GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding](https://mlsanity.com/paper/1903.00757/) (2019)
- [MLMLM: Link Prediction with Mean Likelihood Masked Language Model](https://mlsanity.com/paper/2009.07058/) (2020)
- [BESS: Balanced Entity Sampling and Sharing for Large-Scale Knowledge Graph Completion](https://mlsanity.com/paper/2211.12281/) (2022)
- [Start Small, Think Big: On Hyperparameter Optimization for Large-Scale Knowledge Graph Embeddings](https://mlsanity.com/paper/2207.04979/) (2022)

## Built on by (most cited first, 9 of 9)

- [LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models](https://mlsanity.com/paper/2503.03313/) (2025)
- [Deja vu: Contrastive Historical Modeling with Prefix-tuning for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2404.00051/) (2024)
- [Enhancing Knowledge Graph Completion with Entity Neighborhood and Relation Context](https://mlsanity.com/paper/2503.23205/) (2025)
- [A Benchmark for Semi-Inductive Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2310.11917/) (2023)
- [Better Together: Enhancing Generative Knowledge Graph Completion with Language Models and Neighborhood Information](https://mlsanity.com/paper/2311.01326/) (2023)
- [In-Context Learning with Topological Information for Knowledge Graph Completion](https://mlsanity.com/paper/2412.08742/) (2024)
- [KG-TRICK: Unifying Textual and Relational Information Completion of Knowledge for Multilingual Knowledge Graphs](https://mlsanity.com/paper/2501.03560/) (2025)
- [Autoregressive Models for Knowledge Graph Generation](https://mlsanity.com/paper/2602.06707/) (2026)
- [Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2605.18211/) (2026)

## Abstract (arXiv)

We propose KGT5-context, a simple sequence-to-sequence model for link prediction (LP) in knowledge graphs (KG). Our work expands on KGT5, a recent LP model that exploits textual features of the KG, has small model size, and is scalable. To reach good predictive performance, however, KGT5 relies on an ensemble with a knowledge graph embedding model, which itself is excessively large and costly to use. In this short paper, we show empirically that adding contextual information - i.e., information about the direct neighborhood of the query entity - alleviates the need for a separate KGE model to obtain good performance. The resulting KGT5-context model is simple, reduces model size significantly, and obtains state-of-the-art performance in our experimental study.