# LPFormer: An Adaptive Graph Transformer for Link Prediction

arXiv: https://arxiv.org/abs/2310.11009 · HTML: https://mlsanity.com/paper/2310.11009/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [Rethinking Graph Transformers with Spectral Attention](https://mlsanity.com/paper/2106.03893/) (2021)
- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification](https://mlsanity.com/paper/2306.08385/) (2023)
- [Pure Transformers are Powerful Graph Learners](https://mlsanity.com/paper/2207.02505/) (2022)
- [Scaling Graph Neural Networks with Approximate PageRank](https://mlsanity.com/paper/2007.01570/) (2020)
- [Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning](https://mlsanity.com/paper/2010.16103/) (2020)
- [NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphs](https://mlsanity.com/paper/2206.04910/) (2022)
- [Graph Neural Networks for Link Prediction with Subgraph Sketching](https://mlsanity.com/paper/2209.15486/) (2022)
- [Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction](https://mlsanity.com/paper/2206.04216/) (2022)
- [Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking](https://mlsanity.com/paper/2306.10453/) (2023)
- [Neural Common Neighbor with Completion for Link Prediction](https://mlsanity.com/paper/2302.00890/) (2023)
- [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020)
- [Revisiting Link Prediction: A Data Perspective](https://mlsanity.com/paper/2310.00793/) (2023)
- [Leveraging Node Attributes for Incomplete Relational Data](https://mlsanity.com/paper/1706.04289/) (2017)
- [Structural Diversity and Homophily: A Study Across More than One Hundred Big Networks](https://mlsanity.com/paper/1602.07048/) (2016)
- [A Retrieve-and-Read Framework for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2212.09724/) (2022)
- [Structural transition in social networks: The role of homophily](https://mlsanity.com/paper/1808.05035/) (2018)
- [Localization in Seeded PageRank](https://mlsanity.com/paper/1509.00016/) (2015)

## Built on by (most cited first, 17 of 17)

- [Efficient Neural Common Neighbor for Temporal Graph Link Prediction](https://mlsanity.com/paper/2406.07926/) (2024)
- [Mixture of Link Predictors on Graphs](https://mlsanity.com/paper/2402.08583/) (2024)
- [LinkGPT: Teaching Large Language Models To Predict Missing Links](https://mlsanity.com/paper/2406.04640/) (2024)
- [TFMLinker: Universal Link Predictor by Graph In-Context Learning with Tabular Foundation Models](https://mlsanity.com/paper/2602.08592/) (2026)
- [Reconsidering the Performance of GAE in Link Prediction](https://mlsanity.com/paper/2411.03845/) (2024)
- [A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation](https://mlsanity.com/paper/2508.04645/) (2025)
- [ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs](https://mlsanity.com/paper/2507.19031/) (2025)
- [Towards Understanding Link Predictor Generalizability Under Distribution Shifts](https://mlsanity.com/paper/2406.08788/) (2024)
- [Subgraph Generation for Generalizing on Out-of-Distribution Links](https://mlsanity.com/paper/2507.11710/) (2025)
- [HeSRN: Representation Learning On Heterogeneous Graphs via Slot-Aware Retentive Network](https://mlsanity.com/paper/2510.09767/) (2025)
- [An Interdisciplinary and Cross-Task Review on Missing Data Imputation](https://mlsanity.com/paper/2511.01196/) (2025)
- [Generalized Graph Transformer Variational Autoencoder](https://mlsanity.com/paper/2512.00612/) (2025)
- [A Community-Enhanced Graph Representation Model for Link Prediction](https://mlsanity.com/paper/2512.21166/) (2025)
- [Plain Transformers are Surprisingly Powerful Link Predictors](https://mlsanity.com/paper/2602.01553/) (2026)
- [X-LogSMask: Expand Transformer for Graph-Structured Data](https://mlsanity.com/paper/2607.01553/) (2026)
- [A Survey on GNN-based Link Prediction: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2607.16198/) (2026)
- [Edge-Level Automorphism in GNNs: A Quantitative Framework and Effective Designs For Link Prediction](https://mlsanity.com/paper/2609.34729/) (2026)

## Abstract (arXiv)

Link prediction is a common task on graph-structured data that has seen applications in a variety of domains. Classically, hand-crafted heuristics were used for this task. Heuristic measures are chosen such that they correlate well with the underlying factors related to link formation. In recent years, a new class of methods has emerged that combines the advantages of message-passing neural networks (MPNN) and heuristics methods. These methods perform predictions by using the output of an MPNN in conjunction with a "pairwise encoding" that captures the relationship between nodes in the candidate link. They have been shown to achieve strong performance on numerous datasets. However, current pairwise encodings often contain a strong inductive bias, using the same underlying factors to classify all links. This limits the ability of existing methods to learn how to properly classify a variety of different links that may form from different factors. To address this limitation, we propose a new method, LPFormer, which attempts to adaptively learn the pairwise encodings for each link. LPFormer models the link factors via an attention module that learns the pairwise encoding that exists between nodes by modeling multiple factors integral to link prediction. Extensive experiments demonstrate that LPFormer can achieve SOTA performance on numerous datasets while maintaining efficiency. The code is available at The code is available at https://github.com/HarryShomer/LPFormer.