# Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking

arXiv: https://arxiv.org/abs/2306.10453 · HTML: https://mlsanity.com/paper/2306.10453/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 18 Jun 2023.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [A Fair Comparison of Graph Neural Networks for Graph Classification](https://mlsanity.com/paper/1912.09893/) (2019)
- [Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning](https://mlsanity.com/paper/2010.16103/) (2020)
- [Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks](https://mlsanity.com/paper/2203.00199/) (2022)
- [Graph Neural Networks for Link Prediction with Subgraph Sketching](https://mlsanity.com/paper/2209.15486/) (2022)
- [Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction](https://mlsanity.com/paper/2206.04216/) (2022)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework](https://mlsanity.com/paper/2006.13365/) (2020)
- [Understanding Negative Sampling in Graph Representation Learning](https://mlsanity.com/paper/2005.09863/) (2020)
- [Neural Common Neighbor with Completion for Link Prediction](https://mlsanity.com/paper/2302.00890/) (2023)
- [Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time](https://mlsanity.com/paper/1711.07601/) (2017)
- [Evaluating Link Prediction Methods](https://mlsanity.com/paper/1505.04094/) (2015)
- [Pairwise Learning for Neural Link Prediction](https://mlsanity.com/paper/2112.02936/) (2021)
- [Link Prediction without Graph Neural Networks](https://mlsanity.com/paper/2305.13656/) (2023)

## Built on by (most cited first, 30 of 69)

- [Position: Graph Foundation Models are Already Here](https://mlsanity.com/paper/2402.02216/) (2024)
- [KAGNNs: Kolmogorov-Arnold Networks meet Graph Learning](https://mlsanity.com/paper/2406.18380/) (2024)
- [Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights](https://mlsanity.com/paper/2406.10727/) (2024)
- [Revisiting Link Prediction: A Data Perspective](https://mlsanity.com/paper/2310.00793/) (2023)
- [Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning](https://mlsanity.com/paper/2406.16321/) (2024)
- [Pure Message Passing Can Estimate Common Neighbor for Link Prediction](https://mlsanity.com/paper/2309.00976/) (2023)
- [Efficient Neural Common Neighbor for Temporal Graph Link Prediction](https://mlsanity.com/paper/2406.07926/) (2024)
- [LPFormer: An Adaptive Graph Transformer for Link Prediction](https://mlsanity.com/paper/2310.11009/) (2023)
- [A Topological Perspective on Demystifying GNN-Based Link Prediction Performance](https://mlsanity.com/paper/2310.04612/) (2023)
- [Masked Graph Autoencoder with Non-discrete Bandwidths](https://mlsanity.com/paper/2402.03814/) (2024)
- [Mixture of Link Predictors on Graphs](https://mlsanity.com/paper/2402.08583/) (2024)
- [On the Impact of Feature Heterophily on Link Prediction with Graph Neural Networks](https://mlsanity.com/paper/2409.17475/) (2024)
- [Breaking the Top-K Barrier: Advancing Top-K Ranking Metrics Optimization in Recommender Systems](https://mlsanity.com/paper/2508.05673/) (2025)
- [Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models](https://mlsanity.com/paper/2406.01899/) (2024)
- [TFMLinker: Universal Link Predictor by Graph In-Context Learning with Tabular Foundation Models](https://mlsanity.com/paper/2602.08592/) (2026)
- [A Comprehensive Benchmark on Spectral GNNs: The Impact on Efficiency, Memory, and Effectiveness](https://mlsanity.com/paper/2406.09675/) (2024)
- [Promoting Fairness in Link Prediction with Graph Enhancement](https://mlsanity.com/paper/2409.08658/) (2024)
- [Learning Scalable Structural Representations for Link Prediction with Bloom Signatures](https://mlsanity.com/paper/2312.16784/) (2023)
- [Reconsidering the Performance of GAE in Link Prediction](https://mlsanity.com/paper/2411.03845/) (2024)
- [BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactions](https://mlsanity.com/paper/2506.05766/) (2025)
- [Bridging Theory and Practice in Link Representation with Graph Neural Networks](https://mlsanity.com/paper/2506.24018/) (2025)
- [What Do Temporal Graph Learning Models Learn?](https://mlsanity.com/paper/2510.09416/) (2025)
- [GENIE: Watermarking Graph Neural Networks for Link Prediction](https://mlsanity.com/paper/2406.04805/) (2024)
- [Link Prediction with Untrained Message Passing Layers](https://mlsanity.com/paper/2406.16687/) (2024)
- [Can GNNs Learn Link Heuristics? A Concise Review and Evaluation of Link Prediction Methods](https://mlsanity.com/paper/2411.14711/) (2024)
- [VisGraphVar: A Benchmark Generator for Assessing Variability in Graph Analysis Using Large Vision-Language Models](https://mlsanity.com/paper/2411.14832/) (2024)
- [MiNT: Multi-Network Training for Transfer Learning on Temporal Graphs](https://mlsanity.com/paper/2406.10426/) (2024)
- [Open Your Eyes: Vision Enhances Message Passing Neural Networks in Link Prediction](https://mlsanity.com/paper/2505.08266/) (2025)
- [A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation](https://mlsanity.com/paper/2508.04645/) (2025)
- [CAT: Can Trust be Predicted with Context-Awareness in Dynamic Heterogeneous Networks?](https://mlsanity.com/paper/2512.11352/) (2025)

## Abstract (arXiv)

Link prediction attempts to predict whether an unseen edge exists based on only a portion of edges of a graph. A flurry of methods have been introduced in recent years that attempt to make use of graph neural networks (GNNs) for this task. Furthermore, new and diverse datasets have also been created to better evaluate the effectiveness of these new models. However, multiple pitfalls currently exist that hinder our ability to properly evaluate these new methods. These pitfalls mainly include: (1) Lower than actual performance on multiple baselines, (2) A lack of a unified data split and evaluation metric on some datasets, and (3) An unrealistic evaluation setting that uses easy negative samples. To overcome these challenges, we first conduct a fair comparison across prominent methods and datasets, utilizing the same dataset and hyperparameter search settings. We then create a more practical evaluation setting based on a Heuristic Related Sampling Technique (HeaRT), which samples hard negative samples via multiple heuristics. The new evaluation setting helps promote new challenges and opportunities in link prediction by aligning the evaluation with real-world situations. Our implementation and data are available at https://github.com/Juanhui28/HeaRT