# Understanding the Performance of Knowledge Graph Embeddings in Drug Discovery

arXiv: https://arxiv.org/abs/2105.10488 · HTML: https://mlsanity.com/paper/2105.10488/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 27 of 27)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [A Fair Comparison of Graph Neural Networks for Graph Classification](https://mlsanity.com/paper/1912.09893/) (2019)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Network Representation Learning: A Survey](https://mlsanity.com/paper/1801.05852/) (2017)
- [Knowledge Graph Embedding for Link Prediction: A Comparative Analysis](https://mlsanity.com/paper/2002.00819/) (2020)
- [PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings](https://mlsanity.com/paper/2007.14175/) (2020)
- [Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework](https://mlsanity.com/paper/2006.13365/) (2020)
- [Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations](https://mlsanity.com/paper/1906.05017/) (2019)
- [A Review of Biomedical Datasets Relating to Drug Discovery: A Knowledge Graph Perspective](https://mlsanity.com/paper/2102.10062/) (2021)
- [On Model Stability as a Function of Random Seed](https://mlsanity.com/paper/1909.10447/) (2019)
- [OpenBioLink: A benchmarking framework for large-scale biomedical link prediction](https://mlsanity.com/paper/1912.04616/) (2019)
- [Utilising Graph Machine Learning within Drug Discovery and Development](https://mlsanity.com/paper/2012.05716/) (2020)
- [Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2103.10367/) (2021)
- [Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings](https://mlsanity.com/paper/2006.13774/) (2020)
- [On the Ambiguity of Rank-Based Evaluation of Entity Alignment or Link Prediction Methods](https://mlsanity.com/paper/2002.06914/) (2020)

## Built on by (most cited first, 18 of 18)

- [Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study](https://mlsanity.com/paper/2106.09700/) (2021)
- [How to Turn Your Knowledge Graph Embeddings into Generative Models](https://mlsanity.com/paper/2305.15944/) (2023)
- [Toward Degree Bias in Embedding-Based Knowledge Graph Completion](https://mlsanity.com/paper/2302.05044/) (2023)
- [LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs](https://mlsanity.com/paper/2402.11804/) (2024)
- [Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2112.06567/) (2021)
- [Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Tasks](https://mlsanity.com/paper/2305.19979/) (2023)
- [BioBLP: A Modular Framework for Learning on Multimodal Biomedical Knowledge Graphs](https://mlsanity.com/paper/2306.03606/) (2023)
- [Improving Molecule Generation and Drug Discovery with a Knowledge-enhanced Generative Model](https://mlsanity.com/paper/2402.08790/) (2024)
- [A Knowledge Graph-Enhanced Tensor Factorisation Model for Discovering Drug Targets](https://mlsanity.com/paper/2105.10578/) (2021)
- [Know2BIO: A Comprehensive Dual-View Benchmark for Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2310.03221/) (2023)
- [VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings](https://mlsanity.com/paper/2603.02435/) (2026)
- [A review of feature selection strategies utilizing graph data structures and knowledge graphs](https://mlsanity.com/paper/2406.14864/) (2024)
- [On the Theoretical Limitations of Embedding-based Link Prediction](https://mlsanity.com/paper/2506.22271/) (2025)
- [Neural Reasoning for Robust Instance Retrieval in SHOIQ](https://mlsanity.com/paper/2510.20457/) (2025)
- [Link Prediction or Perdition: the Seeds of Instability in Knowledge Graph Embeddings](https://mlsanity.com/paper/2606.03365/) (2026)
- [Causal Intervention for Measuring Confidence in Drug-Target Interaction Prediction](https://mlsanity.com/paper/2306.00041/) (2023)
- [Fast Knowledge Graph Completion using Graphics Processing Units](https://mlsanity.com/paper/2307.12059/) (2023)
- [Domain and Range Aware Synthetic Negatives Generation for Knowledge Graph Embedding Models](https://mlsanity.com/paper/2411.14858/) (2024)

## Abstract (arXiv)

Knowledge Graphs (KG) and associated Knowledge Graph Embedding (KGE) models have recently begun to be explored in the context of drug discovery and have the potential to assist in key challenges such as target identification. In the drug discovery domain, KGs can be employed as part of a process which can result in lab-based experiments being performed, or impact on other decisions, incurring significant time and financial costs and most importantly, ultimately influencing patient healthcare. For KGE models to have impact in this domain, a better understanding of not only of performance, but also the various factors which determine it, is required. In this study we investigate, over the course of many thousands of experiments, the predictive performance of five KGE models on two public drug discovery-oriented KGs. Our goal is not to focus on the best overall model or configuration, instead we take a deeper look at how performance can be affected by changes in the training setup, choice of hyperparameters, model parameter initialisation seed and different splits of the datasets. Our results highlight that these factors have significant impact on performance and can even affect the ranking of models. Indeed these factors should be reported along with model architectures to ensure complete reproducibility and fair comparisons of future work, and we argue this is critical for the acceptance of use, and impact of KGEs in a biomedical setting.