# Comprehensive Analysis of Negative Sampling in Knowledge Graph Representation Learning

arXiv: https://arxiv.org/abs/2206.10140 · HTML: https://mlsanity.com/paper/2206.10140/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019)
- [Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework](https://mlsanity.com/paper/2006.13365/) (2020)
- [Notes on Noise Contrastive Estimation and Negative Sampling](https://mlsanity.com/paper/1410.8251/) (2014)
- [Structure Aware Negative Sampling in Knowledge Graphs](https://mlsanity.com/paper/2009.11355/) (2020)
- [Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding](https://mlsanity.com/paper/2106.07250/) (2021)
- [KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning](https://mlsanity.com/paper/2205.02460/) (2022)

## Built on by (most cited first, 12 of 12)

- [Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Tasks](https://mlsanity.com/paper/2305.19979/) (2023)
- [Combating Bilateral Edge Noise for Robust Link Prediction](https://mlsanity.com/paper/2311.01196/) (2023)
- [A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning](https://mlsanity.com/paper/2308.07889/) (2023)
- [Subsampling for Knowledge Graph Embedding Explained](https://mlsanity.com/paper/2209.12801/) (2022)
- [Literal-Aware Knowledge Graph Embedding for Welding Quality Monitoring: A Bosch Case](https://mlsanity.com/paper/2308.01105/) (2023)
- [Predicting clinical outcomes from patient care pathways represented with temporal knowledge graphs](https://mlsanity.com/paper/2502.21138/) (2025)
- [Model-based Subsampling for Knowledge Graph Completion](https://mlsanity.com/paper/2309.09296/) (2023)
- [CL4KGE: A Curriculum Learning Method for Knowledge Graph Embedding](https://mlsanity.com/paper/2408.14840/) (2024)
- [Capturing and Anticipating User Intents in Data Analytics via Knowledge Graphs](https://mlsanity.com/paper/2411.01023/) (2024)
- [Domain and Range Aware Synthetic Negatives Generation for Knowledge Graph Embedding Models](https://mlsanity.com/paper/2411.14858/) (2024)
- [Towards Pattern-aware Data Augmentation for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2501.00252/) (2024)
- [ImbalancE: Inference-Time Latent Search Against Degree Imbalance in Link Prediction](https://mlsanity.com/paper/2609.36996/) (2026)

## Abstract (arXiv)

Negative sampling (NS) loss plays an important role in learning knowledge graph embedding (KGE) to handle a huge number of entities. However, the performance of KGE degrades without hyperparameters such as the margin term and number of negative samples in NS loss being appropriately selected. Currently, empirical hyperparameter tuning addresses this problem at the cost of computational time. To solve this problem, we theoretically analyzed NS loss to assist hyperparameter tuning and understand the better use of the NS loss in KGE learning. Our theoretical analysis showed that scoring methods with restricted value ranges, such as TransE and RotatE, require appropriate adjustment of the margin term or the number of negative samples different from those without restricted value ranges, such as RESCAL, ComplEx, and DistMult. We also propose subsampling methods specialized for the NS loss in KGE studied from a theoretical aspect. Our empirical analysis on the FB15k-237, WN18RR, and YAGO3-10 datasets showed that the results of actually trained models agree with our theoretical findings.